tag:blogger.com,1999:blog-296106852024-03-11T13:09:55.830+02:00M&E BlogThis blog is intended as a home to some musings about M&E, the challenges that I face as an evaluator and the work that I do in the field of M&E.Often times what I post here is in response to a particularly thought-provoking conversation or piece of reading. This is my space to "Pause and Reflect".Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.comBlogger150125tag:blogger.com,1999:blog-29610685.post-2049122742506601072024-03-05T12:03:00.001+02:002024-03-05T12:03:23.234+02:00Applying systems theoretical concepts to understand sustainability of education intervention outcomes<p> </p><h1 _ngcontent-dje-c173="" class="ng-star-inserted" dir="auto" style="background-color: #fafafa; box-sizing: inherit; font-family: "", blinkmacsystemfont, -apple-system, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: calc(-1.65488px + 1.47566rem + 0.172383vw); hyphens: auto; margin-block: 0px 0.6em; margin: 0px; padding: 0px; word-break: break-word;"><span style="font-size: 16px; white-space-collapse: preserve;">This Master’s dissertation addresses the research question: To what extent can the systems concept ‘extended dynamic sustainability’ be used to explain why some results of a donor-funded education development intervention were sustained ten years after its conclusion? </span></h1><div><span style="font-size: 16px; white-space-collapse: preserve;"><br /></span></div><div><span style="font-size: 16px; white-space-collapse: preserve;"><br /></span></div><div><span style="background-color: #fafafa; font-family: "", blinkmacsystemfont, -apple-system, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; white-space-collapse: preserve;">To address that question, the researcher identified a specific case to explore with systems thinking: an ex-post evaluation conducted in 2016, and commissioned by an international donor, the United States Agency for International Development (USAID). That ex-post evaluation confirmed that an education development intervention, the Kimberly Thusanang Programme (KTP) implemented between 1998 and 2006, resulted in sustained outcomes, which were directly linked to the KTP’s goal of improving school governance in the Francis Baard education district the Northern Cape. </span></div><div><span style="background-color: #fafafa; font-family: "", blinkmacsystemfont, -apple-system, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; white-space-collapse: preserve;">
The Master’s research builds on the ex-post evaluation’s analysis. Using qualitative data analysis, the researcher identified the types of sustainability found in the ex-post evaluation data set. Then, by applying Stockmann’s (1993a) ‘extended dynamic sustainability’ concept, the Master’s research found that the KTP intervention and some of its benefits were dynamically sustained through the general causal sustainability mechanisms of problem-solving, modelling and multiplication. </span></div><div><span style="background-color: #fafafa; font-family: "", blinkmacsystemfont, -apple-system, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; white-space-collapse: preserve;">
These findings are likely useful to research intervention sustainability, to design sustainable development interventions, and to evaluate intervention success. Further exploration of these general sustainability mechanisms needs to be conducted to determine if these mechanisms are generalisable to other development interventions and their sustained outcomes.</span></div><div><span style="background-color: #fafafa; font-family: "", blinkmacsystemfont, -apple-system, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; white-space-collapse: preserve;"><br /></span></div><div><span style="background-color: #fafafa; font-family: "", blinkmacsystemfont, -apple-system, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; white-space-collapse: preserve;"><a href="https://ujcontent.uj.ac.za/esploro/outputs/9922809207691">Available on the University of Johannesburg Content Repository</a></span></div>Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-91342885404490944732024-03-05T11:56:00.001+02:002024-03-05T11:56:15.530+02:00EvalEdge Podcast about Storytelling in Evaluation<p> </p><p><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEim4aLbfqx1pLJ1jZ4lNt0KNNuBaNr7PJHExnWCUDEvoCA6AJT2z8OkUunb10nHnCucO1x36TP-YsUm2-kOR87ZMP1m6WmRuPK5-vMjAq2uqCrviiQKENVaOSMj5ZRg9JoMtU3ad5rExq0TbHxp2inz02_-kjFB6Ea-udjOGp7-GdeV-i-_BFYH/s614/Screenshot%202024-03-05%20113916.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="EVALEDGE PODCATS LOGO" border="0" data-original-height="581" data-original-width="614" height="189" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEim4aLbfqx1pLJ1jZ4lNt0KNNuBaNr7PJHExnWCUDEvoCA6AJT2z8OkUunb10nHnCucO1x36TP-YsUm2-kOR87ZMP1m6WmRuPK5-vMjAq2uqCrviiQKENVaOSMj5ZRg9JoMtU3ad5rExq0TbHxp2inz02_-kjFB6Ea-udjOGp7-GdeV-i-_BFYH/w200-h189/Screenshot%202024-03-05%20113916.png" width="200" /></a></div><p><a href="https://soundcloud.com/europeanevaluationsociety/ees-podcast-episode-16?si=b54f46dbd6174b55b257360cdd31d46f&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing" style="font-family: Inter, sans-serif; font-size: 14px;">In this episode of EvalEdge</a><span style="background-color: white; color: #333333; font-family: Inter, sans-serif; font-size: 14px;">, Asgar Bhikoo and I talk about Storytelling in Evaluation Practice. This episode focused on exploring current lessons related to the use of story-telling as innovation in evaluation practice in Africa. For more information, check out Digital Stories for Impact and Social Impact Storytelling: Using Impact Data to Drive Change.</span></p><p><span style="background-color: white; color: #333333; font-family: Inter, sans-serif; font-size: 14px;">Here is the <a href="https://dgmt.co.za/working-rigorously-with-stories-introducing-the-impact-story-tool/ " target="_blank">impact story tool </a>referenced in the podcast, also here on the <a href="https://monitoring-toolkits.civicus.org/toolkit/impact-story/" target="_blank">Civicus repository</a>. </span></p><p><span style="background-color: white; color: #333333; font-family: Inter, sans-serif; font-size: 14px;"><br /></span></p><p><br /></p>Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-54112025920602017432019-07-29T10:19:00.001+02:002019-07-29T10:36:12.096+02:00There are alternatives to Experimental and Quasi-Experimental Impact Evaluation Methods. <br />
<div style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/67427/design-method-impact-eval.pdf" target="_blank"><img border="0" data-original-height="452" data-original-width="494" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTAe3ZEltxAU2heUaM2L7vrUz-Z4vOciziwL5RyQb5qEyMYATqr8q5w3frflZxv8zMxkKuot0JAJmmOZVZcICEu55RbnpPUgRUqBNqttQKHx3JcBLt9y1QyXpWeRbW5l2TVSAu/s320/dfid.png" width="320" /></a></div>
<div style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<span style="background: white; color: #14171a; font-family: inherit;">Some of my clients are really interested in measuring their impact. RCTs and other quasi-experiments are first on their list of suggested designs. But </span><span style="background-color: white; font-family: inherit; white-space: pre-wrap;"><span data-offset-key="cvdkr-1-0"><span data-text="true">our repertoire of IE designs and </span></span></span><span data-offset-key="cvdkr-1-0" style="font-family: inherit; white-space: pre-wrap;"><span data-text="true">methods have grown</span></span><span style="font-family: inherit; white-space: pre-wrap;">. </span></div>
<br />
<span style="font-family: inherit; white-space: pre-wrap;"><br /></span>
<span style="font-family: inherit; white-space: pre-wrap;"> </span><br />
<span style="white-space: pre-wrap;"><span style="font-family: inherit;"><br /></span></span>
<span style="white-space: pre-wrap;"><span style="font-family: inherit;"><a href="https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/67427/design-method-impact-eval.pdf">This DfID working paper</a> says: </span></span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<blockquote class="tr_bq" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; line-height: normal; margin-bottom: 0.0001pt;">
<span style="font-family: inherit;"><span style="color: #14171a;">Most development interventions are ‘contributory causes’. They ‘work’ as
part of a causal package in combination with other ‘helping factors’ such as
stakeholder behaviour, related programmes and policies, institutional
capacities, cultural factors or socio-economic trends. Designs and methods for
IE need to be able to unpick these causal packages. </span><br /><span style="color: black; mso-color-alt: windowtext;">Demonstrating
that interventions cause development effects depends on theories and rules of
causal inference that can support causal claims. Some of the most potentially
useful approaches to causal inference are not generally known or applied in the
evaluation of international development and aid. Multiple causality and
configurations; and theory-based evaluation that can analyse causal mechanisms
are particularly weak. There is greater understanding of counterfactual logics,
the approach to causal inference that underpins experimental approaches to IE. </span></span></blockquote>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Methods that I am currently interested in include </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><a href="https://www.betterevaluation.org/en/plan/approach/QUIP">Qualitative Impact Assessment Protoc</a>ol </span><br />
<blockquote class="tr_bq">
<span style="font-family: inherit;">The QuIP gathers evidence of a project’s impact through <strong>narrative causal statements</strong> collected directly from intended project beneficiaries. Respondents are asked to talk about the main changes in their lives over a pre-defined recall period and prompted to share what they perceive to be the main <strong>drivers </strong>of these changes, and to whom or what they <strong>attribute </strong>any change - which may well be from multiple sources.<br />Typically, a QuIP study involves 24 semi-structured <a href="http://betterevaluation.org/evaluation-options/interviews" style="color: #1c88dd; text-decoration-line: none;">interviews</a> and four <a href="https://betterevaluation.org/en/evaluation-options/FocusGroups" style="color: #1c88dd; text-decoration-line: none;">focus groups</a>, conducted in the native language by highly-skilled, local researchers. However, this number is not fixed and will depend on the sampling approach used. The research team conducting interviews are independent and <strong>blindfolded </strong>where appropriate; they are not aware who has commissioned the research or which project is being assessed. This helps to mitigate and reduce pro-project and confirmation bias, as well as enable a broader and more open discussion with respondents about all outcomes and drivers of change.</span></blockquote>
<span style="font-family: inherit;"><a href="https://www.betterevaluation.org/en/evaluation-options/qualitative_comparative_analysis">Qualitative Comparative Analysis</a> </span><br />
<div class="summary">
<div class="field field-name-field-summary field-type-text-long field-label-hidden">
<div class="field-items">
<div class="field-item even">
<blockquote class="tr_bq" style="margin-bottom: 1.5em; margin-top: 1.5em;">
<span style="font-family: inherit;">Qualitative Comparative Analysis (QCA) is a means of analysing the causal contribution of different conditions (e.g. aspects of an intervention and the wider context) to an outcome of interest. QCA starts with the documentation of the different configurations of conditions associated with each case of an observed outcome. These are then subject to a minimisation procedure that identifies the simplest set of conditions that can account all the observed outcomes, as well as their absence. The results are typically expressed in statements expressed in ordinary language or as Boolean algebra. QCA is able to use relatively small and simple data sets. There is no requirement to have enough cases to achieve statistical significance, although ideally there should be enough cases to potentially exhibit all the possible configurations. </span></blockquote>
</div>
</div>
</div>
</div>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-21589453372536501602019-07-19T11:48:00.000+02:002019-07-19T11:48:03.851+02:00Picture this- ComplexityThis handy poster made by Johanna Boehnert explains 16 terms that often pop up in thining about complex systems. It's a bit like a gateway drug to reading more on Complex Systems.<br />
<br />
If found it in a tweet by @Heinomatti which refers to the <a href="https://www.cecan.ac.uk/index.php/news/visual-representation-of-complexity">website of CECAN</a> .<br />
But Better Evaluation also h<a href="https://www.betterevaluation.org/en/resources/visual-representation-complexity-definitions-examples-and-learning-points">as a really nice summary of it.</a><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<img border="0" data-original-height="857" data-original-width="611" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNFO325A318I-mH22ELrQNDiLVxgPn138TQTi8btjLc2rPBSGNiYP92Jq19xFWaJsbWoVzHKQoHnWwkrJIGavL3_rX8n0-oG25QZrNLwjPRkaFYPve9v4A4Wdhdd6XwwPjKxaA/s640/boehnert.png" width="456" /><a href="https://www.cecan.ac.uk/sites/default/files/2018-06/The%20Visual%20Communication%20of%20Complexity%20-%20May2018%20-%20EcoLabs.pdf">https://www.cecan.ac.uk/sites/default/files/2018-06/The%20Visual%20Communication%20of%20Complexity%20-%20May2018%20-%20EcoLabs.pdf</a></div>
<br />
<br />
<br />
<br />
<br />
<br />Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-25428715178962092172019-07-17T09:33:00.001+02:002019-07-17T16:06:24.327+02:00Systems Science and Complexity Science - related but not the sameI'm studying again and for that, I'm reading. A lot. I'm reading about systems thinking and factors that support sustained outcomes of development interventions. Often I stumble on things that make me go: "Ooh - I should remember this next time I do ABC" So this blog is being revived a bit to help keep track of these random thoughts.<br />
<br />
I read about the history of systems thinking and complexity science and how both fields have similar challenges. Two great resources:<br />
<br />
<div class="MsoNormal">
Midgley and Richardson comparison of paradigms in the <a href="https://www.researchgate.net/publication/43527349_Systems_thinking_for_community_involvement_in_policy_analysis"><b>Systems Field and the Complexity </b></a>Field. </div>
<div class="MsoNormal">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPiTK3zPM_OF7uK9BBQrkz7mL3MOoP5dOPOJTznAaL3sOPJ98OACDrDaQktaeYUTveyO95VQr6qV_TCZhIR7JF7ciK7H3LAjjndoo2m2tSrVasJSq-toT15S4TNwKVln8kHrwZ/s1600/midgley+richardson.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="448" data-original-width="794" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPiTK3zPM_OF7uK9BBQrkz7mL3MOoP5dOPOJTznAaL3sOPJ98OACDrDaQktaeYUTveyO95VQr6qV_TCZhIR7JF7ciK7H3LAjjndoo2m2tSrVasJSq-toT15S4TNwKVln8kHrwZ/s400/midgley+richardson.png" width="400" /></a></div>
<br />
<br />
<div class="MsoNormal">
<a href="https://www.researchgate.net/publication/316540205_Four_domains_of_complexity"><b>Midgley's reflection </b>o</a>n the history of paradigm wars between systems scientists amongst themselves, and complexity scientists amongst themselves. He says: </div>
<div class="MsoNormal">
<br /></div>
<blockquote class="tr_bq">
<span style="font-family: "calibri" , sans-serif; font-size: 13.5pt; line-height: 107%;">Systems scientists were embroiled in a paradigm war, which threatened to
fragment the systems research community. This is relevant... because the same paradigms are evident in the
complexity science community, and therefore it potentially faces the same risk
of fragmentation.</span></blockquote>
<div class="MsoNormal">
<br /></div>
My interest in reading about the relationship between systems science and complexity science got sparked when I looked for examples of emergence, feedback and self-organization in my data and couldn't figure out what that would look like. A colleague suggested that while the concept "feedback" definitely occurs in multiple branches of the systems field (oh and there are<b> <a href="https://thesystemsthinker.com/%EF%BB%BFall-methods-are-wrong-some-methods-are-useful/">so very very many</a></b>), that the concepts "emergence" and "self-organization" are from complexity science.<br />
<br />
One may argue that it probably doesn't matter into which categories these concepts fall, but actually, it does. Because the ontological and epistemological assumptions that underly these paradigms may or may not be similar and should be questioned.<br />
<br />
So to get my thinking about the concepts straight, I need to get my thinking about the paradigms straight. Its a work in progress....Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-87015580650257102852017-01-27T11:58:00.000+02:002017-01-27T11:58:57.831+02:00Can you tell me "What works in..."Although we reportedly now live in a post-evidence era, I still choose to cling to the minority view that programmes should be informed by research about what works. But where do you find the evidence?<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
About two years ago I attended a training course presented by Phil Davies from <a href="http://www.3ieimpact.org/">3ie</a>. He had many interesting insights to share, but today I was reminded of this excellent list of synthesised evidence that he shared.<br />
<br />
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="font-family: Arial;">•</span><a href="http://www.3ieimpact.org/en/evidence/impact-evaluations/">3ie Impact Evaluations Database</a></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="text-indent: -0.25in;"><span style="font-family: Arial; mso-special-format: bullet;">•</span></span><span style="text-indent: -0.25in;"><a href="http://www.bestevidence.org/">Best Evidence Encyclopedia</a></span></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="font-family: Arial; mso-special-format: bullet;">•</span><a href="http://www.cochrane.org/">Cochrane Collaboratio</a>n</div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
•<a href="http://www.campbellcollaboration.org/">Campbell Collaboration</a> </div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
•<a href="http://coexgov.securesites.net/index.php?keyword=a432fbc34d71c7">Coalition for Evidence-Based Policy </a></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="font-family: Times; mso-special-format: bullet;">•</span><a href="http://www.evidence.nhs.uk/">NHS Evidence</a></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="font-family: Times; mso-special-format: bullet;">•</span><a href="http://www.guidelines.gov/">National Guidelines Clearinghouse</a> (USA)</div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="font-family: Times; mso-special-format: bullet;">•</span><a href="http://www.crd.york.ac.uk/prospero/">Prospero: International Prospective Register of Systematic Reviews</a></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
•<a href="http://evidencebasedprograms.org/">Social Programs That Work</a></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;"><span style="font-family: Arial; mso-special-format: bullet;">•</span></span><span style="text-indent: -0.25in;"><a href="http://www.3ieimpact.org/en/evidence/systematic-reviews/">3ie Systematic Reviews Database</a></span></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">•</span><span style="text-indent: -0.25in;"><a href="http://www.nice.org.uk/">National </a></span><span style="text-indent: -0.25in;"><a href="http://www.nice.org.uk/">Institute for Health and Clinical Excellence</a></span></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<span style="text-indent: -0.25in;"><span style="font-family: Times; mso-special-format: bullet;">•</span></span><span style="text-indent: -0.25in;"><a href="http://www.scie.org.uk/">Social Care Institute for Excellence</a></span></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<br /></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
One of my recent favourite systematic reviews, conducted by 3ie is <a href="http://www.3ieimpact.org/en/publications/systematic-review-publications/3ie-systematic-review-education-effectivenes-srs7/">The impact of education programmes on learning and school participation in low- and middle-income countries by Snilstveit et al </a>. It has evidence about supplementary education programmes, feeding programmes, ICT in education programmes and a wide range of others. </div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDkEpd37RhEeG4hCB-06u5QuYOvx0bYe_SQ5O3lDpa7a94cs2EVkZWEySbYRNqti9iLENdWd75Z1y4Rd-JLGroQo-oVJiw-EXya1KBRFlplvuEiYR1f4WYFfWiGZHpr-VLLZ2R/s1600/pic.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDkEpd37RhEeG4hCB-06u5QuYOvx0bYe_SQ5O3lDpa7a94cs2EVkZWEySbYRNqti9iLENdWd75Z1y4Rd-JLGroQo-oVJiw-EXya1KBRFlplvuEiYR1f4WYFfWiGZHpr-VLLZ2R/s400/pic.jpg" width="381" /></a></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<br /></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<br /></div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
Happy Reading! </div>
<div style="direction: ltr; margin-bottom: 2pt; margin-left: 0.25in; margin-top: 0pt; text-indent: -0.25in; unicode-bidi: embed; vertical-align: baseline;">
<br /></div>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-45379497328008080162016-07-13T15:20:00.000+02:002016-07-13T15:21:35.682+02:00Evaluative Rubrics - Helping you to make sense of your evaluation data<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7PgfxuuRFUruPdrRslRxeuRg_GBqYPOhZwLDGOZ3l_F28_w13Wu5ba8jC1PIs5NjPVJNypQC54n9LaBjN_4suA1U80Hpr-i7Q-B2rvbeEE6OGMdA4FFc8xxHbGYXzfGVZDtKh/s1600/Rorschach_blot_01.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="209" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7PgfxuuRFUruPdrRslRxeuRg_GBqYPOhZwLDGOZ3l_F28_w13Wu5ba8jC1PIs5NjPVJNypQC54n9LaBjN_4suA1U80Hpr-i7Q-B2rvbeEE6OGMdA4FFc8xxHbGYXzfGVZDtKh/s320/Rorschach_blot_01.jpg" width="320" /></a></div>
Three times in one week I've now found myself explaining the use of evaluation rubrics to potential evaluation users. I usually start with an example like this, that people can relate to:<br />
<blockquote class="tr_bq">
When your high school creative writing paper was graded, your teacher most likely gave you an evaluative rubric which specified that you do well if you 1) used good grammar and spelling, 2) structured your arguments well, and 3) found an innovative and interesting angle on your topic. In essence, this rubric helped you to know what is "good" and what is "not good".</blockquote>
In an evaluation, a rubric does exactly the same. What is a good outcome if you judge a post- school science and maths bridging programme? How does the outcomes of "being employed" or "busy with a third year B Sc. Degree at university" compare to an outcome like "being a self-employed university drop-out with three registered patents" or to an outcome like "being unemployed and not sure what to do about the future". A rubric can help you to figure this out.<br />
<br />
E. Jane Davidson has some excellent resources on rubrics <a href="http://www.managingforimpact.org/resource/evaluation-rubrics-e-jane-davidson">here</a> and <a href="http://aea365.org/blog/e-jane-davidson-on-evaluative-rubrics/">here.</a> If you need a rubric on evaluating value for investment, Julian King has a good resource <a href="http://www.julianking.co.nz/wp-content/uploads/2014/09/160527-VFI-jk8-web.pdf">here</a>. And of course, there is the usual great content on better evaluation <a href="http://betterevaluation.org/evaluation-options/rubrics">here</a>.<br />
<br />
I love how Jane describes why we need evaluation rubrics:<br />
<blockquote class="tr_bq">
Evaluative rubrics make transparent how quality and value are defined and applied. I sometimes refer to rubrics as the antidote to both ‘Rorschach inkblot’ (“You work it out”) and ‘divine judgment’ (“I looked upon it and saw that it was good”)-type <a href="http://aea365.org/blog/e-jane-davidson-on-evaluative-rubrics/">evaluations</a>.</blockquote>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-74543158515606475602016-02-29T12:58:00.000+02:002016-02-29T12:58:31.828+02:00Writing Summaries for Evaluation Reports<span style="font-size: 28pt;"></span><br />
<div class="separator" style="clear: both;">
Last year I attended a course on "Using Evidence for Policy and Practice" presented by Philip Davies from the International Initiative for Impact Evaluation [<a href="http://www.3ieimpact.org/">3ie</a>]. I found his guidelines for what should go into the 1:3:25 summaries most helpful. Here they are:<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-atsYfFbIkU2Nc_AJpAdG4jhhZrkNPVULIakiOAtBouZxqO5uYXg1xQ1Nqbub8r9DSOW28oQBD0RR0NrrLNQ0i-5OYXR75SeXBGBbBFWat4PmCIQqvrWcjOdGJtuy_DsuZNE4/s1600/1_3_25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-atsYfFbIkU2Nc_AJpAdG4jhhZrkNPVULIakiOAtBouZxqO5uYXg1xQ1Nqbub8r9DSOW28oQBD0RR0NrrLNQ0i-5OYXR75SeXBGBbBFWat4PmCIQqvrWcjOdGJtuy_DsuZNE4/s640/1_3_25.png" width="272" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
The full course material is available on the website of the <a href="http://www.africaevidencenetwork.org/">African Evidence Network's</a> Website. <a href="https://www.google.co.za/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiFi_iS55zLAhVDVBQKHa7QAlwQFggaMAA&url=http%3A%2F%2Fwww.africaevidencenetwork.org%2Fwp-content%2Fuploads%2F2014%2F06%2F7.-Using-Evidence-for-Policy-and-Practice.pptx&usg=AFQjCNHwA4VP-zgLO1pw4UAHvEEmKbWyAw&sig2=AiwAAqQbAKF9JTFMypxDTQ">Here</a>. </div>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-61145055348563325922015-10-07T12:03:00.002+02:002015-10-07T12:03:49.539+02:00What I'm up to at the 2015 SAMEA Conference<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">The <a href="http://www.sameaconference.co.za/">SAMEA conference</a> is happening from 12 to 16 October and I'm looking forward to it. </span><br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span>
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjN626bs59RbPLKe6G-rcQygULM8xbWHRZyfdJKuhFmgMkN0Wp7OUeCuxzbaSM89C1zRZ4N7OotIlG7Yd97XLNMysb4wxr8Ee1GrgMc5obrfzxgQQaJCKsLfe8_Xvog83YPbzw/s1600/SAMEAcon.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><img alt="5thSAMEA Conference Logo" border="0" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjN626bs59RbPLKe6G-rcQygULM8xbWHRZyfdJKuhFmgMkN0Wp7OUeCuxzbaSM89C1zRZ4N7OotIlG7Yd97XLNMysb4wxr8Ee1GrgMc5obrfzxgQQaJCKsLfe8_Xvog83YPbzw/s320/SAMEAcon.png" title="5thSAMEA Conference Logo" width="320" /></span></a><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">Since January, I've had to temporarily downscale my professional involvement in the M&E and Educational networks and I had to neglect this little blog a bit because of a second long term development project I took on in January 2015. The project has lovely brown eyes, an infectious laugh and goes by the name of Clarissa. I'm happy to report that no major clashes with the first development project, (Named Ruan) has so far occurred, but its been a bit of an adjustment to balance work, and volunteering, and life in general. </span><br />
<br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">So what am I up to at the conference?</span><br />
<span style="background-color: white;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">I'll be tweeting from </span></span><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">@benitaw if you are interested in my perspective of the conference.</span><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"> I will also attend an IOCE stand at the conference, aiming to promote the <a href="http://www.vopetoolkit.ioce.net/en/page/about-toolkit">VOPE Institutional Capacity Toolkit</a> which my consultancy developed under the <a href="http://www.mymande.org/evalpartners">EvalPartners</a> leadership of Jennifer Bisgard, Patricia Rogers, Jim Rugh, and Matt Galen. This is an online toolkit full of helpful resources aimed to equip VOPEs </span><span style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">(Voluntary Organisations for Professional Evaluation)</span><span style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;"> to become more accountable and more active. </span><br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">Then, I'll be teaming up with <a href="http://www.clear-aa.co.za/author/carawaller/">Cara Waller</a> (from CLEAR) and <a href="http://www.otherwise.co.za/">Donna Podems</a> (from OtherWise) in a session for African VOPEs on <b>Friday 16th October</b>. This is a ‘world-café’ style event, <b>from 10 –11:30am,</b> to be held as a joint ‘Made in Africa’ and ‘Discussing the Professionalisation of Evaluation and Evaluators’ stream session. <span style="background-color: white;"> The aim of the session is to provide a space for those involved with VOPEs in the region (and those with an interest in strengthening African VOPEs) to come together to discuss current topics around building quality supply and generating demand for evaluation in contextually-specific ways. </span><span style="background-color: white;">So please come and chat all things VOPE on the day!</span></span><br />
<span style="background-color: white;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span></span>
<span style="background-color: white;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">Good luck to my colleague Fazeela Hoosen and the rest of the SAMEA board on hosting this year's conference with the <a href="http://www.thepresidency-dpme.gov.za/Pages/default.aspx">DPME</a> and the <a href="http://www.psc.gov.za/">PSC</a>. I know (and boy.... do I know) it is very hard work. So thanks in advance for all of the hours you are putting in, to make this event happen. </span></span>Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-31509913428213988222014-11-20T16:25:00.000+02:002014-11-20T16:25:32.930+02:00My consultancy is one year old so I celebrated by making my first Infographic!<pre class="auto-highlight-on-focus" id="output-tab-embed-code-html" style="background-color: whitesmoke; border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; border-top-left-radius: 4px; border-top-right-radius: 4px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; color: #333333; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-size: 12px; font-stretch: inherit; line-height: 1.428571429; margin-bottom: 25px; padding: 9px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><pre class="auto-highlight-on-focus" id="output-tab-embed-code-html" style="border-bottom-left-radius: 4px; border-bottom-right-radius: 4px; border-top-left-radius: 4px; border-top-right-radius: 4px; border: 1px solid rgb(204, 204, 204); box-sizing: border-box; font-family: Menlo, Monaco, Consolas, 'Courier New', monospace; font-stretch: inherit; line-height: 1.428571429; margin-bottom: 25px; padding: 9px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><iframe frameborder="0" height="1412" scrolling="no" src="https://magic.piktochart.com/embed/3485195-annual-reflection" style="overflow-y: hidden;" width="465"></iframe></pre>
</pre>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com1tag:blogger.com,1999:blog-29610685.post-74871601981300733742014-10-16T13:20:00.001+02:002014-10-16T14:17:01.133+02:00True Confessions of an Economic Evaluation PhobicYou know how the forces at work in the universe sometimes conspire and confronts you with a persistent nudge... over an over again? Well this week's nudge was "You know nothing about economic evaluation... do something about it - Other than ignoring it".<br />
<br />
Words like "cost-benefit analysis, cost-efficiency analysis, cost-utility analysis"... actually anything with the word "cost" or "expenditure" in it... makes me nervous. So my usual strategy is to ignore the "<a href="http://www.oecd.org/development/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm" target="_blank">Efficiency" criterion suggested by the OECD DAC,</a> or I start fidgeting around for the contact details of one of my economist friends, and pass the job along. I have even managed to be part of a team doing a <a href="http://www.unicef.org/southafrica/SAF_resources_pets.pdf" target="_blank">Public Expenditure Tracking Survey</a> without touching the "Expenditure" side of the data.<br />
<br />
But then I found these two resources that helped me to start to make a little bit more sense of it all. They are:<br />
<br />
The South African Department of Planning Monitoring and Evaluation's <a href="http://www.google.co.za/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CCMQFjAA&url=http%3A%2F%2Fwww.thepresidency-dpme.gov.za%2Fkeyfocusareas%2FevaluationsSite%2FEvaluations%2FGL%25202%25202%25205%2520Economic%2520Guideline%2520%252014%252003%252020.pdf&ei=2KU_VK21OMOc7gbQqIC4Dg&usg=AFQjCNHCVHj1aUQdfdzezxcYYXgJPP03wA&sig2=J1XPPw59jZHpjDpUh77LWg&bvm=bv.77648437,d.ZGU" target="_blank">Guideline on Economic Evaluation </a>At least it starts to explain the very many different kinds of economic evaluation you should consider if you work within the context of South Africa's <a href="http://www.thepresidency-dpme.gov.za/publications/Reports%20and%20Other%20Information%20Products/Evaluation_Policy_Framework%20approved%2011%2011%2023.pdf" target="_blank">National Evaluation Policy Framework</a>. <br />
<br />
<br />
And then <a href="http://www.julianking.co.nz/downloads/" target="_blank">this. </a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.julianking.co.nz/downloads/" imageanchor="1" style="margin-left: 1em; margin-right: 1em;" target="_blank"><img alt="http://www.julianking.co.nz/downloads/" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgn-pKEZF98Fd9_SBOUQAdBHtczzE9hFVx5PAte-7-J6502yPq5a_tOSeCAgRBTNJOT_LRNIUr0BupaGjwWv5vBvOx4ycDimmEHAYNyKLAekxgNzJo5LS8jp9Z5sbLvwMv89-F/s1600/VFI-300x225.png" /></a></div>
<br />
<br />
A free ebook by Julian King that presents a short theory for helping to answer the question "Does XYZ deliver good (enough) value for investment?" - Essentially the question any evaluator is supposed to help answer.<br />
<br />
So, now, there is one more topic on my ever expanding reading list! If there is a "Bible" of economic evaluation, let me have the reference, ok? Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-51475932809282518052014-09-12T16:46:00.000+02:002014-11-14T11:27:13.853+02:00What if, mid career as a researcher, you become interested in Evaluation?<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiinq-W7ajHFRc-BWpc2NeTU5KwQT-11YZ1scfkbXTXILXimdfh3uMBgCr-gSpz1K4XiIOGTlTPmygUNvXf7SKIwoC-NhA1U-sD6_IgJv8MAFzAuV2RuFRzHiVJHUdrQTiGjDSK/s1600/books.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiinq-W7ajHFRc-BWpc2NeTU5KwQT-11YZ1scfkbXTXILXimdfh3uMBgCr-gSpz1K4XiIOGTlTPmygUNvXf7SKIwoC-NhA1U-sD6_IgJv8MAFzAuV2RuFRzHiVJHUdrQTiGjDSK/s1600/books.jpg" /></a></div>
An old classmate, that took the market research route after completing her Research Psych Master's Degree, asked me for a couple of references to check out if she wanted to develop her evaluation knowledge and skills. What came to mind is the following professional development resources. I'm sure there's many more easily accessible ones, but this is a good start for a list!<br />
<br />
<div>
<div>
<ul>
<li>If you are willing to spend some time learning from online lectures, try out any of the<a href="http://www.mymande.org/elearning" target="_blank"> free online courses developed by EvalPartners, Rockefeller and the IOCE. </a>New entrants allowed in January, March and September of each year, and learning is totally self-paced. They are certified. </li>
<li>If you are looking for less intense professional development - Why not check out the American Evaluation Association's <a href="http://comm.eval.org/coffee_break_webinars/home" target="_blank">Coffee Break Webinars? </a>(I think you do have to be an AEA member though!)</li>
<li>If you are looking for something to read about any evaluation method, approach, tool or task, check out <a href="http://www.betterevaluation.org/" target="_blank">Better Evaluation</a>. Subscribe to <a href="http://betterevaluation.org/blog" target="_blank">their blog</a> and their <a href="https://twitter.com/BetterEval" target="_blank">twitter stream</a> to get handy little tips. An amazing resource made available totally free! <span id="goog_1595236129"></span><span id="goog_1595236130"></span></li>
<li>Do you only have time for a short email or blog every now and again? Sign up for the American Evaluation Association <a href="http://aea365.org/blog/" target="_blank">Tip-a-Day blog/ emails</a> or check out the collection of <a href="http://evalcentral.com/" target="_blank">Evaluation Blogs</a> curated at EvalCentral</li>
<li> If you are looking for an accredited online course, try out the <a href="http://www.cgu.edu/workshops" target="_blank">Claremont E-learning options</a>. They usually have bursaries available for Developing Country Evaluators. </li>
<li>What are the two Evaluation books I suggest you should read first?<a href="http://www.amazon.com/Utilization-Focused-Evaluation-Michael-Quinn-Patton/dp/141295861X/ref=sr_1_1?s=books&ie=UTF8&qid=1410532289&sr=1-1&keywords=utilization+focused+evaluation" target="_blank"> Utilization Focused Evaluation</a> - Michael Quinn Patton and<a href="http://www.amazon.com/Purposeful-Program-Theory-Effective-Theories/dp/0470478578" target="_blank"> Purposeful Programme Theory</a> - Sue Funnell & Patricia Rogers</li>
<li>If you are planning to work in the M&E of Government programmes in South Africa, you have to be familiar with The South African Department of Planning, Monitoring and Evaluation's <a href="http://www.thepresidency-dpme.gov.za/publications/Policy%20Framework/National%20Evaluation%20Policy%20Framework%20(NEP).pdf" target="_blank">National Evaluation Policy Framework</a>, and their <a href="http://www.thepresidency-dpme.gov.za/publications/Pages/Policy-Frameworks.aspx#" target="_blank">guidelines</a>. </li>
</ul>
</div>
<div>
</div>
</div>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com1tag:blogger.com,1999:blog-29610685.post-29854893820569517902014-08-12T15:41:00.000+02:002014-08-12T15:46:09.330+02:00Further Resources and Links for those who attended the Bridge M&E Colloquium on 12 August 2014<br />
Today, I got the opportunity to present to the <a href="http://www.bridge.org.za/" target="_blank">Bridge M&E Colloquium</a> on the work I'm doing with the CSIR Meraka Institute on the ICT4RED project. My first presentation gave some background about the ICT4RED project. <br />
<br />
<iframe frameborder="0" height="400" marginheight="0" marginwidth="0" scrolling="no" src="//www.slideshare.net/slideshow/embed_code/37909159" width="476"></iframe><br />
<br />
<br />
I referred to the availability of the Teacher Professional Development course under a creative commons licence <a href="http://www.ict4red.blogspot.com/p/tpd-course.html" target="_blank">here</a>, - This resource also includes a full description of the micro-accreditation system or Badging system. <br />
<br />
What seemed to get the participants in the meeting really excited is the 12 Component model of the project - which seems to suggest that one has to pay attention to much more than just technology when you implement a project of this nature. My colleagues published a paper on this topic <a href="https://www.dropbox.com/s/c1fm6fy2kr84evm/IST%20Africa%20paper%2035%20-%20final_Merryl_Marlien_Adele.pdf" target="_blank">here</a>.<br />
<br />
Participants also resonated with the "Earn as you Learn" model that the project follows - If teachers demonstrate that they comply with certain assessment criteria, they earn technology and peripherals for themselves and for their schools. A paper on the gamification philosophy that underlies the course, is available <a href="https://www.dropbox.com/s/tkzw46a7vumz5t3/Botha%2C%20Herselman%2C%20Ford_Gamification%20beyond%20badges_ISTAfrica_Paper_ref_179.pdf" target="_blank">here</a>. The Learn to Earn model was documented in a learning brief <a href="https://www.dropbox.com/s/7522etfigiujcwv/7%20ICT4RED%20Learning%20Brief%20-%20Professional%20Development%20-%20The%20learn%20to%20earn%20model%20of%20teacher%20incentives.pdf" target="_blank">here</a>.<br />
<br />
<br />
And then I was able to speak a little more about the evaluation design of the project. The paper that underlies this work is <a href="https://www.dropbox.com/s/v2vresytuzki66v/Evaluation%20Design%20of%20the%20Cofimvabav2_with_references.pdf" target="_blank">available here</a>, and the presentation is accessible below: <br />
<br />
<iframe frameborder="0" height="400" marginheight="0" marginwidth="0" scrolling="no" src="//www.slideshare.net/slideshow/embed_code/37909158" width="476"></iframe><br />
<br />
I think what sets our project evaluation apart from many others being conducted in South Africa, is that it truly uses "Developmental Evaluation" as the evaluation approach. For more information about this (and for a very provocative evaluation read in general), make sure you get your hands on <a href="http://www.guilford.com/books/Developmental-Evaluation/Michael-Quinn-Patton/9781606238721" target="_blank">Michael Patton's book</a>. A short description of the approach and a list of other resources can also be found <a href="http://betterevaluation.org/plan/approach/developmental_evaluation" target="_blank">here</a>. <br />
<br />
People really liked the idea of using Learning Briefs to document learning for / from team members, and to share with a wider community. This is an idea inspired by the <a href="http://www.dgmt.co.za/category/learning-briefs-education-to-read-write/" target="_blank">DG Murray Trust</a>. I blogged about the process and template we used <a href="http://mandeblog.blogspot.com/2013/11/im-contributing-to-evaluation-of.html" target="_blank">before</a>. An example of the learning brief that the M&E team developed for the previous round, is available <a href="https://www.dropbox.com/s/0umco6j8nfakvz3/5%20ICT4RED%20Learning%20Brief%20-%20Monitoring%20and%20Evaluation%20-%20Distinguishing%20kinds%20of%20success%20and%20planning%20M%20and%20E.pdf" target="_blank">here</a>. More learning briefs are available on the <a href="http://www.ict4red.blogspot.com/p/resources.html" target="_blank">ICT4RED blog</a>. <br />
<br />
I also explained that we use the<a href="https://www.dropbox.com/s/jue481xm3obe7yv/Example%20Story%20Template%20Version31%20October%202013%20to%20share.pdf" target="_blank"> Impact Story Tool</a> for capturing and verifying an array of anticipated and unanticipated impacts. I've explained the use and analysis of the tool in more detail in <a href="http://mandeblog.blogspot.com/2014/02/working-rigorously-with-stories-impact.html" target="_blank">another blog post</a>. There was immediate interest in this simple little tool. <br />
<br />
A neat trick that also got some people excited, is how we use Survey Monkey. To make sure that our data is available quickly to all potential users on the team, we capture our data (even data collected on paper) in Survey Monkey, and then share the results with our project partners via the sharing interface on Surveymonkey - even before we've really been able to analyse the data. The Survey Monkey site, <a href="https://www.surveymonkey.com/mp/tour/sharableresults/" target="_blank">explains this in a little more detail</a> with examples.<br />
<br />
The idea of using non-traditional electronic means to help with data collection also got some participants excited. I explained that we have a Whatsapp group for facilitators, and we monitor this, together with our more traditional post-training feedback forms, to ascertain if there are problems that need solving. In an upcoming blog post, I'll share a little bit about exactly how we used the WhatsApp data, and what we were able to learn from it. <br />
<br />Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com1tag:blogger.com,1999:blog-29610685.post-79825917902843090022014-06-24T10:53:00.001+02:002014-06-24T13:24:29.791+02:00Exciting Learning from people involved in South African ICT in Education<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYgWPqGSPE-1-LAeAaxmbJpKsVq06-tCPX_UNN_H_Bx35bXxmlk7Cy0HdIVRzYfT3tTUBzqHEtEN589eJE7KcK1DbNAqRZQT47uDn3ab_EWT8kAw6rlACYPNMMe2FGGQ6ayqgh/s1600/tablet.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYgWPqGSPE-1-LAeAaxmbJpKsVq06-tCPX_UNN_H_Bx35bXxmlk7Cy0HdIVRzYfT3tTUBzqHEtEN589eJE7KcK1DbNAqRZQT47uDn3ab_EWT8kAw6rlACYPNMMe2FGGQ6ayqgh/s1600/tablet.jpg" height="155" width="320" /></a></div>
I've been fortunate to be invited to a small gathering of people working with Coza Cares in the ICT space in South Africa. The luxury of sitting down for two days and listening to people talk about what they are passionate about, is something to truly savour.<br />
<br />
I did a presentation on some ideas I have to define and measure learners' 21st Century Skills in the context of the ICT4RED project. I currently have more questions than answers, but I'm sure we will get there soon. <a href="http://ow.ly/y7sAD" target="_blank">Here </a>is a link to a summary table comparing different Definitions of 21st Century Skills.<br />
<br />
Other presentations I really enjoyed was<br />
* Barbara Dale Jones on the role of <a href="http://www.bridge.org.za/ipoint" target="_blank">Bridge</a> and learning communities and knowledge management<br />
* Fiona Wallace, on the <a href="http://cozacares.co.za/" target="_blank">CoZaCares</a> model of ICT intervention<br />
* John Thole on <a href="http://www.edunova.org/john-blog" target="_blank">Edunova's</a> programme to train and deploy youth to support ICT implementation in Schools<br />
* Siobhan Thatcher from Edunova's presentation on Edunova's model for deploying <a href="https://sites.google.com/a/edunova.org/elcdm/home" target="_blank">Learning Centres </a>in support of Schools<br />
* Brett Simpson from <a href="http://breadbin.co.za/" target="_blank">Breadbin Interactive</a> on the learning they've done on the deployment of their content repository.<br />
*Ben Bredenkamp from Pendula ICT talking about their model for ICt integration and experience of the <a href="http://one.laptop.org/" target="_blank">One Laptop per Child</a> project in South Africa. <br />
* Dylan Busa from <a href="http://learn.mindset.co.za/" target="_blank">Mindset </a>speaking about the relaunch of their website content. <br />
* Merryl Ford and Maggie Verster talking about the I<a href="http://ict4red.blogspot.com/" target="_blank">CT4RED</a> project<br />
<br />
<br />
<br />
<br />Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com1tag:blogger.com,1999:blog-29610685.post-56479127276005687222014-05-28T10:44:00.000+02:002014-05-28T10:44:00.285+02:00Impact Evaluation Guidance for Non-profits<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiW6I8NHZRZrww6yINTcV4AHciGKoOHrtsOFTqRwV67cHyRyY9eYK0yz94-M6eIVy823dTcJRDT5JG9UgIKJGFtzGunZ91eC4U6Zbype7XVn_BCifYz1NYxfw56G4228ZhuQJlH/s1600/logo_0.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiW6I8NHZRZrww6yINTcV4AHciGKoOHrtsOFTqRwV67cHyRyY9eYK0yz94-M6eIVy823dTcJRDT5JG9UgIKJGFtzGunZ91eC4U6Zbype7XVn_BCifYz1NYxfw56G4228ZhuQJlH/s1600/logo_0.jpg" height="60" width="200" /></a><a href="http://www.interaction.org/impact-evaluation-notes" target="_blank">Interaction</a> has this lovely Guidance note and Webinar Series on Impact Evaluation available on their website. <br />
<br />
<blockquote class="tr_bq">
<h2 class="title title-node-326" id="page-title">
Impact Evaluation Guidance Note and Webinar Series</h2>
<div class="left-caption">
</div>
With financial support from the <a href="http://www.rockefellerfoundation.org/">Rockefeller Foundation</a>,
InterAction developed a four-part series of guidance notes and webinars
on impact evaluation. The purpose of the series is to build the
capacity of NGOs (and others) to demonstrate effectiveness by increasing
their understanding of and ability to conduct high quality impact
evaluation.<br />
The four guidance notes in the series are:<br />
<ol>
<li><a href="http://bit.ly/AwYbT1"><em>Introduction to Impact Evaluation</em></a>, by Patricia Rogers, Professor in Public Sector Evaluation, RMIT University</li>
<li><a href="http://bit.ly/IGeIoE"><em>Linking Monitoring & Evaluation to Impact Evaluation</em></a>, by Burt Perrin, Independent Consultant</li>
<li><a href="http://bit.ly/TiAvrQ"><em>Introduction to Mixed Methods in Impact Evaluation</em></a>, by Michael Bamberger, Independent Consultant</li>
<li><a href="http://bit.ly/SNq1iR"><em>Use of Impact Evaluation Results</em></a>, by David Bonbright, Chief Executive, Keystone Accountability</li>
</ol>
Each guidance note is accompanied by two webinars. In the first
webinar, the authors present an overview of their note. In the second
webinar, two organizations - typically NGOs - present on their
experiences with different aspects of impact evaluation. In addition,
each guidance note has been translated into several languages, including
Spanish and French. Webinar recordings, presentation slides and the
translated versions of each note are provided on the <a href="http://www.interaction.org/impact-evaluation-notes" target="_blank">website</a>. </blockquote>
<br />Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-88094828425677377692014-05-21T13:30:00.000+02:002014-05-22T10:47:30.144+02:00Resources on Impact Evaluation<blockquote class="tr_bq">
This post consolidates a list of impact evaluation resources that I usually refer to when I am asked about impact evaluations. </blockquote>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVg8ZRq-_RKly18Nxlh6L8BlS1Rbgh6oI-IcbuTrTbxoKNMHPxlYExGeNhI82fFEVeWH2RlpMq-EZU1TMqrIEtXMebaGyCRb90i1C6roytQiNnGJsLDoVkk6h-c84dzs5xz5u6/s1600/impact.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVg8ZRq-_RKly18Nxlh6L8BlS1Rbgh6oI-IcbuTrTbxoKNMHPxlYExGeNhI82fFEVeWH2RlpMq-EZU1TMqrIEtXMebaGyCRb90i1C6roytQiNnGJsLDoVkk6h-c84dzs5xz5u6/s1600/impact.jpg" height="150" width="200" /></a></div>
This <a href="https://cdnapisec.kaltura.com/index.php/extwidget/openGraph/wid/1_nnm86fbi" target="_blank">cute video</a> explains the factors that distinguishes impact evaluation from other kinds of evaluation, in two minutes. Of course randomization isn't the only way of credibly attributing causes and effects - and this is a particularly hot evaluation methodology debate. For an example of why this is sometimes an irrelevant debate - <a href="http://popperfont.files.wordpress.com/2014/04/parachute.pdf" target="_blank">see this write up on parachutes </a>and <a href="http://freshspectrum.com/6-rct-randomista/" target="_blank">Chris Lysy's cartoons on the topic</a>. <br />
<br />
<div>
</div>
<b>Literature on the Impact Evaluation Debate</b> <br />
<br />
The Impact Evaluation debate flared up after this report, <a href="http://www.cgdev.org/publication/when-will-we-ever-learn-improving-lives-through-impact-evaluation" target="_blank">titled "When will we ever learn"</a> was released in 2006. In the States there also was a prominent funding mechanism which required programmes to include experimental <span class="il">evaluation</span> methods in their design, or not get funding (from about 2003 or so). <br />
<div>
<br /></div>
<div>
The bone of contention was that Randomized Control Trials (RCTs) and Experimental methods (and to some extent Quasi Experimental Designs) were held up as the "gold standard" in <span class="il">evaluation</span>.
Which, in my opinion, is nonsense. So the debate about what
counts as evidence started again. The World Bank and big corporate
donors were perceived to push for Experimental Methods, <span class="il">Evaluation</span>
Associations (with members committed to mixed methods) pushed back
saying methods can't be determined without knowing what the questions are. And others pushed back saying that RCTs are probably applicable in only about 5% of the cases in which <span class="il">evaluation</span> is necessary. <br />
<br />
The methods debate in <span class="il">Evaluation</span> is really an old debate. Some
really prominent evaluators decided to leave the AEA because they
embarked on a position that they equated with "The flat earth movement"
in geography. <a href="http://journals.sfu.ca/jmde/index.php/jmde_1/article/viewFile/101/116" target="_blank">Here is a nice overview article, (The 2004 Claremont Debate: Lipsey vs. Scriven. DeterminingCausality in Program Evaluation and Applied Research: Should ExperimentalEvidence Be the Gold Standard?)</a>
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:DoNotShowPropertyChanges/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-ZA</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="267">
<w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" Priority="39" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" Name="toc 9"/>
<w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
</w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-fareast-language:EN-US;}
</style>
<![endif]--> to summarise some of it.</div>
<div>
The Network of Networks in <span class="il">Impact</span> <span class="il">evaluation</span> then <a href="http://siteresources.worldbank.org/EXTOED/Resources/nonie_guidance.pdf" target="_blank">sought to write a guidance document</a>, but even after this was released, there was a feeling that not enough was said to counter the "gold standard" mentality. This <a href="http://betterevaluation.org/resource/overview/Designing_impact_evaluations_different_perspectives" target="_blank">document, titled "Designing impact evaluations, different perspectives</a>" provides a bit more information on the "other views".<br />
<br /></div>
<div>
</div>
<div>
<b>Literature on Impact Evaluation Methods</b></div>
If you are interested in literature on Evaluation Methods, look at <a href="http://betterevaluation.org/" target="_blank">Better <span class="il">Evaluation</span></a> to get a quick overview. <br />
<br />
<div>
I like <a href="http://www.amazon.com/Experimental-Quasi-Experimental-Designs-Generalized-Inference/dp/0395615569" target="_blank">Cook, Campbell and Shadish</a> to understand
experimental and quasi experimental methods, but this <a href="http://www.socialresearchmethods.net/kb/design.php" target="_blank">online knowledge base resource </a>is good too.<br />
<cite></cite> </div>
<div>
For some resources on other more mixed methods approaches to impact evaluation, you need to look at <a href="http://betterevaluation.org/evaluation-options/realistsynthesis" target="_blank">Realist Synthesis</a>, <a href="http://betterevaluation.org/evaluation-options/list_possible_causes_general_elimination_methodology" target="_blank">General Elimination Method</a>, <a href="http://www.3ieimpact.org/media/filer/2012/05/07/Working_Paper_3.pdf" target="_blank">Theory Based <span class="il">Evaluation</span></a>, and something that I think has potential, the <a href="https://www.blogger.com/-%20http://betterevaluation.org/plan/approach/cort" target="_blank">Collaborative Outcomes Reporting</a> approach. <br />
<br />
<br />
The <a href="http://www.google.co.za/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CCwQFjAA&url=http%3A%2F%2Fwww.thepresidency-dpme.gov.za%2Fkeyfocusareas%2FevaluationsSite%2FEvaluations%2FGL%25202%25202%252013%2520Impact%2520Evaluation%252014%252003%252020.pdf&ei=Nol8U6zXL8bB7Abb7oHQAg&usg=AFQjCNGg2FlHEWB0BFzX12gWt2REYigwQw&sig2=XbxmLJY_iJspzW_9kBl7lg&bvm=bv.67229260,d.ZGU" target="_blank">South African Department for Performance Monitoring and Evaluation's guideline on Impact Evaluation</a> is also relevant if you are interested in work in the South African Context. </div>
<br />Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-91331581141247577862014-04-02T15:08:00.000+02:002014-04-02T15:08:02.834+02:00Getting authorisation to do Research and Evaluation in Schools<div class="MsoNormal">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8UqJQLJy1JFmBQ1qROqc7jGt9qCYhDUEOZl_zAxxB4Z0znYzm4jxW1JfJMAgcQ29GCmlGqoEKdOGVVQ2CsUlkYXPUMj7EOcdiSOgjrFWuwBHDQUv9_3zogoLtW_ljx8u4mwyw/s1600/ETHICS.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8UqJQLJy1JFmBQ1qROqc7jGt9qCYhDUEOZl_zAxxB4Z0znYzm4jxW1JfJMAgcQ29GCmlGqoEKdOGVVQ2CsUlkYXPUMj7EOcdiSOgjrFWuwBHDQUv9_3zogoLtW_ljx8u4mwyw/s1600/ETHICS.jpg" height="133" width="200" /></a><span lang="EN-US"> </span></div>
<div class="MsoNormal">
<span lang="EN-US">A colleague working in an educational NGO asked this question, about working in schools in South Africa: </span></div>
<div class="MsoNormal">
<br /></div>
<blockquote class="tr_bq">
<div class="MsoNormal">
<span lang="EN-US">I just wanted to ask a quick
question. Do I need to get permission from the relevant Provincial Department
of Education to carry out research in schools if the schools are part of
a project we’re running? In other words, the district is aware of us
and probably interacting with us?</span></div>
</blockquote>
<div class="MsoNormal">
<span lang="EN-US"> </span></div>
<div class="MsoNormal">
<span lang="EN-US">My answer: </span></div>
<div class="MsoNormal">
<span lang="EN-US">I've only done research or evaluations in a few Provinces, not all of them, but in all of those Provinces the Education Departments
have guidelines for researchers that require you to fill in forms, submit your research proposal (and sometimes evaluation instruments) for review, and
also binds you to some promises about the use of your research or evaluation findings. (E.g. the Province may require copies of reports, may require you to present your findings, etc.) Check any of the Provinces' annual reports to see which Director in the Provincial office is in charge of Research, and lodge your enquiry about requirements there, if you can't find details on the Provincial Education Website. </span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US">The officials in Education Districts are often not aware of the Provincial requirements, so one
might be able to get away without Provincial authorization, but this is a
bad idea for at least two reasons: </span></div>
<div class="MsoNormal">
<br /></div>
<div dir="ltr">
* It helps if the Research Directorate in the Provincial Education Department
have your details on their database because it promotes use and
coordination of research, and</div>
<div dir="ltr">
*It can solve a lot of headaches for you
should someone complain about your research going forward. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Since Education in schools is a Provincial competence, I have been
unable to get blanket approval from National Education to work in
multiple Provinces - so that meant filling in the different forms and providing the different details to the different Provinces, and following up on the outcome of each of these processes. </div>
<div dir="ltr">
<br /></div>
Besides Provincial approval, some clients might also
require that any human subject research gets vetted by a research ethics
approval board, like the ones attached to universities, or science
councils. I've only dealt with a few of these, but they mostly require you to prove that you have authorization to conduct the research, so the two goes hand in hand.<br />
<br />
Of course approval by the Province and Research Ethics Boards are still not all that you need to do to ensure that you conduct your work ethically - Some fields (E.g. Marketing Research - see the <a href="http://www.esomar.org/publications-store/codes-guidelines.php" target="_blank">ESOMAR</a> guidelines), have guidelines about ethics... so it would be good to study these and make sure your practice remains above board.<br />
<br />
And then this, of course, is also true:<br />
<br />
<blockquote class="tr_bq">
Live one day at a time emphasizing ethics rather than rules.<br />
Wayne Dyer</blockquote>
<br />
<div class="MsoNormal">
<br /></div>
<span lang="EN-US"><br /></span>Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com2tag:blogger.com,1999:blog-29610685.post-59478249413034177802014-03-27T22:11:00.001+02:002014-03-27T22:11:28.671+02:00I am because you are<p dir="ltr">In a <a href="http://betterevaluation.org/blog/ubuntu_in_evaluation">previous blogpost</a> I reflected on how African values shape my practice of Evaluation.</p>
<p dir="ltr">This week I attended a seminar during which Gertjan Van Stam shared some provocative views on development in Africa. I started reading his book '<a href="http://www.vanstam.net">Placemark</a>'. I love the way he gives voice to rural Africa. I find it interesting that this Dutch Engineer manages to give voice to Africa in a way that I can relate with. </p>
<p dir="ltr">His beautifully written take on Ubuntu:</p>
<p dir="ltr"><i><b>I am, because You are</b></i></p>
<p dir="ltr"><i>Is it possible that people in rural areas of Africa can connect with people in urban areas around the world?</i></p>
<p dir="ltr"><i>That one can walk into a scene and meet someone who walks into the same scene, even if it is geographically separated?</i></p>
<p dir="ltr"><i>That we explore and connect rural and urban worlds worldwide without anyone being forced into cultural suicide?</i></p>
<p dir="ltr"><i>That we meet around the globe and relate, embrace, love, and build meaningful relationships?</i></p>
<p dir="ltr"><i>That we find ways to be of significance and support to each other and together shuffle poverty and disease into the abyss?</i></p>
<p dir="ltr"><i>That we encourage each other to withstand drunkenness and drugs, bullying, self harm, and greed?</i></p>
<p dir="ltr"><i>That we share spiritual nutrition to deal with wealth, loss, alienation and pain in this generation?</i></p>
<p dir="ltr"><i>That we unite through social networks, overcoming divides and separations?</i></p>
<p dir="ltr"><i>That we share ancient, tested, and new resources, opportunities, visions, and dreams that lead to knowledge, understanding and wisdom?</i></p>
<p dir="ltr"><i>That we collaborate to discuss, and engineer tools, taking into account the integral health of all systems?</i></p>
<p dir="ltr"><i>That together, South and North, build capacity, mutual accountability, and progress, for justice and fairness?</i></p>
<p dir="ltr"><i>That I am, because You are?</i></p>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-41980116431373531822014-03-17T12:11:00.002+02:002014-03-17T15:23:46.734+02:0021st Century Skills of Rural African Teachers and Learners<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]--><br />
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:DoNotShowPropertyChanges/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-ZA</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="267">
<w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" Priority="39" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" Name="toc 9"/>
<w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
</w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-fareast-language:EN-US;}
</style>
<![endif]-->
<br />
<div class="MsoNormal" style="mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
I’m
evaluating a project that aims to build the 21<sup>st</sup> century skills of
rural African teachers and learners. Until recently I did not even know what
people meant when they used the phrase 21<sup>st</sup> Century skills, but I
have been enlightened and must now find a way to measure it for our evaluation. </div>
<div class="MsoNormal" style="mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoNormal" style="mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
It seems I’m not the only one struggling with the problem of having to measure
something very broad - There are a range of resources available that wrangle
with the idea of defining and measuring 21<sup>st</sup> Century Skills – Some
of the resources I found particularly useful include: </div>
<div class="MsoNormal" style="mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<a href="http://www.p21.org/about-us/p21-framework">http://www.p21.org/about-us/p21-framework</a>
</div>
<div class="MsoNormal" style="mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<a href="http://atc21s.org/index.php/about/what-are-21st-century-skills/">http://atc21s.org/index.php/about/what-are-21st-century-skills/</a>
</div>
<div class="MsoNormal" style="mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<a href="http://21stcenturyskillsbook.com/">http://21stcenturyskillsbook.com/</a></div>
<div class="MsoNormal" style="mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<br /></div>
<div class="MsoNormal" style="mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
Everything
I read, however, seems to have the focus on a context that is not rural and not
African. Perhaps there is scope for our project to contribute to the general
discussion on 21st Century Skills by adapting the definitions and measures specifically for our
context? Perhaps this is an opportunity to develop an example of African Made,
African Owned Evaluation? </div>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-61393893193921796792014-02-26T21:54:00.001+02:002014-02-26T21:54:45.930+02:00My plans for AfrEA 2014 conference<p dir="ltr">I'm off to Cameroon on Sunday for a week of networking, learning and sharing at the 2014 <a href="http://www.afrea.org">AfrEA</a> conference in Yaondé. I love seeing bits of my continent. If internet access is available I'll try to tweet from @benitaW. </p>
<p dir="ltr">I am facilitating a workshop on Tuesday together with the wise Jim Rugh and the efficient Marie Gervais to share a bit about a VOPE toolkit EvalPartners is developing. ( A VOPE is an evaluation association or society... voluntary organization for professional evaluation)</p>
<p dir="ltr"><b>Workshop title</b><b>:Establishing and strengthening VOPEs: testing and applying the EvalPartners Institutional Capacity Toolkit</b></p>
<p dir="ltr"><b>Abstract:</b> One of the EvalPartners initiatives, responding to requests received from leaders of many VOPEs (Voluntary Organizations for Professional Evaluation), is to develop a toolkit which provides guidance to those who wish to form even informal VOPEs, and leaders of existing VOPEs who seek guidance on strengthening their organization’s capacities.  During this workshop participants will be introduced to the many subjects addressed in the VOPE Institutional Capacity Toolkit, and asked to test the tools as they determine how they could help them apply such resources in strengthening their own VOPEs.</p>
<p dir="ltr">The workshop will be very interactive with lots of exploring, engaging, and evaluating of the toolkit resources. Participants should not come to this workshop expecting that they will sit still for more than 30 minutes at a time. We'll use a combination of learning stations and fishbowls as the workshop methodology.  I'm really looking forward to it!</p>
<p dir="ltr">Eventually the toolkit will be made available online. Follow @vopetoolkit on twitter for more news about developments.</p>
<p dir="ltr">I served on the boards of both AfrEA and SAMEA so I hope that the resources that the Toolkit task force and their marvellous team of collaborators put together in the toolkit will be of use to colleagues across the continent who are still founding or strengthening their VOPEs. It is hard and sometimes thankless work to serve on a VOPE board, and if this toolkit can make someone's life a little easier with examples, tools and advice, I would count this as a worthy effort. </p>
<p dir="ltr">I expect that the workshop will be a good opportunity to get some Feedback to guide us in the completion of the work.</p>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-68521357334401775792014-02-17T08:07:00.000+02:002014-02-17T08:07:00.394+02:00Working Rigorously with Stories - Impact Story Tool<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEZw4WEJpkFKc_qZybEEIDCEorK3XDc3nuOEpB-XdTveXpSaZ8FnKl3ncMnbWEr2MbSXYyR9Yo3pEh9fzmTM2RY0aug-RM6-o5_PRd5yEsMHyMOOcTtV0LTrmo-NbvevRS0vdA/s1600/Impact+Story+Picture.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEZw4WEJpkFKc_qZybEEIDCEorK3XDc3nuOEpB-XdTveXpSaZ8FnKl3ncMnbWEr2MbSXYyR9Yo3pEh9fzmTM2RY0aug-RM6-o5_PRd5yEsMHyMOOcTtV0LTrmo-NbvevRS0vdA/s1600/Impact+Story+Picture.jpg" height="240" width="320" /></a></div>
<br />
<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]--><br />
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:DoNotShowPropertyChanges/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-ZA</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="267">
<w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" Priority="39" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" Name="toc 9"/>
<w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
</w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-fareast-language:EN-US;}
</style>
<![endif]-->
<br />
<div class="MsoNormal" style="margin-bottom: 12.0pt;">
<span lang="EN-GB" style="font-family: "Arial","sans-serif"; font-size: 10.0pt;">I've had some people email me about a paper I presented at the 2013 SAMEA conference. This paper introduces a tool for collecting and rigorously analysing
<b>impact stories</b> that could be used as part of an evaluation. The <a href="https://dl.dropboxusercontent.com/u/36503689/Williams%20Chilalika%20Impact%20Story.pdf" target="_blank">full paper with the tool can be accessed here</a>. The abstract is presented below:</span></div>
<br />
<blockquote class="tr_bq">
<div class="MsoNormal" style="margin-bottom: 12.0pt;">
<span lang="EN-GB" style="font-family: "Arial","sans-serif"; font-size: 10.0pt;"> Beneficiary stories
are an easily collected data source, but without specific information in the
story, it may be impossible to attribute the mentioned changes to an
intervention or to verify that the change actually occurred. Approaches such as
Appreciative Inquiry and the Most Significant Change Technique have been
developed in response to the need to work more rigorously with this potentially
rich form of data. The “Impact Story Tool” is yet another attempt to make the
most of rich qualitative data and was developed and tested in the context of a
few programme evaluations conducted by Feedback RA. </span></div>
<div class="MsoNormal" style="margin-bottom: 12.0pt;">
<span lang="EN-GB" style="font-family: "Arial","sans-serif"; font-size: 10.0pt;">The tool consists of
a story collection template and an evaluation rubric that allows for the story
to be captured, verified and analysed. Project participants are encouraged to
share examples of changes in skills, knowledge, attitudes, motivations,
individual behaviours or organizational practice. The tool encourages
respondents to think about the degree to which the evaluated programme
contributed towards the mentioned change, and also asks for the details of
another person that may be able to verify the reported change. The analyst
collects the story, verifies the story and then codes the story using a rubric.
When a number of stories are collected in this way, they are then analysed
together with other evaluation data. It may illuminate which parts of a specific
intervention are most frequently credited with contributing towards a change. </span></div>
<span lang="EN-GB" style="font-family: "Arial","sans-serif"; font-size: 10.0pt; mso-ansi-language: EN-GB; mso-bidi-language: AR-SA; mso-fareast-font-family: Times; mso-fareast-language: EN-US;">Besides introducing the tool as it was used in
three different evaluations, the usefulness of this tool and possible drawbacks
are discussed.</span></blockquote>
(The picture above is of a character known as "Benny Bookworm" from a South African TV show called "Wielie Walie" which I watched as a child)Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com2tag:blogger.com,1999:blog-29610685.post-77835376934572255262014-02-10T08:14:00.000+02:002014-02-13T09:55:00.135+02:00Reflection: Thanks Tom GraysonSo since I started a stroll down memory lane, I thought I'd share this too. In 2002 I decided I wanted to be an evaluator. I was working at an evaluation company, and I decided to start my own consultancy, so I was getting great practical exposure. But I really did not have a good academic grounding in the theory and literature surrounding Evaluation. This was back in the day when there weren't MOOCS and webinars... so I had to READ to get my education.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbaY7YZ_O53PVkJcCjedkmDIKZtMcJ-nD-U_iidXoS4P_qNGyBaFA0FqA3dbB5_a23aUG1pZPIrbjxGX3MibdRpzDr5QFpGZLPivfAysXB6iz-XjDvt2w0u4EObqgACJgbFmvD/s1600/2014-01-28+10.52.29.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbaY7YZ_O53PVkJcCjedkmDIKZtMcJ-nD-U_iidXoS4P_qNGyBaFA0FqA3dbB5_a23aUG1pZPIrbjxGX3MibdRpzDr5QFpGZLPivfAysXB6iz-XjDvt2w0u4EObqgACJgbFmvD/s1600/2014-01-28+10.52.29.jpg" height="320" width="180" /></a>During my studies I had read <a href="http://www.amazon.com/Experimental-Quasi-Experimental-Designs-Generalized-Inference/dp/0395615569" target="_blank">Cook and Campbell</a>, and somehow I also stumbled upon <a href="http://www.amazon.com/Fourth-Generation-Evaluation-Egon-Guba/dp/0803932359" target="_blank">Guba and Lincoln</a>. I was introduced to <a href="http://www.amazon.com/Utilization-Focused-Evaluation-Michael-Quinn-Patton/dp/141295861X" target="_blank">Utilization Focused Evaluation</a>. In 2004 I got <a href="http://www.amazon.com/Evaluation-Systematic-Peter-Henry-Rossi/dp/0761908943" target="_blank">Rossi, Lipsey and Freeman</a> for a going away present from Khulisa, and I read any evaluation journal articles I could lay my hands on.<br />
<br />
Its after reading something that Tom Grayson (from the University of Illinois at Urbana-Champaign) wrote in a journal article about teaching evaluation, that I decided to email him. I asked him for some reading material that will give me a good basis in Evaluation. He responded by sending me a package of course reading materials via post... This was such an unexpected gesture of goodwill. Above is a little handwritten note that he sent with the material.<br />
<br />
So Tom, thanks a lot. And this is me letting you know about my adventures in evaluation! <br />
<br />Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com1tag:blogger.com,1999:blog-29610685.post-8647909938483117492014-02-03T09:00:00.000+02:002014-02-03T22:48:56.534+02:00Reflection: I decided to become an evaluator in May of 2002<div class="separator" style="clear: both; text-align: left;">
As I was packing up the FeedbackRA office, I found a few files that I needed to clear out. This one is special, because it was during this workshop that I chose to relate to the identity of Evaluator.</div>
<div class="separator" style="clear: both; text-align: left;">
<br></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrXK-L7yYd-Wpmos0zCo88EV-BOLGUwnImAaUXozPQkGrur9JYLL7HNdkDHSLuvXg0p00tItpgIHXqcCOgXHtqYSK9QAjM8j1o8qWTn9jagKIGKEzdG0Z6yQr25a7xkz_7CEI_/s1600/2014-01-28+10.56.36.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrXK-L7yYd-Wpmos0zCo88EV-BOLGUwnImAaUXozPQkGrur9JYLL7HNdkDHSLuvXg0p00tItpgIHXqcCOgXHtqYSK9QAjM8j1o8qWTn9jagKIGKEzdG0Z6yQr25a7xkz_7CEI_/s1600/2014-01-28+10.56.36.jpg" height="320" title="Course File: Evalaution for Development 2002" width="180"></a>It is a file for a workshop titled "Evaluation for Development: An Advanced Course in Evaluation". It was presented by Michael Quinn Patton
in Pretoria in 2002, and it was arranged by Zenda Ofir from Evalnet.</div>
<div style="text-align: left;">
</div>
<div style="text-align: left;">
My academic training in the field of Research Psychology meant that I
was comfortable with research methods, but I also wanted to be involved
in Development... I didn't have a good idea of what I wanted to do
with the research
skills, and frankly, before joining Khulisa I had never heard of
evaluation as a career option. The Community Psychology training I did
at Honours and Masters level
resonated deeply with me. Previously I had thought that I wanted to be a
project officer at an NGO or international development organization, but
I also realised that I like doing the research. I think that after
about a year's working experience I started to think of myself as a
researcher. </div>
<div class="separator" style="clear: both; text-align: left;">
<br></div>
<div class="separator" style="clear: both; text-align: left;">
I will forever be grateful for the experience I got at Khulisa
Management Services, and the fact that Jennifer Bisgard let me go to
this workshop in 2002. Thanks to Zenda Ofir for arranging it, and thanks
for Michael Quinn Patton for preaching/teaching so convincingly.</div>
Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0tag:blogger.com,1999:blog-29610685.post-77603765957717796472014-01-30T13:26:00.003+02:002014-01-30T22:02:06.099+02:00So long FeedbackRA, and thanks for all the fish!Below is a picture of myself, my business partner, and our spouses on
the day we moved into our offices in April 2006. And then a picture of
the people in the Feedback Offices today, for one last time (Terence is away in Rwanda on a jobbie). <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTTYGMuPTc3i8dVWavzizcUvUqd1FQGUHgV-_TvHu6W7SFGxF2Q3P-Z7bCbbG_w-nJbtw3fy8vS29vCZViNZoFTShhbeo2CMK4oKBhre2Ql4wzTb6O0Xd3HwaKh-DQlNVqCBRn/s1600/DSC02278.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTTYGMuPTc3i8dVWavzizcUvUqd1FQGUHgV-_TvHu6W7SFGxF2Q3P-Z7bCbbG_w-nJbtw3fy8vS29vCZViNZoFTShhbeo2CMK4oKBhre2Ql4wzTb6O0Xd3HwaKh-DQlNVqCBRn/s1600/DSC02278.JPG" height="240" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUpd6F7r_-UmVFc3vOfdLC1640wJAyZuKBlRh9IVvy60IrPAKHpL0skzEWUwO93nOUvf6PUQN9JuJNOCllASlGJ1xmxrR_tlywQIJO_Rtq_8zMPpIswlI4yt29GHYo6ni7iQLP/s1600/20140130_122746.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUpd6F7r_-UmVFc3vOfdLC1640wJAyZuKBlRh9IVvy60IrPAKHpL0skzEWUwO93nOUvf6PUQN9JuJNOCllASlGJ1xmxrR_tlywQIJO_Rtq_8zMPpIswlI4yt29GHYo6ni7iQLP/s1600/20140130_122746.jpg" height="180" width="320" /></a></div>
<br />
Today is a little bit of a sad day for me. We’re packing up the FeedbackRA offices which have been the place where I practiced my profession for the past eight years. I'm starting at my new offices tomorrow, and it means that my relationship with FeedbackRA is one step closer to dissolving further. <br />
<br />
The timeline:<br />
I started the business in 2002 together with two colleagues. Our first job was a survey we did for MTN. Our first evaluation job was for the Gauteng Education Development Trust.<br />
<br />
In 2004 I quit my fulltime job to earn my own salary at Feedback. In this year we did a nice piece of research for the Department of Science and Technology, and started expanding our CSI business base.<br />
<br />
In 2006 we graduated to proper offices. We appointed our first employees and embarked on a range of projects that just saw the business grow – in terms of its focus, our skill and the turnover.<br />
<br />
In 2007 and 2008 we did a strategic piece of research on JIPSA, and I got to interview ministers and captains of industry. In this time I was also involved with setting up SAMEA and in 2009 I started to contribute to AFREA too. <br />
<br />
In September 2009 I took a step back to reflect on the important things in my life. Up and ‘til 2009 I managed the business and business finances, and delivered as a key consultant, while keeping an eye on things at SAMEA and AFREA. I just couldn’t do it all any longer.<br />
<br />
In 2010 I scaled down on my work and volunteer responsibilities, and we got Daleen involved to help us run the business. My life was much simpler after that… but it was still tough, because the business kept on growing. I did some lovely work with colleagues from Stellenbosch on a Public expenditure tracking survey… and I learnt so much from being the “junior” on the team. <br />
<br />
In 2010 we started negotiations with a range of other high-profile consultants to see if they would like to join as business partners. We worked together on a few projects, we had a strategic planning session… everything looked good. Our business was expanding and we decided to take up more office space. We started two big contracts which allowed us the scope to do some longer term planning, but it also opened the business up to risk… because things did not always go according to plan. <br />
<br />
In 2011 an advertisement for the only other job I ever thought I’d consider, crossed my desk. I decided to apply, and let my business partners know. We let the conversation with the other potential business partners cool down a bit. After 6 months in limbo, I found out that I did not get the other job, but it took only 1 month for me to realise that what I’ve gained at Feedback was significant enough for me to call it quits. The stress of managing a business and a full consulting plate was just too much. I took on one too many assignments – because a colleague that I respect a lot twisted my arm. This had bad consequences for me as an individual. I didn’t do my best work any longer. I wanted to do things well again… I wanted to focus on things that I could do well, instead of just taking on jobs to make sure cash flow was sorted. A week after my decision, I found out I was going to be a mom. So suddenly I had another reason to reevaluate my priorities.<br />
<br />
I sold my interest in Feedback RA in September 2011, and handed over all management responsibility to my colleagues. I worked as a consultant at FeedbackRA until April 2012, and then returned on a part time basis on selected assignments from October 2012. In this time I realised that I really preferred working on Education issues, and the area of ICT for Education became my core focus.<br />
<br />
I continued working at Feedback until October 2013, when I decided to start my own consultancy again – Benita Williams Evaluation Consultants. I still helped out with some of the FeedbackRA work, and by January 2014 I was able to take over some of the FeedbackRA staff.<br />
<br />
Tomorrow will be the first day in our new offices, which is quite exciting. But there is a side of me that is really sad and nostalgic for what was.<br />
<br />
Thanks colleagues, collaborators, clients and friends. Thanks to my family. It is the end of an era, so.... "So long, and thanks for all the fish!" <br />
<br />Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com6tag:blogger.com,1999:blog-29610685.post-38315057425626681992013-12-20T14:28:00.000+02:002013-12-20T14:31:16.475+02:00Making a serious point... With a Mini-mouse Ribbon on my headThe ICT4RED project released this video of team members sharing how their lives were affected by the project (Follow <a href="http://www.youtube.com/channel/UCFCGAyAtyT9WffG3y4JJ0DA?feature=watch" target="_blank">Mobilina Cofimvaba</a>'s channel on Youtube for more videos about the project). In the video I share how I was motivated to use Twitter as professional learning tool. <br />
<br />
<iframe allowfullscreen="" frameborder="0" height="315" src="//www.youtube.com/embed/7WMESELTY_U" width="560"></iframe><br />
<br />
For some reason the M&E people got associated with being mice. Maybe because we snoop around everywhere, maybe because we need big ears to listen, maybe because our job involves us being quiet... (Though I haven't managed being quiet yet...) So for the tablet fun day on 2 November I donned a mini-mouse ribbon to man the M&E Mouse station. <br />
<br />
<br />
<br />Benita Williamshttp://www.blogger.com/profile/10124085540716614485noreply@blogger.com0