My plans for AfrEA 2014 conference

I'm off to Cameroon on Sunday for a week of networking, learning and sharing at the 2014 AfrEA conference in Yaondé. I love seeing bits of my continent. If internet access is available I'll try to tweet from @benitaW.

I am facilitating a workshop on Tuesday together with the wise Jim Rugh and the efficient Marie Gervais to share a bit about a VOPE toolkit EvalPartners is developing. ( A VOPE is an evaluation association or society... voluntary organization for professional evaluation)

Workshop title:Establishing and strengthening VOPEs: testing and applying the EvalPartners Institutional Capacity Toolkit

Abstract: One of the EvalPartners initiatives, responding to requests received from leaders of many VOPEs (Voluntary Organizations for Professional Evaluation), is to develop a toolkit which provides guidance to those who wish to form even informal VOPEs, and leaders of existing VOPEs who seek guidance on strengthening their organization’s capacities.  During this workshop participants will be introduced to the many subjects addressed in the VOPE Institutional Capacity Toolkit, and asked to test the tools as they determine how they could help them apply such resources in strengthening their own VOPEs.

The workshop will be very interactive with lots of exploring, engaging, and evaluating of the toolkit resources. Participants should not come to this workshop expecting that they will sit still for more than 30 minutes at a time. We'll use a combination of learning stations and fishbowls as the workshop methodology.  I'm really looking forward to it!

Eventually the toolkit will be made available online. Follow @vopetoolkit on twitter for more news about developments.

I served on the boards of both AfrEA and SAMEA so I hope that the resources that the Toolkit task force and their marvellous team of collaborators put together in the toolkit will be of use to colleagues across the continent who are still founding or strengthening their VOPEs. It is hard and sometimes thankless work to serve on a VOPE board, and if this toolkit can make someone's life a little easier with examples, tools and advice, I would count this as a worthy effort.

I expect that the workshop will be a good opportunity to get some Feedback to guide us in the completion of the work.

Working Rigorously with Stories - Impact Story Tool

I've had some people email me about a paper I presented at the 2013 SAMEA conference. This paper introduces a tool for collecting and rigorously analysing impact stories that could be used as part of an evaluation. The full paper with the tool can be accessed here. The abstract is presented below:

 Beneficiary stories are an easily collected data source, but without specific information in the story, it may be impossible to attribute the mentioned changes to an intervention or to verify that the change actually occurred. Approaches such as Appreciative Inquiry and the Most Significant Change Technique have been developed in response to the need to work more rigorously with this potentially rich form of data. The “Impact Story Tool” is yet another attempt to make the most of rich qualitative data and was developed and tested in the context of a few programme evaluations conducted by Feedback RA.

The tool consists of a story collection template and an evaluation rubric that allows for the story to be captured, verified and analysed. Project participants are encouraged to share examples of changes in skills, knowledge, attitudes, motivations, individual behaviours or organizational practice. The tool encourages respondents to think about the degree to which the evaluated programme contributed towards the mentioned change, and also asks for the details of another person that may be able to verify the reported change. The analyst collects the story, verifies the story and then codes the story using a rubric. When a number of stories are collected in this way, they are then analysed together with other evaluation data. It may illuminate which parts of a specific intervention are most frequently credited with contributing towards a change.

Besides introducing the tool as it was used in three different evaluations, the usefulness of this tool and possible drawbacks are discussed.

 (The picture above is of a character known as "Benny Bookworm" from a South African TV show called "Wielie Walie" which I watched as a child)

Reflection: Thanks Tom Grayson

So since I started a stroll down memory lane, I thought I'd share this too. In 2002 I decided I wanted to be an evaluator. I was working at an evaluation company, and I decided to start my own consultancy, so I was getting great practical exposure.  But I really did not have a good academic grounding in the theory and literature surrounding Evaluation. This was back in the day when there weren't MOOCS and webinars... so I had to READ to get my education.

During my studies I had read Cook and Campbell, and somehow I also stumbled upon Guba and Lincoln. I was introduced to Utilization Focused Evaluation.  In 2004 I got Rossi, Lipsey and Freeman for a going away present from Khulisa, and I read any evaluation journal articles I could lay my hands on.

Its after reading something that Tom Grayson (from the University of Illinois at Urbana-Champaign) wrote in a journal article about teaching evaluation, that I decided to email him. I asked him for some reading material that will give me a good basis in Evaluation. He responded by sending me a package of course reading materials via post... This was such an unexpected gesture of goodwill. Above is a little handwritten note that he sent with the material.
 
So Tom, thanks a lot. And this is me letting you know about my adventures in evaluation!

Reflection: I decided to become an evaluator in May of 2002

As I was packing up the FeedbackRA office, I found a few files that I needed to clear out. This one is special, because it was during this workshop that I chose to relate to the identity of Evaluator.

It is a file for a workshop titled "Evaluation for Development: An Advanced Course in Evaluation". It was presented by Michael Quinn Patton in Pretoria in 2002, and it was arranged by Zenda Ofir from Evalnet.

  

My academic training in the field of Research Psychology meant that I was comfortable with research methods, but I also wanted to be involved in Development...  I didn't have a good idea of what I wanted to do with the research skills, and frankly, before joining Khulisa I had never heard of evaluation as a career option. The Community Psychology training I did at Honours and Masters level resonated deeply with me. Previously I had thought that I wanted to be a project officer at an NGO or international development organization, but I also realised that I like doing the research. I think that after about a year's working experience I started to think of myself as a researcher. 

I will forever be grateful for the experience I got at Khulisa Management Services, and the fact that Jennifer Bisgard let me go to this workshop in 2002. Thanks to Zenda Ofir for arranging it, and thanks for Michael Quinn Patton for preaching/teaching so convincingly.