My plans for AfrEA 2014 conference

I'm off to Cameroon on Sunday for a week of networking, learning and sharing at the 2014 AfrEA conference in Yaondé. I love seeing bits of my continent. If internet access is available I'll try to tweet from @benitaW.

I am facilitating a workshop on Tuesday together with the wise Jim Rugh and the efficient Marie Gervais to share a bit about a VOPE toolkit EvalPartners is developing. ( A VOPE is an evaluation association or society... voluntary organization for professional evaluation)

Workshop title:Establishing and strengthening VOPEs: testing and applying the EvalPartners Institutional Capacity Toolkit

Abstract: One of the EvalPartners initiatives, responding to requests received from leaders of many VOPEs (Voluntary Organizations for Professional Evaluation), is to develop a toolkit which provides guidance to those who wish to form even informal VOPEs, and leaders of existing VOPEs who seek guidance on strengthening their organization’s capacities.  During this workshop participants will be introduced to the many subjects addressed in the VOPE Institutional Capacity Toolkit, and asked to test the tools as they determine how they could help them apply such resources in strengthening their own VOPEs.

The workshop will be very interactive with lots of exploring, engaging, and evaluating of the toolkit resources. Participants should not come to this workshop expecting that they will sit still for more than 30 minutes at a time. We'll use a combination of learning stations and fishbowls as the workshop methodology.  I'm really looking forward to it!

Eventually the toolkit will be made available online. Follow @vopetoolkit on twitter for more news about developments.

I served on the boards of both AfrEA and SAMEA so I hope that the resources that the Toolkit task force and their marvellous team of collaborators put together in the toolkit will be of use to colleagues across the continent who are still founding or strengthening their VOPEs. It is hard and sometimes thankless work to serve on a VOPE board, and if this toolkit can make someone's life a little easier with examples, tools and advice, I would count this as a worthy effort.

I expect that the workshop will be a good opportunity to get some Feedback to guide us in the completion of the work.

Working Rigorously with Stories - Impact Story Tool

I've had some people email me about a paper I presented at the 2013 SAMEA conference. This paper introduces a tool for collecting and rigorously analysing impact stories that could be used as part of an evaluation. The full paper with the tool can be accessed here. The abstract is presented below:

 Beneficiary stories are an easily collected data source, but without specific information in the story, it may be impossible to attribute the mentioned changes to an intervention or to verify that the change actually occurred. Approaches such as Appreciative Inquiry and the Most Significant Change Technique have been developed in response to the need to work more rigorously with this potentially rich form of data. The “Impact Story Tool” is yet another attempt to make the most of rich qualitative data and was developed and tested in the context of a few programme evaluations conducted by Feedback RA.

The tool consists of a story collection template and an evaluation rubric that allows for the story to be captured, verified and analysed. Project participants are encouraged to share examples of changes in skills, knowledge, attitudes, motivations, individual behaviours or organizational practice. The tool encourages respondents to think about the degree to which the evaluated programme contributed towards the mentioned change, and also asks for the details of another person that may be able to verify the reported change. The analyst collects the story, verifies the story and then codes the story using a rubric. When a number of stories are collected in this way, they are then analysed together with other evaluation data. It may illuminate which parts of a specific intervention are most frequently credited with contributing towards a change.

Besides introducing the tool as it was used in three different evaluations, the usefulness of this tool and possible drawbacks are discussed.

 (The picture above is of a character known as "Benny Bookworm" from a South African TV show called "Wielie Walie" which I watched as a child)

Reflection: Thanks Tom Grayson

So since I started a stroll down memory lane, I thought I'd share this too. In 2002 I decided I wanted to be an evaluator. I was working at an evaluation company, and I decided to start my own consultancy, so I was getting great practical exposure.  But I really did not have a good academic grounding in the theory and literature surrounding Evaluation. This was back in the day when there weren't MOOCS and webinars... so I had to READ to get my education.

During my studies I had read Cook and Campbell, and somehow I also stumbled upon Guba and Lincoln. I was introduced to Utilization Focused Evaluation.  In 2004 I got Rossi, Lipsey and Freeman for a going away present from Khulisa, and I read any evaluation journal articles I could lay my hands on.

Its after reading something that Tom Grayson (from the University of Illinois at Urbana-Champaign) wrote in a journal article about teaching evaluation, that I decided to email him. I asked him for some reading material that will give me a good basis in Evaluation. He responded by sending me a package of course reading materials via post... This was such an unexpected gesture of goodwill. Above is a little handwritten note that he sent with the material.
 
So Tom, thanks a lot. And this is me letting you know about my adventures in evaluation!

Reflection: I decided to become an evaluator in May of 2002

As I was packing up the FeedbackRA office, I found a few files that I needed to clear out. This one is special, because it was during this workshop that I chose to relate to the identity of Evaluator.

It is a file for a workshop titled "Evaluation for Development: An Advanced Course in Evaluation". It was presented by Michael Quinn Patton in Pretoria in 2002, and it was arranged by Zenda Ofir from Evalnet.

  

My academic training in the field of Research Psychology meant that I was comfortable with research methods, but I also wanted to be involved in Development...  I didn't have a good idea of what I wanted to do with the research skills, and frankly, before joining Khulisa I had never heard of evaluation as a career option. The Community Psychology training I did at Honours and Masters level resonated deeply with me. Previously I had thought that I wanted to be a project officer at an NGO or international development organization, but I also realised that I like doing the research. I think that after about a year's working experience I started to think of myself as a researcher. 

I will forever be grateful for the experience I got at Khulisa Management Services, and the fact that Jennifer Bisgard let me go to this workshop in 2002. Thanks to Zenda Ofir for arranging it, and thanks for Michael Quinn Patton for preaching/teaching so convincingly.

So long FeedbackRA, and thanks for all the fish!

Below is a picture of myself, my business partner, and our spouses on the day we moved into our offices in April 2006. And then a picture of the people in the Feedback Offices today, for one last time (Terence is away in Rwanda on a jobbie).

Today is a little bit of a sad day for me. We’re packing up the FeedbackRA offices which have been the place where I practiced my profession for the past eight years. I'm starting at my new offices tomorrow, and it means that my relationship with FeedbackRA is one step closer to dissolving further.

The timeline:
I started the business in 2002 together with two colleagues. Our first job was a survey we did for MTN. Our first evaluation job was for the Gauteng Education Development Trust.

In 2004 I quit my fulltime job to earn my own salary at Feedback. In this year we did a nice piece of research for the Department of Science and Technology, and started expanding our CSI business base.

In 2006 we graduated to proper offices.  We appointed our first employees and embarked on a range of projects that just saw the business  grow – in terms of its focus, our skill and the turnover.

 In 2007 and 2008 we did a strategic piece of research on JIPSA, and I got to interview ministers and captains of industry. In this time I was also involved with setting up SAMEA and in 2009 I started to contribute to AFREA too.

In September 2009 I took a step back to reflect on the important things in my life. Up and ‘til 2009 I managed the business and business finances, and delivered as a key consultant, while keeping an eye on things at SAMEA and AFREA. I just couldn’t do it all any longer.

In 2010 I scaled down on my work and volunteer responsibilities, and we got Daleen involved to help us run the business. My life was much simpler after that… but it was still tough, because the business kept on growing. I did some lovely work with colleagues from Stellenbosch on a Public expenditure tracking survey… and I learnt so much from being the “junior” on the team.

In 2010 we started negotiations with a range of other high-profile consultants to see if they would like to join as business partners. We worked together on a few projects, we had a strategic planning session… everything looked good.  Our business was expanding and we decided to take up more office space. We started two big contracts which allowed us the scope to do some longer term planning, but it also opened the business up to risk… because things did not always go according to plan.

In 2011 an advertisement for the only other job I ever thought I’d consider, crossed my desk. I decided to apply, and let my business partners know. We let the conversation with the other potential business partners cool down a bit. After 6 months in limbo, I found out that I did not get the other job, but it took only 1 month for me to realise that what I’ve gained at Feedback was significant enough for me to call it quits. The stress of managing a business and a full consulting plate was just too much. I took on one too many assignments – because a colleague that I respect a lot twisted my arm.  This had bad consequences for me as an individual. I didn’t do my best work any longer.  I wanted to do things well again… I wanted to focus on things that I could do well, instead of just taking on jobs to make sure cash flow was sorted. A week after my decision, I found out I was going to be a mom. So suddenly I had another reason to reevaluate my priorities.

I sold my interest in Feedback RA in September 2011, and handed over all management responsibility to my colleagues. I worked as a consultant at FeedbackRA until April 2012, and then returned on a part time basis on selected assignments from October 2012. In this time I realised that I really preferred working on Education issues, and the area of ICT for Education became my core focus.

I continued working at Feedback until October 2013, when I decided to start my own consultancy again – Benita Williams Evaluation Consultants. I still helped out with some of the FeedbackRA work, and by January 2014 I was able to take over some of the FeedbackRA staff.

Tomorrow will be the first day in our new offices, which is quite exciting. But there is a  side of me that is really sad and nostalgic for what was.

Thanks colleagues, collaborators, clients and friends. Thanks to my family. It is the end of an era, so.... "So long, and thanks for all the fish!"