Wednesday, November 20, 2013

ICT4RED Learning Workshop

I'm contributing to the evaluation of the ICT4RED (Information Communication Technology for Rural Education Development) initiative - A very ambitious project that rolls out teacher professional development to enable teachers and learners to use 21st century methods and tablet computers in rural schools in Cofimvaba in the Eastern Cape. More information about the project here and here.

We decided to use a developmental evaluation approach - I'm practically embedded in the organization that's responsible for implementing the project. I'm finding that this is a wonderful opportunity to influence what happens... But because this project is so different from the many failed technology projects that I've evaluated before, I sometimes wonder whether I am "objective" enough to add actual value.

We organised the M&E team's work into four categories -
  • Monitoring - Measuring progress made on outputs, facilitating ocassional debrief meetings with team members, reflecting on abundant data from various social media streams and participating in weekly project management meetings
  • Evaluation - Measuring the success after various implementation phases - incorporating some self-evaluation workshops, and other more standard evaluation measures including ethnograpic descriptions, baseline and follow up surveys, and a small scale RCT and tracer study
  • Learning - Asking team members to ocassionally reflect on what theyve learnt - from successes or failures
  • Model Development - Developing a modular theory of action underpinned by a theory of change that can be used to support scale up and replication. 

 Yesterday we held a "learning workshop" where team members had to reflect on what they've learnt.
Our first template for the learning briefs asked for the following information:

Project Name
    Give the project name here
Submitted by
    Give the name and component name
    Give the date on which you submit the learning brief
What was the learning?
    Please describe the learning that occurred
Learning brief type
    Indicate which of the following three is applicable and provide a short description
    •    Learning from failure during implementation
    •    Learning from implementation success
    •    Learning from review of previous research and practice (i.e. not practically tested yet)
The Context
    Say something about the context of the learning / project context, add relevant pictures if they are available
Why is this learning important?
    Please describe why this learning is important
Evidence Base
    Please indicate what the evidence base is for this learning brief, if possible, provide references that may help the reader track down the evidence base.
Recommendations for future similar projects
    Please provide your recommendations in a list wise form. If possible add pictures, graphs or diagrams
Recommendations that should be taken into account by the current project
    Please indicate which of the above recommendations should be taken into account for the current project

My colleagues, who are great at packaging information, asked that we focus the learning presentations on
& What we designed
? What we learnt
# What we're doing now
! Advice for Policy and practice

This made for quite nice presentations. We'll probably adapt our learning brief templates accordingly.

Monday, November 11, 2013

Monitoring and Evaluation framework and design for SAESC - July 2013

In July 2013, I did a presentation on the M&E framework I helped to develop for the South African Extraordinary Schools Coalition.

Better Evaluation, OECD DAC Evaluation criteria, and Michael Quinn Patton's Developmental Evaluation all featured in there.