True Confessions of an Economic Evaluation Phobic

You know how the forces at work in the universe sometimes conspire and confronts you with a persistent nudge... over an over again? Well this week's nudge was "You know nothing about economic evaluation... do something about it - Other than ignoring it".

Words like "cost-benefit analysis, cost-efficiency analysis, cost-utility analysis"... actually anything with the word "cost" or "expenditure" in it... makes me nervous. So my usual strategy is to ignore the "Efficiency" criterion suggested by the OECD DAC, or I start fidgeting around for the contact details of one of my economist friends, and pass the job along. I have even managed to be part of a team doing a Public Expenditure Tracking Survey without touching the "Expenditure" side of the data.

But then I found these two resources that helped me to start to make a little bit more sense of it all. They are:

The South African Department of Planning Monitoring and Evaluation's Guideline on Economic Evaluation  At least it starts to explain the very many different kinds of economic evaluation you should consider if you work within the context of South Africa's National Evaluation Policy Framework.


And then this. 

A free ebook by Julian King that presents a short theory for helping to answer the question "Does XYZ deliver good (enough) value for investment?" - Essentially the question any evaluator is supposed to help answer.

So, now, there is one more topic on my ever expanding reading list! If there is a "Bible" of economic evaluation, let me have the reference, ok?

22 Approaches for Raising Student Achievement

I’m working my way through a book by Stuart S. Yeh Entitled “The Cost-Effectiveness of 22 Approaches for Raising Student Achievement” (Also available as an ebook http://ow.ly/4R62s or on paper from www.loot.co.za see: http://ow.ly/4RRll)

The book (based on Studies in the States) concludes that:

The review of cost-effectiveness studies suggests that rapid assessment is more cost effective with regard to student achievement than comprehensive school reform (CSR), cross-age tutoring, computer-assisted instruction, a longer school day, increases in teacher education, teacher experience or teacher salaries, summer school, more rigorous math classes, value-added teacher assessment, class size reduction, a 10% increase in per pupil expenditure, full-day kindergarten, Head Start (preschool), high-standards exit exams, National Board for Professional Teaching Standards (NBPTS) certification, higher teacher licensure test scores, high-quality preschool, an additional school year, voucher programs, or charter schools

.

I find this interesting, because this makes a very compelling argument for using computer based learning in schools. The first chapter of the book presents a nice theoretical overview that indicates how kids become disheartened if they don’t consistently have mastery experiences. If assessment and teaching can be individualized so that each learner progressively improves compared to their previous performance (rather than a comparison with peers) they are likely to feel that they are in control of their learning, and they would be more likely to stay engaged in the learning process. The chapter states:

A theory of learning may be deduced: Individualization of assessment, task difficulty and performance expectations for each student on a daily basis, in combination with performance feedback, autonomy in task execution and an accelerating standard of performance, ensures that students achieve success and feel successful on a daily basis, fostering student engagement, increased effort, and further improvements in achievement in a virtuous cycle.

Software can automate the provision of corrective feedback, and assigning of content (on a daily basis) and this has been shown to have powerful effects on learning and achievement. Yeh reports that in a study of Math Assessment, (involving 1,880 students in grades 2 through 8, 80 classrooms and seven states) they found an effect size of 0.324 Standard deviations over a 7 month period. This is a huge increase in learner performance.

Of course we need to remember that this is based on the American school system that is very different from ours. And note the study is about "Cost Effectiveness". It is not saying that the other strategies are not effective - In their context the other strategies were effective but at a higher cost than Rapid Assessment

Of course there are a few other assumptions that will have to be checked if a similar intervention is implemented in South Africa:
1) Computer infrastructure, technical support, and teacher’s abilities must be supportive of successful implementation. It is a lot harder to implement an ICT based project than one might imagine at the outset.
2) Rapid assessment cannot replace the role of the teacher – It can help learners improve, but they still have to get quality tuition from a qualified teacher.
3) Learners’ ability to interact with the software must not be blocked by poor reading / language capabilities.