Interesting Initiatives That May Help Us Make Research and Evaluation More Visible

A recurring challenge in education is that we do not have a good view of what research and evaluation is happening, where it is happening, and what has already been learned. Yet we are not starting from scratch. Several existing initiatives already hold different pieces of this puzzle. The opportunity may be less about creating another repository and more about exploring how these pieces could connect.

1. NED Connect – linking interventions to the research and evaluation around them

What exists: NED Connect, developed and stewarded by NASCEE, is building a national picture of who is doing what, where, and with whom in the education ecosystem. It includes organisation profiles, geographic information and interactive maps, including school-level mapping of reported support.

The possibility: Could NED Connect include an additional research and evaluation layer? An intervention could be tagged to indicate that research or evaluation is planned, underway or completed, with basic metadata and links to reports or other evidence. We could then begin to see not only where interventions are happening, but also where evidence is being generated about them.

2. AfrED – building on an existing repository of African evaluation evidence

What exists: CREST's African Evaluation Database (AfrED) already captures bibliographic information about evaluation reports, journal articles and doctoral theses. It currently contains around 7,800 documents, including more than 4,000 evaluation reports, and is specifically intended to make African evaluation production more visible.

The possibility: Rather than creating another repository for completed evaluation reports, could we strengthen the connection between AfrED and other education platforms? NED Connect might show that an intervention has been evaluated, while AfrED could provide the searchable evidence record. Over time, this could create a much clearer pathway from intervention → evaluation → evidence.

3. DDD – connecting evaluation with longitudinal system data

What exists: Data Driven Districts (DDD), a partnership involving the DBE and New Leaders Foundation, turns SA-SAMS data into accessible information for education officials. It currently incorporates data on around 12 million learners across approximately 24,000 schools, including information on attendance and performance.

The possibility: DDD raises a different opportunity. Could evaluation information be linked to the rich longitudinal information already available about schools and learners? Knowing which interventions and evaluations have taken place in a school, alongside its administrative and performance trajectory, could open new possibilities for secondary analysis, longer-term follow-up and new evaluation designs. It may also allow us to learn more from data already being collected rather than repeatedly creating parallel datasets. And here is a novel idea - Why don't we set up the system so that the schools can also use the evaluation and research output? 

4. A more connected ethics and research approval process – making planned research visible earlier

What exists: Research and evaluation currently passes through multiple processes: funders and researchers commission work; ethics bodies review studies; and provincial education departments manage research approvals. Each process has a legitimate purpose, but they are largely separate. No single actor therefore has a complete view of the research and evaluation landscape.

The possibility: Could a central or coordinated education ethics review function become another point of connection? It might draw basic metadata from provincial approval processes and ethics applications, identify possible overlaps, and trigger follow-up where appropriate: Another evaluation is already taking place in these schools. Similar data have recently been collected. Have the two teams spoken? Is there existing evidence that should be considered? The aim would not be another layer of bureaucracy, but a light-touch coordinating function that helps the system notice connections before data collection begins. This builds on the idea of pre-ethics review, a research and evaluation registry, and a small enabling coordination function already explored in the Weave concept.

5. The Zenex–CREST meta-review – showing the value of bringing evaluation evidence back together

What exists: The Zenex Foundation has recently supported CREST to undertake a meta-review of selected evaluations commissioned by the Foundation since 2015, across Foundation, Intermediate and Senior Phase work. Rather than looking at another individual programme, the exercise brings together a body of evaluation evidence accumulated over roughly a decade and examines it systematically.

The possibility: This illustrates why visibility matters. Individual evaluations answer important questions about individual programmes. But when we can find, organise and synthesise evaluations across time, another level of learning becomes possible: What findings recur? Which approaches appear promising across contexts? Where does evidence disagree? What have we repeatedly tried? And what important questions remain unanswered?

---

The bigger opportunity

These initiatives do different things, and they should not necessarily be combined into one large system. But together they suggest the beginnings of an evidence infrastructure:

NED Connect helps us see what is happening where.
Provincial approvals and ethics processes could help us see what research is about to happen.
AfrED can help us find what research and evaluation has already been completed.
DDD provides longitudinal system data that can potentially deepen what we can learn from evaluations.
Meta-analysis and synthesis allow us to step back periodically and ask what the accumulated evidence is telling us.

The opportunity may therefore not be to build one more database. It may be to make the connections between what already exists — so that each new research or evaluation investment contributes not only to an individual programme, but progressively to what the education system knows.

AI disclosure: I used ChatGPT to help structure, refine and edit this post, based on my own ideas, experience and source material. I reviewed and take responsibility for the final content. 

We have a visibility problem in our education research ecosystem

 One of the challenges in education research and evaluation is surprisingly basic: we don't really know what is happening.

At any given time, universities, government departments, foundations, NGOs and evaluators are conducting research and evaluations across the education system. Schools are participating in studies. Learners are being assessed. Teachers are completing surveys and being observed. Programmes are generating monitoring data. Reports are being produced.

But it is remarkably difficult to get a picture of what research and evaluation is happening where, what has already been completed, what data have already been collected, and what we have learned from it all.

This matters for very practical reasons. A funder may commission an evaluation without knowing that somebody else has investigated a similar question. A researcher may collect data that already exist elsewhere. A school may participate in several studies while another comparable school participates in none. And an evaluation may produce useful findings that live on a funder's website—or in someone's Google Drive—without becoming part of the evidence available to the wider system.

The problem, then, is not simply that research isn't disseminated well enough. We have a visibility problem across the research and evaluation lifecycle. We struggle to see what is planned, what is underway, what has been completed, what evidence and data already exist, and how individual pieces of work relate to one another.

That makes it much harder to move from lots of individual evaluations to cumulative learning about the education system.

The interesting thing is that we may not need to build a giant new repository to solve this. Several pieces of the infrastructure already exist, or are beginning to emerge. The question I've been thinking about is whether we could connect them differently. In the next post I share some ideas about how we can make research and evaluation more visibile. 

AI disclosure: I used ChatGPT to help structure, refine and edit this post, based on my own ideas, experience and source material. I reviewed and take responsibility for the final content.

A potentially important shift in teacher development

Some exciting news from the DBE/VVOB 2026 Teacher Development Colloquium today: the DBE launched its new Online Teacher Development Platform.

Teachers can access targeted courses and microcredentials, build their own learning pathways, and have completed learning linked directly to SACE CPTD points. External providers can also link their training to the platform.

Very exciting.

Of course, we should not expect miracles. Internet access and technology will remain barriers. Some people may game the system for points. Completion does not automatically mean learning, and learning does not automatically translate into classroom practice.

But I think it would be hard to argue that this is not a step up. In many places, subject advisors simply cannot reach every teacher. And even where teachers attend workshops, we often know surprisingly little about how much of the training they actually engaged with. Sometimes even tracking who completed most of the programme is difficult.

So this is potentially more than another online platform. It could change some of the infrastructure of teacher professional development.

The next question is how we support it well. Districts often do not have the ICT support capacity that a system like this will need. Could funders and partners help strengthen that public infrastructure, rather than build yet another parallel programme?

There are so many possibilities. I’m excited to see where the next two days lead us.

Analytical approaches to establish causality

Over the past week, I taught the Impact Evaluation module for the MPhil programme in Evaluation at CREST. That meant Impact Evaluation was on my mind all week, and I got to dig into some analytical approaches behind different traditions. 

 What I tried to get across: 

There is sometimes an assumption in impact evaluation that the most credible way to establish whether a programme had an effect is an experiment or quasi-experiment. These are powerful approaches. But they represent one way of building a causal argument. There are others.

A dice provides a surprisingly simple way of seeing the difference.

Let's start with a dice

Suppose I throw a fair six-sided dice five times.

Before I start, I can calculate the probability of getting exactly three sixes. It is about 3.2%.

Then I start throwing.

My first throw is a six.

Suddenly my chances of reaching exactly three sixes have changed. I now need two more sixes in the remaining four throws. The probability of that happening is about 11.6%.

Nothing about the dice has changed. I simply know something I didn't know before.

This is conditional probability: given what I now know, what is the probability of a particular outcome?

But let's use the same dice to think about three rather different ways of reasoning with evidence.

1. How surprising is what I observed?

I throw the dice five times and get three sixes.

Let's start by assuming that nothing unusual is going on. It is an ordinary fair dice.

Probability theory tells me what kinds of results I should expect if that assumption is true.

So I can ask:

If this really is a fair dice, how surprising is it that I got three sixes in five throws?

Quite surprising. Exactly three sixes would happen only about 3.2% of the time.

The reasoning goes something like this:

Assume nothing unusual is happening

↓

Work out what we would expect to observe under that assumption

↓

Compare what actually happened with that expectation

↓

Ask how surprising our observed result is

This is the intuition behind frequentist statistical testing.

Importantly, I am not asking, "What is the probability that my dice is fair?"

I am asking:

"If my dice is fair, how compatible is what I observed with that assumption?"

That distinction becomes important when we move into impact evaluation.

2. Given what I have seen, what should I now believe?

Now let's change the dice problem slightly.

This time, I don't know whether the dice is fair.

Before I throw it, I think it probably is. Most ordinary dice are.

Then I throw five times and get three sixes.

Hmm.

That doesn't prove that the dice is loaded. A fair dice can produce three sixes.

But perhaps I am now a little more suspicious than I was before.

So I throw another ten times. Suppose sixes continue appearing unusually often.

I become more suspicious.

I throw another twenty times and the pattern continues.

At each point I am asking:

Given what I believed before, and the evidence I have now observed, what should I believe now?

That is the basic intuition behind Bayesian reasoning:

What I believed before + new evidence → what I believe now.

And different pieces of evidence can change my confidence by different amounts.

Someone saying, "That dice looks funny", might shift my belief slightly.

Discovering that it came from a box labelled "Trick Dice" would shift it rather more.

Discovering that it has been deliberately weighted would shift it enormously.

So it isn't simply a question of accumulating lots of evidence.

We can also ask:

How much should this particular piece of evidence change my confidence in an explanation?

3. What combination produces the outcome?

Now let's change the game again.

Suppose there are several ways to win.

Perhaps:

two sixes + a five = WIN

but:

one six + three fives = WIN

also works.

And perhaps, after watching hundreds of games, we notice that nobody ever wins without at least one six.

Now I am asking a different kind of question.

Not:

"How probable is winning?"

but:

"What needs to come together for winning to happen?"

Perhaps there are several different combinations that are enough to produce a win.

Perhaps one condition is necessary but isn't enough on its own.

That is the basic intuition behind set-theoretic reasoning and approaches such as Qualitative Comparative Analysis (QCA).

So our dice has given us three questions:

Frequentist: If nothing unusual is happening, how surprising is what I observed?

Bayesian: Given what I believed before, how should the evidence I have observed change what I believe now?

Set-theoretic: What combination of conditions needs to come together to produce the outcome?

Now let's replace the dice with an education programme.

An education programme

Imagine we introduce a literacy programme into 100 schools.

The programme provides teacher training and ongoing coaching, with the intention that these improve classroom teaching and ultimately children's reading.

After two years, children in the programme schools have improved their reading scores by 10 points.

Excellent.

But have we established impact?

Not yet.

Children might have improved anyway. They are two years older. Teachers have gained experience. Perhaps government introduced new reading materials. Perhaps another education initiative was operating in the same districts.

So we need some way of asking:

What would probably have happened without our programme?

The experimental or quasi-experimental approach

Suppose we have a credible group of similar schools that did not receive the programme.

Children in those schools improved by 4 points.

Our programme schools improved by 10 points.

That gives us an estimated programme effect of 6 points.

This is where counterfactual reasoning gives experimental and quasi-experimental designs their power.

We have tried to construct a credible estimate of what would have happened to programme participants in the absence of the programme.

But there is still uncertainty.

Even if the programme had absolutely no effect, we would not expect two groups of children to produce precisely identical scores. There is ordinary variation in the world and in our samples.

And this is where we can return directly to our dice.

With the dice we asked:

If the dice really is fair, how surprising is it that I observed three sixes?

In the education evaluation we can ask:

If the programme really had no effect, how surprising would it be to observe a difference of 6 points or more between these groups?

The logic is the same:

Assume there is no programme effect

↓

Work out the kinds of differences we would expect under that assumption

↓

Compare our observed 6-point difference with that expectation

↓

Ask how surprising our result would be if there really were no effect

This is the frequentist statistical reasoning commonly used alongside experimental and quasi-experimental impact evaluations.

There is an important distinction here.

The counterfactual and the statistical test are not the same thing.

The experimental or quasi-experimental design helps us construct the causal comparison: what would probably have happened without the programme?

Frequentist statistical inference is one way of assessing the uncertainty around the effect we estimate from that comparison.

Indeed, an RCT could be analysed using Bayesian statistics instead.

But now suppose we cannot construct a credible counterfactual.

Does that mean we can say nothing rigorous about whether the programme contributed to the observed improvement?

I don't think so.

Contribution analysis and process tracing: How confident are we in the causal explanation?

Suppose the programme was deliberately introduced into the weakest districts.

Schools received different amounts of support. Government introduced a new reading policy at the same time. Other organisations were working in some of the same schools.

Finding a genuinely comparable group may be extremely difficult.

But we have a theory about how our programme was supposed to produce change:

Training

↓

Improved teacher knowledge

↓

Coaching helps teachers put that knowledge into practice

↓

Classroom teaching changes

↓

Children's reading improves

Now we can investigate that explanation.

Did teachers actually learn the intended practices?

Did their classroom practice subsequently change?

Were the changes the particular practices targeted by the programme?

What role did coaching play?

Did teachers receiving stronger coaching change more?

Did the aspects of children's reading that should have responded to the changed teaching actually improve?

Did the sequence of changes happen in the order our theory predicts?

And what else could explain what we observed?

Perhaps the government policy is the real explanation. Perhaps another literacy programme was responsible. Perhaps reading improved everywhere.

Each piece of evidence can strengthen or weaken our confidence in our explanation.

This takes us back to our possibly loaded dice.

We are asking:

Given everything we have now observed, how confident should we be in this particular explanation of what happened?

That kind of reasoning has a strong affinity with process tracing and, somewhat more loosely, contribution analysis.

We aren't pretending we have constructed a counterfactual when we haven't.

But neither are we simply observing that outcomes improved and claiming credit.

We are systematically testing the causal story and competing explanations.

QCA and realist approaches: Under what conditions does it work?

Now suppose something else happens.

Our average result hides enormous variation.

Some schools improve dramatically. Some improve a little. Others don't improve at all.

Perhaps the interesting question isn't simply:

"Did the programme work?"

Perhaps it is:

"What needs to come together for the programme to work?"

When we look across schools, we might discover one pathway to success:

Good coaching + strong school leadership + adequate materials → improved reading

But another group of successful schools might have:

Good coaching + strong teacher collaboration + district support → improved reading

There may be more than one route to the same outcome.

This is the territory of QCA and set-theoretic reasoning.

Realist evaluation approaches the problem somewhat differently. It asks us to investigate:

What works, for whom, in what circumstances, and why?

Both draw our attention to something that an average programme effect can easily obscure:

Context matters, and programmes may produce outcomes through different pathways in different circumstances.

So which one establishes impact?

This is where I think we sometimes get ourselves into trouble.

We talk about the "strongest" evaluation design as though all impact evaluations are trying to answer exactly the same causal question.

They aren't.

A good experimental or quasi-experimental evaluation can provide powerful evidence that:

People exposed to the programme experienced better outcomes than comparable people who were not exposed to it.

Contribution analysis or process tracing can investigate:

How confident are we that the programme contributed to the observed change through the causal processes we expected, rather than through plausible alternative explanations?

QCA can investigate:

Which combinations of programme and contextual conditions are associated with the outcome, and are there different pathways to success?

A realist evaluation can investigate:

What works, for whom, under what circumstances, and through what mechanisms?

These approaches do not make identical causal claims.

And that is the point.

If what we really need is a credible estimate of the difference an intervention made compared with what would otherwise have happened, then a good experimental or quasi-experimental design may be exactly what we need.

But sometimes that isn't the most important question.

Sometimes we need to understand whether and how a programme contributed to change in a complex environment.

Sometimes we need to know why it worked in 12 schools and not in eight others.

Sometimes we need to understand which combination of programme support and contextual conditions produces success.

And sometimes we need several of these answers.

So perhaps the first question in designing an impact evaluation shouldn't be:

"Can we do a quasi-experiment?"

It should be:

"What exactly do we need to know about whether, how, for whom and under what circumstances this programme made a difference?"

Then we can decide what kind of evidence — and what kind of causal reasoning — gives us the most credible answer.

AI disclosure: I used ChatGPT as a thinking and writing aid while developing this post — to explore the distinctions between the different approaches, test and refine the examples, check my explanations, and help edit the final text. The argument, interpretation and final choices are my own.

AI made me worry about transitions

I am worried about AI. 'Cause researchers wrote a scary paper with scary words like "autonomous replication and adaptation".

I'm worried about increasingly capable systems becoming more autonomous than we intended. I'm worried about what bad actors might do with them. I'm worried about the concentration of extraordinary power in a handful of companies and countries. And I'm also worried about the less spectacular possibility that we are living through an enormous technology and investment bubble which bursts.

I don't know which of these risks is most likely. Nobody really does.

But thinking about AI led me to a slightly different question.

What do we do when we know a system is changing, but we cannot know what it is changing into?

Because AI isn't the only place where this feels relevant.

We are seeing profound changes in how philanthropy and development are funded. Governments are under fiscal pressure. Traditional development aid is shifting and drying up. Foundations are reconsidering their roles. Organisations are having to rethink sustainability. In South Africa, we are simultaneously worrying about institutional capacity, inequality, social cohesion and whether our democratic institutions are resilient enough to withstand future shocks.

Different transitions, certainly. But they pose a similar problem.

We can identify futures we would prefer. We can work towards them. But none of us controls the system well enough to produce them.

So where is our agency?

I have found myself increasingly attracted to a fairly simple answer:

Perhaps we can seed the transition with patterns that we would like the future system to have available.

---

2. What does it mean to seed a transition?

Consider a donor deciding to put substantial money behind an issue.

The obvious intervention is the money. It allows organisations to do things.

But something else happens too.

By deciding that this issue matters, the donor helps set an agenda. Other organisations pay attention. Researchers become interested. Government officials engage. Other funders may follow. Of course, agenda setting is an exercise of power. A funder can seed patterns we think are desirable, but it can just as easily reinforce patterns we don't. The point isn't that seeding is inherently good. It is that it happens — and perhaps we should be more deliberate about it.

Now suppose the donor does something more deliberate. Instead of funding a collection of separate projects, it creates incentives for collaboration. Organisations share information, work across their usual boundaries and collectively make sense of what they are learning.

And then suppose the donor introduces another practice: system-wide sensing.

The collaborators don't only monitor whether their projects are delivering. They continually look outward.

What is changing in the wider system? Where are new actors appearing? What are communities experiencing? Where is resistance emerging? Where are unexpected opportunities? What are other organisations learning? What are we not seeing?

The initiative itself may eventually end.

But imagine that some of those practices remain.

Another funder starts convening its partners in the same way. An NGO takes the sensing practice into another programme. A government department adopts a similar mechanism. People who met through the collaboration continue working together.

What has been replicated isn't necessarily the programme.

It is a pattern:

focus attention → bring different actors together → collaborate around a shared problem → sense what is happening across the system → learn and adapt.

That is what I mean by a seed.

And it made me wonder: could deliberately creating good patterns be a sensible strategy when we don't know what the future system will look like?

There is actually quite a lot of theory suggesting that it might be.

---

3. Finding agency without pretending we have control

The idea that first helped me was path dependence.

The basic proposition is intuitive: where a system can go next is partly shaped by the path it has already travelled.

A new system doesn't arrive on a blank page. It inherits institutions, relationships, capabilities, technologies, habits, inequalities, values and ways of working.

That means what exists before and during a transition matters.

If collaboration already exists, collaboration is available to the emerging system.

If trusted networks exist, they are available.

If institutions have developed ways of sensing and adapting, those capabilities are available.

If everything has become fragmented, competitive and distrustful, those patterns are available too.

We are seeding future systems whether we intend to or not.

The interesting question is whether we can become more deliberate about which patterns we strengthen.

THEORY BOX: Path dependence

Social systems have histories. Once particular institutions, technologies, rules or behaviours become established, they can reinforce themselves and make some future trajectories easier than others.

The future isn't predetermined. But it doesn't begin from scratch either.

Why it matters: If today's patterns influence tomorrow's possible pathways, then creating and strengthening desirable patterns now can matter even when we cannot predict the eventual outcome. 

        Pierson, P. (2000). “Increasing Returns, Path Dependence, and the Study of Politics.” American             Political Science Review, 94(2), 251–267. 

---

4. What kind of patterns do we want?

This is where I find the latest Indlulamithi South Africa Scenarios 2035 particularly useful.

They give us three possible South African futures through wonderfully evocative bird metaphors.

There is Hadeda Home: the Recrimination Nation. Noisy, anxious, defensive and fragmented.

There is Vulture Culture: the Desperation Nation. A darker trajectory of institutional deterioration, insecurity and capture.

And there is Weaver Work: the Cooperation Nation.

I love the weaver metaphor.

A sociable weaver doesn't become resilient because one exceptionally powerful bird builds the perfect nest. Many birds contribute to a shared structure. The collective structure, in turn, gives them greater resilience.

That raises an interesting question.

If we want a more cohesive and resilient society, what Weaver-like patterns should we be putting into our institutions now?

Collaboration across institutional boundaries.

Relationships that survive individual projects.

Institutions that can learn and adapt.

Ways of sharing information without pretending that there is only one interpretation of it.

Collective mechanisms for sensing what is happening across a system.

Participation that gives people genuine agency.

Accountability.

Ways of dealing with disagreement without retreating into our respective laagers.

Perhaps even the mundane practice of people who don't usually work together learning how to do so.

These aren't outcomes like social cohesion or resilience.

They are patterns from which cohesion and resilience might emerge.

---

5. Meadows: preserve the system's ability to evolve

This takes me to Donella Meadows.

One of the most powerful leverage points she identifies is a system's capacity to self-organise.

And self-organisation requires diversity and experimentation.

That suggests something slightly counter-intuitive for funders and evaluators.

In an uncertain transition, our job may not always be to identify the intervention that works, standardise it and scale it.

Sometimes we may need to maintain a sufficiently rich ecology of good possibilities.

Try things.

Protect experimentation.

Learn.

Allow different institutional arrangements to develop.

Keep alternatives alive.

In other words: seed.

THEORY BOX: Donella Meadows and self-organisation

Meadows argued that changing things like budgets and targets is relatively weak systems intervention. Much stronger leverage can come from changing information flows, rules, goals and the capacity of a system to self-organise.

Self-organisation depends partly on having diversity from which new structures and behaviours can emerge.

My takeaway: Don't only optimise the system we have. Preserve its ability to become something else.

        Meadows, D.H. (1999). Leverage Points: Places to Intervene in a System. Hartland, VT: The Sustainability Institute. 

          Abson, D.J. et al. (2017). “Leverage points for sustainability transformation.” Ambio, 46, 30–39. 

---

6. Transition theory: today's little experiments may be tomorrow's alternatives

Transition theory gives us another useful idea.

Established systems are hard to change because so many things hold them together: regulations, institutions, technologies, money, professional practices, skills and expectations.

But around their edges are smaller experiments — what transition scholars call niches.

Most remain small.

Some disappear.

But some survive long enough for people to learn from them, build networks around them and develop them into credible alternatives.

Then something disrupts the established system.

Suddenly, an alternative that previously looked marginal may become useful.

THEORY BOX: Transition theory

One influential approach distinguishes between:

Landscape: large external pressures and changes.

Regime: the established system of institutions, rules, technologies and practices.

Niches: spaces in which alternatives can develop.

When pressure destabilises the existing regime, sufficiently developed alternatives may suddenly have opportunities to spread.

Why it matters: A small experiment doesn't have to transform the current system to be worthwhile. It may be developing a pattern or capability that becomes important under different conditions.

        Geels, F.W. & Schot, J. (2007). “Typology of sociotechnical transition pathways.” Research Policy, 36(3), 399–417. An author manuscript is openly available through the University of Manchester repository 

Geels, F.W. (2026). “The Multi-level Perspective on Sustainability Transitions: Background, Overview and Current Research Topics.” In Introduction to Sustainability Transitions Research. Cambridge University Press. The whole volume is open access

---

7. Patton: transformation happens through connections

Michael Quinn Patton approaches this from an evaluation perspective through his idea of a Theory of Transformation.

A programme can reasonably have a Theory of Change:

If we do these things, through these mechanisms, we expect these outcomes.

But transformation doesn't work quite like that.

Nobody runs “the system”.

Transformation involves multiple actors, interventions, institutions and changes interacting across different levels.

And that brings Patton to an idea I find particularly useful: interconnectedness momentum.

Think back to our donor.

Perhaps the most significant thing it achieved wasn't its programme outcomes.

Perhaps it helped establish a new agenda.

Perhaps organisations that previously worked separately formed relationships.

Perhaps system-wide sensing became normal practice.

Perhaps those organisations started influencing other organisations.

Perhaps another funder adopted the approach.

Perhaps people carried those practices into completely different institutions.

The original intervention hasn't necessarily “scaled”.

The pattern has travelled.

THEORY BOX: Theory of Transformation

A Theory of Change asks how an intervention is expected to produce outcomes.

A Theory of Transformation asks how multiple changes across actors, organisations, sectors and levels might interact to contribute to a much larger transformation.

Nobody controls that transformation.

Patton therefore draws attention to interconnectedness momentum: whether growing connections and interactions between initiatives are creating momentum towards transformation.

        Patton, M.Q. (2021). “Evaluation Criteria for Evaluating Transformation: Implications for the Coronavirus Pandemic and the Global Climate Emergency.” American Journal of Evaluation, 42(1), 53–89 

That suggests a different set of evaluation questions.

Not only:

Did the intervention work?

But also:

What patterns did it introduce?

Which survived?

Who picked them up?

How did they change as they travelled?

What did they connect with?

Are those connections becoming self-sustaining?

Is something happening in the wider system that wasn't happening before?

---

8. This has to extend beyond evaluation — and beyond development

I started thinking about this because I was frightened about AI.

Then I recognised it in philanthropy.

But neither evaluation nor philanthropy is big enough.

If we really are living through a period of multiple overlapping transitions, patterns are being created everywhere: in schools, universities, families, businesses, government departments, communities, professional associations, civil society, technology companies and the media.

Some will reinforce Hadeda patterns.

Some may create Vulture patterns.

And some may be Weaver patterns.

That gives us a different way of thinking about agency.

We don't all have to be working on the same intervention.

We don't even have to agree on exactly what the future will look like.

But perhaps we can become better at recognising patterns worth carrying forward.

And then we can help them find each other.

---

9. There's a catch: we may have to loosen our grip on the outcome

There is something uncomfortable about all of this.

If I plant a seed because I need my seed to become the tree, complexity is likely to disappoint me.

Most experiments won't transform systems.

Some good ideas will disappear. Some will be absorbed into existing institutions and lose what made them interesting. Some will mutate. Someone else may take an idea and do something better with it.

And something that looks like failure today might unexpectedly become useful years from now.

That makes this difficult for those of us trained to specify outcomes and demonstrate contribution towards them.

Perhaps sometimes success needs to mean something more modest:

We introduced a pattern we thought was worth having in the system.

We learned from it.

We allowed other people to change it.

We connected it to other things.

And we paid attention to what happened next.

---

10. Finding the other weavers

I am still worried about AI.

And I'm worried about institutional decline, social fragmentation and some of the transitions happening in development and philanthropy.

Systems thinking doesn't give me reassurance that everything will work out.

But it does give me an alternative to the choice between controlling the future and being powerless before it.

We have agency without having control.

We can seed useful patterns.

We can protect experimentation.

We can strengthen institutions capable of learning.

We can notice when something interesting is emerging.

We can help useful patterns travel.

And, perhaps most importantly, we can connect.

Because if Patton is right about interconnectedness momentum, and transition theory is right about the importance of alternatives that already exist when systems begin to shift, then connection isn't something we do after we've figured out the solution.

Connection may itself be part of how the transition happens.

We don't know which seeds will grow.

We don't know what the system will select.

We don't know what South Africa — or the world — will look like on the other side of the transitions we are living through.

But we do have some agency over what we put into the system now.

Perhaps step one is finding the other weavers.

Image Source: https://indlulamithi.org.za/

A note on AI: There is a certain irony in writing this post with AI. I used ChatGPT as a thinking and drafting partner: to test the argument, connect it to theory, find sources, and help shape the writing. The ideas and examples I wanted to explore were mine; I made the substantive choices and checked the sources. I remain responsible for what is written here.