Stories of Change

‘Success’ isn’t clearly mappable through outcomes alone.

The very set up of the project centres ‘how’ something is done over ‘what’ is done — so how do you design evaluation within this approach — and make it work with the programme, rather than being an add on (or hindrance)?

Getting Started

I have to admit that when we found out that we had been selected as evaluators for Make/Shift, I responded by ringing Rachel and leaving an over-enthusiastic message about how excited I was.

This is because Make/Shift seems to me to be designed to have learnt from the best of the Creative People and Places experience in the past. That is, starting from scratch to really lean into the principle of taking time to get to know the place, building relationships with people, uncovering the assets in the communities and place.

This is what action research looks like.

Not being sure of the right way, trying things out and sometimes getting it wrong but learning from that. Taking time and building relationships can be nerve wracking as you don’t see the outputs immediately. You can’t ‘count’ the people reached (or you can but it isn’t many).

And it can seem even to those with a real commitment to an asset-based approach that they ‘haven’t really done anything’.

It has been a delight to be able to remind Rachel and the team that this is in fact the best way to do things, and that in the long term this will lead to far greater longevity of the value of the work. The scale is different, but it reminds me of the early work of Transported — both the evaluation and the strength of ongoing relationships there show the value of working in this way.

I am also reminded frequently of the learning from the first phase of Creative People and Places. This was drawn out so well within More Than 100 Stories, one of the best creative evaluations I’ve ever seen.

Starting With A Story Of Change

We started — as we try and always do — by putting together a Story of Change. This is an approach which has two key purposes:

1. It gets people thinking about where they want to get to — rather than focusing on what they need to do right now.
2. It is a really good way of sharing intentions and co-creating not just the evaluation, but also the overall approach to the work.

Story of Change, as we use it, was developed as part of the Happy Museum project — a fantastic project and resource bank of ideas, it’s well worth a look. There’s a brief video guide to doing your own and many templates to download.

For people who’ve done evaluation before, it is similar to a Logic Model or Theory of Change approach — focusing on the inputs, outputs, outcomes and impacts, but kept as simple as possible. We try and use accessible language throughout — so ‘story’ feels more accessible than ‘theory’ for example.

Building A Story Of Change

We like to involve a range of stakeholders in building the Story of Change. So in the case of Make/Shift, the Consortium and all active collaborators were invited — and we ended up with a real range of people. For the future, it would be good to include an even wider group.

Everything is done via a workshop approach — in practice this means big pieces of paper, post-it notes and examples. For Make/Shift, we ended up discussing someone’s ‘Uncle Tony’ as the sort of person who might be interested in one aspect of the work. But there are many approaches — other projects we’ve worked with have thrown themselves into acting elements out to explore things.

So we get to the four key elements of a Story of Change — The Why, The Who, The How and The What.

i. The Why — what’s the difference we are trying to make?

We start by looking at the ‘Why?’ — which is ‘why are we doing this’ — ‘what’s the difference we are trying to make?’.

Otherwise known as the outcomes and impacts. At a basic level you can combine these, and decide within each what are the short, medium and long term steps on the way.

ii. The Who

Next we consider ‘Who?’ — ‘who matters?’ — ‘who are we trying to make a difference to’. This can be the beneficiaries (in traditional language). But bear in mind it is about difference.

So these people might be:

• Potential ‘blockers’ we want to challenge over their power, or
• Potential funders we might want to notice the quality and value of the work, as well as
• The people involved.

One of the ‘who’ groups I nearly always have to remind people to include are themselves.

In an action research approach, we all change — if we don’t change then we aren’t learning. As action research is about learning, then it is about changing ourselves as well as others. It isn’t ‘self-centred’ to include the delivery team in the Story of Change — it is essential!

iii. The How

Having oriented ourselves in terms of the difference we want to make, we then think about how we get there — the ‘How’ is about what we put in:

• Where are we starting from? The context of the project/place,
• What is on offer? The assets and resources we can draw on — which do include money but also so much else
• What principles, values and approaches do we think really matter in making the change we want to see.

iv. The What

Finally — we go to the ‘What’.

What are the activities which we plan to do to make this difference. We list them in a high level way — bearing in mind that nothing will be fixed at the time.

It’s action research — we test and learn — and then check the logic. Do we think that by doing this activity we’ll achieve this difference with these people?

It is pretty easy to see some issues. If you have a change happening to a group, but nothing you are doing will reach this group, then it is likely that you might not have that effect. Often, we map out communication — to reach funders or others, or explicitly add in time for reflection so we can learn and adapt.

Make It One Page

Finally (but not really finally), I have a (self)discipline of ensuring it can be written up on one page.

This does tend to (over) simplify things, and can be a bit uncomfortable. But it is also a way of checking we are clear on the most important things, the essence of the plan.

Suitably for an action research-focused organisation, we are adapting and trialling new ways of writing these up. Our neurodiverse team realised that we all prefer different approaches to laying information out, so we assume others might too.

As a result, Make/Shift has three versions of its Story of Change:

• A visual model (below)
• A diagram
• A text version

You can click on the links above and choose which you prefer and we’ll adapt them based on feedback.

Incidentally, we checked in with a group of Make/Shift stakeholders and the preferences were almost exactly split — with some finding they couldn’t use or read one model, while another was perfect, with others saying the exact opposite — conclusion — you need multiple formats!

So What Next?

We use the Story of Change to design our evaluation — to decide on all the evaluation questions. These turn into a range of data collection methods including observation and discussions, as well as the unfortunately necessary questionnaires.

We will look at whether, and to what extent, Make/Shift is reaching its ‘Who’s and ‘Why’s, but more importantly how the ‘How’ affects this.

Our evaluation is about finding out what it was about how Make/Shift works that made the difference — we aren’t looking at if there was a difference or what that was.

Through this they can change their approach to make it as effective as possible. And we approach all of this by inviting and supporting the team, collaborators, decision-makers and friends to take part in an ongoing discussion about what matters and what works.

It’s our job to retell this in ‘evaluation language’.

Ever Changing

And finally… a Story of Change isn’t static, it changes.

There will be review and reflection sessions every six months or so — anyone is welcome to get involved in contributing to this and the evolution of the evaluation.