What’s a pirate’s favourite model of change – the 5 aaarrrrrrrs

So I guess we’re starting to see two types of article now. Brief, clear structured, practical. Then there’s the other 95%. Let’s try to address that imbalance, albeit it briefly.

Organisational change. Culture change. Behaviour change. Lots of change. A wide-ranging portfolio of change type has fallen on the sector, belatedly, but significantly as per usual. As per elsewhere there have been parallel attempts to be reductionist and mystifying all once. My hope is to demystify the fundamental principles, show them to be familiar, and all safe in the knowledge that these aren’t universal truths, but probabilistic guidelines.

Something to bear in mind. It’s an obvious one, but development and humanitarian work is about change. The principles of community change and organisational change are not wholly dissimilar. Look at principles of participation and you find, almost exactly, the principles of organisational change. The lovely thing here is that it has application at various scales and levels. You have an idea, you’d like to see it take root in peoples’ attention, to get support and flourish with potential barriers removed. That’s the change we’re often looking for. 

The list below is far from complete. I hope it has become unnecessary to add that every time, but it’s worth saying because most models offer that fallacy. But they’re not that wrong either. I hope they’re simple, accessible and applicable. That’d be lovely.

So of the types of change I’ve studied, they require structure, process and people. They need space and support. And they need a lot of communication. Here are some ideas of ways to simply look at it.

REVIEWConsider current internal & external contexts/environments, as well as historical and future factors
RESPECTCreate the space, get senior buy in, remove boundaries, prioritise at a fundamental level
REGALETell people about it, why it’s important, what it means – bang the drum
REINFORCE / REWARDReward progress and alignment; address otherwise; show interest and support
REHEARSE / REPEATEnsure that you repeat the above, the message, the rewarding; perpetual reinforcement enshrines norms
Back to:REVIEWContinue to review, refine and adapt approaches

Now that doesn’t tell you exactly how to do any of that, but if you’re reading this, hopefully you’ve got a decent shot at doing so or finding better sources to figure it out. Still doubtful? Then flip reverse it. Consider what it would be not to do one of the above. To not create space for it, e.g. create a team, assign time and other resources to it. Good luck with that. To not communicate about it. A tree falls in a woods. No-one knew to come hear it. To not reward progress and learning. What’s the ruddy point? So, it does make sense. Whether it’s the real recipe for rectifying reduced results, well, we’ll see.

What’s a pirate’s favourite model of change – the 5 aaarrrrrrrs

I can C clearly now we’re proactive

One thing I’ll say for the old Croix Rouge is that they do decent guides and docs. Accountability is no different, or CEA as they put it. Community Engagement AND Accountability? You ask too much. But whilst there are a cacophony of competing acronyms in this sector already, why not add some more. You’ve had the 5i’s, and soon to get some r’s, so let’s go with c’s today. Sadly there aren’t 7 of them.

To give credit where it’s due, there is little new about what I’m about to share. It’s just a combination of 3 things: IFRC’s approach (CEA), Ground Truth Solution’s approach (the Constituent Voice cycle) with a little sprinkle of my ill-informed opinion. Put them all together and I think you have a fairly solid structure that acts as flow chart and check list.

FacetDescription
COMMUNICATEAbout feedback options, about the programme, using various means as defined, ideally, from communication assessments
COLLECTReactive Mechanisms – e.g suggestion boxes and helplines
Proactive Mechanisms – systematised feedback (more below)
CRITIQUEAnalyse data, fuel internal discussion and focus group discussion with key stakeholders to add colour, depth and understanding to the stats
CLOSE (THE LOOP)Feed back on the feedback – ultimately feeding communication and continuing the momentum in the cycle

As I get round to implementing this kind of cycle myself I’ll add more detail into how it should work and then how/if it does. But the element to point out in particular is the use of proactive mechanisms of feedback. When we think of BFMs, CRMs, BPMs and other iterations of the same concept, we think of classic, reactive mechanisms. And by reactive I mean that we rely on people to give us feedback as and when they feel compelled or comfortable enough to do so. We just make the mechanisms. Be it through helplines, suggestion boxes, through leadership or direct communication with the team, we wait for people to tell us. And that leaves a gap.

The problem this presents itself is self-selection. If everyone felt equally comfortable and capable to feedback about any and all issues, we could worry about this less. But I’m British. And our best whinging is sadly saved for those not involved and, at the height of fury, the melting point of composure, we write sternly worded letters. Off. The. Chain. Luckily most national cultures are more advanced than that, however politeness certainly wasn’t conceived or perfected in my country of birth. So what to do about the self-selection malarkey? What we already do, just slightly differently.

It would be great to say this is all new and cutting edge and innovative, but it’s not. It’s derivative at best and it goes back to the idea of more frequent, light surveying to give an ongoing pulse check as opposed to large, artificial slices of mass-surveying. In a migration programme I’ve worked in we used 15 min exit surveys, followed by a second survey (usually done at distance / via the phone). But that’s in part to cope with the pragmatic issues pertaining to a highly dynamic population. Even with a static one, would small samples on a 1/4ly basis be too difficult? And would a cumulatively representative sample not add time-bound nuance if done in smaller packages as opposed to saving it all up for a midline or similar?

This is how I see proactive mechanisms boosting feedback and using constituent voice to drive adaptation. How I see a more fluid, dynamic project responding to light-touch surveys and insights. The beautiful secondary outcome is that it drives learning. It provides a structure, process and data set from which 1/4 reviews can reflect on and respond to feedback in an ongoing, genuinely adaptive fashion. And that’s where small tweaks to currently practices could start to have a significant shift in programme quality. And in community relations.

Sweeping generalisation aside, programme people tend to be pragmatic, resourceful, action oriented. Abstraction and theory is usually saved for late night discussions, often intoxicated by boredom or other such central nervous system depressants. There are plenty of valid critiques about techocratic influences, reductionism (bingo), over-simplification etc. and a lot of them that I talk to in my posts. But the people who have self selected into and been shaped by this industry, especially in programmes, generally reflect that system. It makes them very effective in a very many number of ways. But it’s a bias (just as being overly abstract is too). It’s one that can and should be challenged, and one that should also be respected. You don’t serve eggs to a vegan. You don’t speak to someone from Senegal in Greek (usually). So getting vague and abstract about reflection and learning and learning logs and taking time out to contemplate life – that’s not knowing or responding to your audience or their needs.

So ways around it? Provide the insight. Provide the structure. Provide the process and the tools. Use it as the framework from which the more transformative change can develop. Where all those processes of reflection and analysis are clearly linked with objectives, outputs and action. It’s simply putting learning into an appropriate language. And the great thing about that language? It’s roots are in action. Get the insight part covered and the desire and ability to action those changes will follow. In theory at least. C what I mean?

I can C clearly now we’re proactive

Vulnerability Griperia

This is a wee bit of a rant, fueled by one of my favourite work discussions ever with some of my favourite colleagues ever. You know who you are. You’re not even reading are you. Awkward.

I’ve got issues with the use of vulnerability criteria. And they may annoy you.

1. Keeping vulnerability in the singular disguises its massively complex (dynamic, pluralistic and relational) nature. Using the term vulnerabilities doesn’t automatically include all of the four (DCPR) aspects, but it least moves the concept and therefore the conception of it, towards them. Vulnerabilities infers that anyone can have many types of vulnerability. That in turn, kind of, infers that they could be relational, even dynamic, and definitely complex. It’s one small step for pedants. A small twist. Many people don’t react warmly to such ‘superficial’ changes. Such hyper-sensitive knit-picking. It’s PC gone mad. But as various people have demonstrated over a prolonged period of time, language and specifically words matter. Let’s pretend to be social scientists for a moment. How many times have you used words in the last week? I’ll make it easier, the last 24 hours? Nope? Even the last minute – how many words did you use, in thought or out loud? Great, so we’ve established that volume is high. High enough to be uncountable in a 60 short seconds. So if words within that are unrepresentative of the complex concepts they aim to articulate, how long does it take such a ‘superficial’ difference to become a more significant one. If you’re on a boat that is 1 degree off course, it doesn’t matter much if you’re crossing a lake. But an ocean, that’s an issue. And we cross oceans of words on a daily basis. 

2. Vulnerability is often something that is broken down into personal, individual, internal aspects. Like health. Like age. Like sex. It’s not so good at situating these as sociologically informed and influenced facets. Being female is often considered a vulnerability. Full stop. Short sighted, no? Even in practical terms, there are situations where men are more vulnerable than women, to certain risks and in particular contexts. And that highlights two issues. The first is touched on above, that assigning vulnerability to people is often done in a reductionist fashion (there should be a game about how often I write that word). That labeling someone as vulnerable is so short-sighted, it’s unhelpful. The second is that our relative proportions of vulnerabilities are hugely influenced by our contexts.

3. Vulnerability is currently situated within the individual, and that is hugely unrepresentative of their actual levels of risk. We should therefore consider both individual factors, and those outside of the individual. Societal factors such as stigma. Legal factors such as migration status. We could then start to situate blame not where vulnerabilities land, but from where they are derived.

The above only alludes to it, but it’s worth being clear that the word ‘intersectionality’ is critical and underpinning much of the above thought. Reducing such complexity to a single score is a tough pill to swallow. But in the name of haste, I wont tempt myself into a tangential rant about the myth of specificity and credibility that we often use as a lens through which we see quantitative information, even if it’s concluded through qualitative methods.

So here’s an ill-informed attempt at something with a cheeky bit more nuance. They at least take into account external marginalisation as well as internal vulnerability. They’re not complete, nor comprehensive, but they hopefully nudge in the right kind of direction.

  • External (marginalisation)
    • Legal
      • Migration status
      • Citizenship
    • Societal
      • Stigma relating to homo- or trans-phobia
      • Stigma relating to being a returnee
  • Internal (vulnerability)
    • Physical
      • Recent injuries
      • Chronic injuries / physical disabilities
    • Psychological
      • PTSD / victime of trauma
      • Psychological disorder, e.g. mental health

Or, as it is the root of all things, it could be something we phrase in terms of agency:

  • Structural agency – state or regionally defined, e.g. legal aspects, infrastructure such as roads
  • Systemic agency – the application of the structural, e.g. state health services or police action
  • Societal agency – the sociological context, e.g. class/gender/ethnicity, but also wealth for example
  • Psychological agency – current and chronic psychological factors
  • Physical agency – current and chronic physical factors

More to follow on agency being root of all things. If you want to see the real thought behind it, have a look at Amartya Sen’s infamous work, or Arjun Appadurai’s ‘Capacity to Aspire’. Poverty isn’t financial. You can have no money in your pocket and be fed by your family. Well-off people generally have greater safety nets in times of immediate financial poverty. They have friends, family, probably qualifications – they have multiple facets of capital that they can still mobilise in times of desperation. As said by Selina Kyle in the Dark Knight: “even the rich don’t go broke like the rest of us”. They still have the agency, the freedom, the ability to navigate out of shocks. Because of their inherent privilege. Mini-rant done – now you start to see why agency is going to a word as repetitious as reductionist in these post.

Vulnerability Griperia

A theory for change – 5i’s for holistic change management

Change. The final frontier. How does it work? How can we make it work? To talk about the development and humanitarian sectors in corporate terms – they’re both all about change management.

The concept of systemic change is an aspirational yet daunting endeavour. Operating within complex systems of layers and stakeholders can lead to narrow thinking, focusing on challenging one part of the system. In reality, a number of different efforts must be made to see change. That said, it isn’t necessary for one organisation to tackle the same problem from every angle. As a result, the 5i’s can be used to support the design of programmes both in terms of activities themselves, but also it can foster opportunities for partnerships or, at the very least, complimentary action to ensure a collective drive for change from multiple angles.

So what are these 5 ‘i’s and where do they come from? Well, as with everything else here, they’ve been adapted, tweaked and rearticulate from various inputs. Most notably, the original was ‘inspire, influence and incubate’, taken from transition theory and shared with me by a former colleague at Bond. Since there, I’ve added some bits and pieces.

So why did it resonate? It’s multi-faceted: it considers change required various inter-relating moves to see real progress made. It had some real face validity to it – it made sense. Without space being created for people to experiment, the practical steps to realising innovative concepts wont take place. As mentioned before, policy is often influenced by practice, but it is in both directions and it is in that reciprocity where change is most likely to occur. Creating space to incubate and the practice to expand that space is a powerful combination. In its full expansion, the model has notes of top-down, bottom-up and lateral dynamics. That feels powerful. So here’re the headlines:

FacetDescriptionExamples
ILLUMINATE
Bang the drum
shed light on the issue through communications, internally, externally, face-to-face and digitallylaunching a blog, writing a paper, contributing to debate in meetings, events and online
INFLUENCE
Create space
space must be created for change, often at policy/political levels but also in other systems such as industries and organisationsadvocate and create space, provide resource for teams and remove hurdles
INCUBATE
Protect pockets
seek pockets of innovation and support they continued developmentshift away from immediate, orthodox results, create freedom to fail and reward for learning, even make teams with specific alternative projects to do so
INSTIGATE
Create conditions to spark
where innovation doesn’t occur and yet the conditions create an opportunity, create pockets of innovation by sparking ideascreate spaces to engage potential pockets for innovation and incubation – seek out where progress is being made
INTERACT
Collaborate
identify the opportunities for individuals, organisations and even systems to interact, share, learn and challenge each othercreate communities of practice, technical groups, platforms for conversation and shared learning

And what that may look like in diagrammatic form:

So now we can get to utility. I genuinely think it has merit in a variety of situations. Whether you’re trying to change the whole humanitarian system or internal practices. Whether you’re trying to get a new or hopefully improved method off the ground, or even designing programmes.

Whatever the change is that you’re looking for. Promote it. Create the space for it. Protect its development. Create the conditions for new contributions and collaboration, especially with those similarly sought goals but perhaps different ways of getting there. And if you spot gaps in that plan – see what you can do to change it.

A theory for change – 5i’s for holistic change management

Flipping accountability on its heed

Accountability. A lovely concept, so often vague enough to disguise its multi-faceted and -functional nature. With a bit of expansion on the topic, we may be able to then drill down into the detail in order to pick apart some ways of improving something we’ve long feigned to improve.

When considering accountability in its broadest sense, it’s hard not to consider the infamous Aid Chain. Money flows downstream and power back up. And ultimately, that means accountability flows back up too. Governments are not held accountable by the organisations they fund. NGOs are not held accountable by the people they serve. Not enough at least. And nowhere near as much as they are accountable upstream. It’s Paulo Freire practically articulated. 

We talk of accountability to affected or at risk people (a cheeky twist on AAP) in a slightly oxymoronic fashion. It’s optional. Accountability as a concept isn’t. To be accountable is not to pick and choose how and when to what degree one can be held accountable. But that’s how we generally operate when it comes to AAP. It’s not like we don’t know what accountability looks like. A donor deadline is gospel. Their requirements written on stone tablets. And we work all the hours we can to ensure that we meet such requirements, regardless of perceived programmatic value, for fear of being financially crucified. Fair point, I’ll stop the religious metaphors there. You get the point.

Whilst earnest conversations have been had for decades, predating my existence let alone participation in this sector, progress has been slow. Change is multi-faceted. It requires holistic interventions that account for various intersecting factors. Yep, complex, dynamic blah blah blah. So how can it change more significantly and soon? 

That’s a tough nut to crack and it hasn’t been ignored in discussion to date. But here’s a thought. If power upstream is so strong, so reinforced, so incentivised, why not co-opt it? What if ‘upward’ accountability were the same as ‘downward’ accountability? Am I being a person possessed of some radical notions? Let me see if I can plant this idea deep enough into your conscious to create some inception style accordance. So let me be more specific. What if we were measured just in terms of constituent voice? What if we took the perspectives and opinions of affected and at risk people as a proxy at least for quality and for outcomes? 

It’s not that drastically radical a concept except that it brings up a topic that often sends people, not least of the M&E persuasion, into meltdown. Satisfaction. Why is this such a controversial concept? Well, I’m not entirely sure to be honest. Perhaps it’s because it’s a concept that is considered very culturally skewed – confirmation bias aplenty of overly positive reporting, especially when asked to report satisfaction to a member of that organisation. Perhaps it’s a fear of control? Of simplicity? Of a shift in power? Perhaps we don’t trust people, not just to be honest, but to be able to report on quality in an informed way. Yes, the height of condescension, we know what’s good for you style. Even in its most simplistic sense, whatever complex index or question set we ask is fundamentally based upon asking opinion, whether for PDMs or impact assessments. The concept of asking the question directly somehow creates a big problem. One that I’ve yet to see someone really break down, explore, pick apart and address. The conversation stops at the problem and I think we need more potential solutions. 

So let’s follow this idea down the rabbit hole. What would happen if we ran with it? If we used satisfaction as a proxy for various aspects of quality? Well, monitoring activities might become lighter, especially if you run with the proxy idea. If they’re more direct questions, lighter, shorter, less time consuming, we could use the space to either get such insights more frequently, more qualitatively and/or focus on following up after gathering insight – closing the loop.

Sounds simple enough, so what’s the problem? Sadly, credibility. And if we’re talking credibility, we have to understand to whom. Yes, we’re back to the Aid Chain. If a donor suddenly required that, beyond a few simple output indicators, the degree of participation, quality of feedback and degree of satisfaction were above all key, well, it could happen. And quickly. I’ve had discussions with people who work in financing institutions, including government donors, who are genuinely interested in the concept. But there are blockers elsewhere, nestled neatly within NGOs, and not least within the M&E ‘community’. To guard our status as specialists, we are incentivised to create complexity in order to reduce accessibility. That’s what makes us needed. Understanding specific, complex issues is what makes us special after all.

As per previous posts, the proof will be in the pudding. It needs to be tried, tested, adapted and discussed. And somewhere that it can be given some space for this kind of experimentation. It’s already being tried in different guises – again, this is nothing new, but it is something that I would like to see grow. To be promoted. To be given space. To be piloted. To be discussed. Because I think it has merits. I think that aligning accountability to affected and at risk people, to accountability to donors is a way of given AAP the force to be really driven forward. It is a concept that is fairly simple, using familiar tools but it has the potential to nudge us further towards radically shifting power dynamics. I hope.

NB: Cornwall and Brock’s article on Buzzwords and Fuzzwords is a glorious piece of work, especially in how it identifies where good intention often is coopted into undermining itself. They argue that certain (cyclical) calls for greater participation were born out of recognising the need for more collective, political agency for affected and at risk people. They also argue that in choosing the ‘community’ as the principle sociological denomination, the have served to undermine the group’s political power as most movements necessarily garner strength from being broader. It’s a great read, please check it out.

Flipping accountability on its heed

What does good look like? We’ll see

Yep – let’s start with a film reference. This one is from Charlie Wilson’s War. An amusing if poignant and potentially emotionally brutal film for anyone who has experienced the recent Afghan conflict.

Cut towards the end of the film. Celebrations. A swanky flat in DC is filled with self-congratulation through the media of music, dance, laughter and champagne. The protagonist, of both the film and the party, heads to the balcony to meet a (not uncharacteristically) sombre looking Philip Seymour Hoffman. He delivers the kind of speech, parable, analogy that manages to be profoundly efficient and efficiently profound (language warning: https://www.youtube.com/watch?v=e2cjVhUrmII):

“A boy is given a horse on his 14th birthday. Everyone in the village says, “Oh how wonderful.” But a Zen master who lives in the village says, “We’ll see.” ‘The boy falls off the horse and breaks his foot. Everyone in the village says, “Oh how awful.” The Zen master says, “We’ll see.” The village is thrown into war and all the young men have to go to war. But, because of the broken foot, the boy stays behind. Everyone says, “Oh, how wonderful.” The Zen master says, “We’ll see.””

When we focus so hard on an objective, we almost by definition, close off our minds to what is beyond it. And what is beyond it is a lot. And it’s complex. And dynamic. And pluralistic. And yes, it’s also relational.

To put it into an industrial coding, we focus so much on our favoured denominations of action, the project and the programme, that we are biased and blinkered to see little beyond. It’s a degree of highly incentivised active ignorance. And after they’re ‘complete’, we label them and store them away. Was it a good programme or a bad one? Did it have a good outcome? As if anything were that simple. And if anything remained that way. What was once a ‘good’ outcome can lead to a ‘bad’ one and vice versa. But we don’t see. Because we don’t look.

This speaks to something psychological and well as sectorial. Our insecurity-derived need to understand in order to control fuels a particular brand of reductionism. It’s pragmatic, but not in the way it’s usually referred to. It’s psychologically pragmatic. In Man of Steel, a young Superman/Clark Kent tries to deal with the sensory overload that his innate abilities bring him. Waves of sound and light to an overwhelming depth and weight floods his perceptions. It becomes too much so he does what all sensible kids would do. He hides in a cupboard. Until his mum arrives (https://www.youtube.com/watch?v=BRfpZD2ggx0):

“The world’s too big, mom”

“Then make it small. Just focus on my voice. Pretend it’s an island out in the ocean”

Despite our lack of super powers, we do this all the time. We focus, an act of prioritisation and of reductionism. We admit, on the thinnest piece of our internal diaries, in the smallest writing, that we can not process everything, at any point of time, let alone all of it. So we choose. These choices are often informed by questions of utility, urgency and value. Incentives and reinforcement. They’re shaped and reshaped. But because they’re done doesn’t meant that they should always be done. We should push the edges of our focus out to be a little wider, even if only for temporary perspective.

What I love particularly about the Zen Master / Taoist proverb is how it so smoothly brings into view both dynamism and causality. How any change has changed the immediate dynamics, and how that can in turn influence others. It challenges a myth that is so often perpetuated, that what becomes good, stays good and what becomes bad, stays bad. Or to quote Battfleck: “What falls, is fallen”.

To get personal, I have profited from my experience of grief. Not financially, certainly not immediately, and I doubt that net, overall experience could find much competition with the negative. But there were positives. And they weren’t insignificant and they were a direct result of a resounding negative. Demonstrations of the fluctuating ebbs and flows of causal influence. So we end up back at chains of events and influence. At concepts of dynamism and perspective. Of relational, intersecting forces that sway one way with knock on effects, then sway back. And if you can get all that from 30-odd seconds of a Tom Hanks film, then it must have been a pretty powerful quote.

So, moving towards a reluctant practicality, a familiar question remains. If we slice up reality into such small segments, and have little to no financial (i.e. cost-neutral) incentives to look further beyond, how do we see? How to open the key hole a little wider? Honestly, I don’t really know. Funding is always answer number one, but, despite it’s merits as an argument, it’s a convenient dissociation of responsibility and agency too. And as David Mosse so magnificently pointed out, it’s practice that shapes policy. Show something can work, align it to preexisting strategic / political objectives and then bang the drum about it. Then political space will be created for it. Perhaps.

Critically, any such change will require a shifting of expectations. Developing lighter tools, to be used more frequently and for longer after official project boundaries have passed by. If we could develop crisp, smart tools instead of large slabs of 160 question-naires, we could learn more frequently, more quickly and – critically – outside of our immediate scope. Some of the perception survey work done by Ground Truth Solutions, and especially their work with the British Red Cross on the 3-step feedback mechanism, hopes to address that. I should be transparent about my inherent bias with this particular piece of work, but light, frequent and actioned feedback will – for me – always trump large data dumps of more rigorous and less useful information. That’s one of the reasons why we worked on developing a mechanism that would try to do so. The priority is utility. A perfect piece of data collection and analysis can mount to as much as a tree falling in the woods, with no one there to hear it.

It’s an unpopular notion, but to be clear: I would rather less rigidly positivistic methods were deemed credible. That light, smart, rapid looped mechanisms were used to develop more learning/action oriented insights. And that these were given space to show their worth despite being usually dismissed out of hand by artificial laws of credibility. I would rather prioritise proximity and utility over rigour. I would rather M&E became a process of active, dare I say it dynamic, dialogue. With beneficiaries, programme teams and contextual shifts all active participants (to name but a few). Wouldn’t that be good? Hopefully, one day, we’ll see.

What does good look like? We’ll see

Minding your P’s and C’s – what donors (and others) want

What donors want – no, not the bizarre horror sequel to a terrible Mel Gibson and Helen Hunt film, but the eternal driving question, so often nestled within M&E as much as any other area. Well, given my relatively few years working closely with them, I’ll choose to comment from a position of ignorance, much like the great and bizarre Alan Moore claims to prefer.

Having been the beneficiary of education in institutions who seek the nuance in otherwise homogenising and/or reductionist fields of analysis, they often have (at least) one critical blind spot: donors. Even through their various lenses of analyses, the very sight of a donor blurs them, leading to smudged and homogenised perceptions (perhaps David Mosse accepted). This is a problem, not least for consistency and integrity, but also for analysis. The same financing institution may have a culture, a reputation, a set of standards and approaches. Look a little more closely and internal dynamics are illuminated, and varied. Earnest desires for change can be contained within rigid frameworks, and yet remain and be influential. Domestic political will and gamesmanship hold sway. Individual personalities and biases skew and complicate. It’s a lot like the inner workings of any organisation, whether ‘humanitarian’ or not. Regardless, I’m going to try and convince you, in a similarly reductionist approach, what they all want to see. Hypocrisy indeed.

Tangent aside, let’s get back to bidness. So I think there is a simple, if incomplete, way to describe what donors want to see. Luckily, it’s a lot like what HQs want to see. What country directors want to see etc. etc. Asking for directions of influence and causality in terms of where those wishes originated (and who influenced who) is a whole other blog in the making, and leads us not into being even more tangential.

So ultimately, I think it boils down to two questions:

  1. What has been done? 
  2. What has changed?

Simple enough. For all my accordion chat, let’s put it into practice and stretch out this simplification. Both of these questions have two key facets:

  1. What has been done? – in terms of:
    • the performance of the programme
    • the progression of the programme
  2. What has changed? – in terms of:
    • the context
    • people the programme is working with

I’m sure you’ve seen many a form that seeks to coax the same points out with alternate articulation. They often start with Context: what has changed and how has that influenced the programme? They may then, broadly, move onto Performance – how has the programme performed, how well has it delivered? Then it’s Progress’s turn – how has the programme adapted, developed and sought to improve its performance? And last but not least, Change – what changes are being experienced, what indications of impact can be seen?

If you keep a ponder on these, some other links appear. What becomes clear is that the former and the latter are external to the programme – they are the context it works within and the people it aims to serve. Performance speaks mostly to outputs. Progress to adaptation. Change to outcomes (realistically speaking) and indications of impact. So if I put that into some semblance of order, it looks like this:

FacetUnderlying QuestionLocusSources
CONTEXTWhat has changed and how has that influenced the programme?ExternalReports: Context analyses, needs assessments, field reports, baselines
PERFORMANCEHow has the programme performed, how well has it delivered?InternalIndicators: inputs, outputs, value for money
Reports: narrative field reports, midlines, endlines
PROGRESSHow has the programme adapted, developed and sought to improve its performance?InternalReports: narrative field reports of adaptation and continuous improvement
Supplementary: change logs, evidence of change in response to feedback
CHANGEWhat changes are being experienced, what indications of impact can be seen?ExternalReports: evaluations, midlines, endlines
Indicators: intermediate outcomes, ultimate outcomes, impact
Supplementary: case studies, quotes, field reports

It may be a little simplified and reductionist in its most basic form, but it has proven a robust structure and, dare I say it, narrative structure. What I like about it is that it works for the long and the short alike. It can be accordion-ed down to a paragraph as an ‘elevator pitch’. It can be expanded out to a full annual report. Put the above together and you will have a strong section of any annual report or full donor report, internal report. In fact put any word in front of report and this wont be a terrible attempt at answering the questions underlying it.

So what do donors want? To know what has been done and what has changed. To better understand the Context, the Performance and the Progress of the programme and, of course, what Change that has all amounted to. It sounds a lot like what we all want.

Minding your P’s and C’s – what donors (and others) want

Welcome to the evidence layer cake

Layer Cake. A terrific film from Matthew Vaughn, back along prompting calls for Daniel Craig to be Bond and barely recognising Tom Hardy as a polished Oxbridge graduate. Great film, great soundtrack, smart directing, lovely times. And here’s a favourite quote from it (SPOILER – there are lightly censored swears in it – if you’re likely to be offended by that, please don’t actively engage in either the film nor the quote – the rest is almost potty-mouth free).

“You’re born, you take sh!*

Get out in the world, take more sh!*

Climb a little higher, take less sh!*

Til one day you’re up in the rarefied atmosphere and you’ve forgotten what sh!* even looks like

Welcome to the layer cake, son”

So what has that got to do with M&E? Well. Nothing immediately springs to mind. But I liked the visual of the layer cake and that quickly takes me to Maslow’s Hierarchy of Needs. Because I believe that there’s a hierarchy of evidence needs, often accidentally articulated through the development of M&E functions. Cos when they start up, you tend to have to take sh!*. 

So, to get specific…

  • When you’re starting out building M&E, you start with the basic quant. Output level. Distribution and beneficiary lists and such. 
  • You then more to more sophisticated quant, perhaps even using digital data gathering. 
  • You may then try to sprinkle on top some case studies and quotes, photos, even video to get the pathos going. 
  • You may even then add another layer, moving into outcomes, aiming to use internationally recognised indices, whether or not they were designed thousands of miles from where they’re meant to be relevant. Yes, that’s a nice slice of ethos there. 
  • You may then try a few focus groups, moving towards the giddy heights of more sophisticated, systematic and representative qualitative insights. 
  • And then what? Well, whatever stage you’re at, you have the chance to wrap it in some nice, all encompassing icing. Something to hold it all together, to compliment all layers. Yep, that’s the theory of change. 

Of course there are other elements you could add too, but hopefully you get the idea. Flip that list on it’s head and you have Michael Gambon’s Hierarchy of Evidence Needs. The idea, such as in the cheese cake reference in a previous post, is to develop a sophisticated, balanced, connected and intellectually delicious combination. One in which you can cut a nice big slice of evidence, from multiple sources and methodologies, that provides the taster with a complex and complementary blend. One that provides ins-ight and del-ight all at the same time.

But that all takes resource, time not the least among them. I guess we need to keep building, layer by layer. Until one day, we’re up in the rarefied atmosphere, and we’ve forgotten what paper-based monitoring even looks like. Welcome to the layer cake.

Welcome to the evidence layer cake

The Polar Express(ion) of Evidence

As previously ranted and raved:
“Evidence has massive political significance. The type of Evidence that is politically compelling is often unrepresentative. Our challenge, as a specialism, is to find better ways to articulate representative insights. That’s the most challenging and infuriating part. It’s why we need more than just bean counters.”

Evidence, is a form of communication and is ultimately about persuasion: whether that’s persuading someone of progress, performance, to change their mind or just to persuade them to engage more, to keep reading. According to Aristotle (name drop), there are 3 key methods of persuasion:

  • Character (ethos): essentially the credibility of the source, whether it has good sense, good will, and good moral character
  • Emotion (pathos): building a common bond through emotion or shared identity (either of the character or of the topics)
  • Logic / reasons (logos): demonstrating clear and logical arguments

What we see currently is that the most politically compelling evidence has become polarised. At one end is the perceived credibility of high level quant stats: of graphs, dashboards and infographics. Fighting for attention out of the qual corner, it’s the emotive story: the case study with photos, names and experiences. Rarely are these two connected in a meaningful way, they are merely associated, and loosely. And that means that we have a few problems:

  • The broad spectrum high level stats remain reductive in terms of explanatory value
  • The narrow, but deep, insights from individual level case studies float disconnected from the statistics that they are supposed to better articulate

The connection and therefore the narrative between the two, are missing, because the stages in between are missing. It’s like having a cheese cake with just the hard, biscuity base and the sharp but sweet fruit topping. No combining, connecting agent inbetween. Nothing to take you from one extreme to the other in a smooth, relevant way. So to go back to Aristotle’s model, the most compelling types of evidence should appeal to cognitive credibility and emotional resonance all within a logical narrative that holds the information all together. And narrative is the critical word there.

So, critiquing’s easy and self-fulfilling. What about solutions? Well, here’s an abstract one: theories of change. If done well (and I’ll go into what I think that may look like in another post) theories of change can provide that narrative structure. Ultimately that’s how I see them. They are a complex narrative, punctuated by inputs, actions, desired change and foreseen contradictions and inhibitions (otherwise known as assumptions). They tell you what you’re aiming for, how you’re going to try and get there, and the potential challenges that will need to be faced, explored and hopefully overcome to do so. Get your theory of change right, and the rest should follow. And by the rest, that could be anything from logframes to communications plans, programme development to learning plans. And of course, whether your theory of change is ‘right’ or not is a matter of perspective and timing, so adapting them is critical. But you can only update them if you’re actively seeking insight throughout them. But I’m sure I’ll need to do much more persuasion on all that, to follow, with some help from the ancient Greeks perhaps.

The Polar Express(ion) of Evidence

Flex your facilitatory BICEPs

Rarely does the scooby snack of the humanitarian and development sectors, the acronym, come along without being wedged into an awkwardly connected abbreviation. I managed to stumble across one though and it’s ridiculous, perhaps more memorable as a result, and potentially even useful.

As with most things within M&E there is both a hyper pragmatism and an theoretical aloofness to conducting focus groups. Either people just get them done without question, or they’re hypersensitive to each nuance. This is an attempt at nestling somewhere in the middle of such clowns and jokers. Here are what I think are constructive behaviours for conducting focus groups. Fortunately, what is good from a research perspective is often good from a principled one too. See what you think:

Encouraging… Behaviour Desired Outcome
Balance freely discussing positives and negatives with equal importance and focus (as opposed to just fishing for compliments and presenting a shallow approach to learning) participants feel encouraged to discuss both positives and negatives, that we receive more honest, balanced feedback and also that they know that we’re are self-critical in order to improve 
Inclusion ensure that different people interact from across the group, that everyone has a chance to have their say – the loudest voices shouldn’t carry the farthest the quiet voice may provided the deepest insight and regardless, everyone should have a say, both on principled ground and those relating to conducting research
Comprehension take time to explain what can be abstract and/or unfamiliar concepts – use examples, check understanding with the group the most valid answers can only be achieved if the concepts being discussed are clear and well understood across the group
Engagement use appropriate tone, energy and humour to encourage people to be stimulated by the discussion – it is their time you’re spending to get your insights, so make sure it’s engaging engagement can make it easier for the group to participate, can maintain attention and avoid frustrating people through prolonged, dry uses of their time
Pace ensure that the focus group keeps moving, that each exercise is given fair time and are done with a degree of flex for important points and discussions being allowed to run communities’ time is respected and more focus groups can be conducted if time is kept well (and therefore wider perspectives sought); sections are given their due time to explore each concept

And how to use them? As a framework to check your own approach, as a model to share with facilitators during training, as a development tool for observing facilitators and giving them structured feedback, there are plenty of options. Why not stick scores on it and pretend it’s hyper-accurate? The choice is yours. But it’s not a bad starter for ten to get those muscles flexing.

Flex your facilitatory BICEPs