How can funders measure the impact of enterprise grant-making in ways that are useful for them, and for the organisations they fund? By being clear about what you are trying to achieve, avoiding too great a reliance on numerical data, encouraging curiosity, and exercising judgement, says Richard Harries, Director of Caritas Westminster and member of the Enterprise Grants Taskforce, in conversation with Steve Wyler, advisor to the Enterprise Grants Taskforce.
More than just numbers
Steve: Hi Richard, to start with, can you tell me a little about your past experience of impact measurement?
Richard: When I was working in Government, in the late 1990s and early 2000s, one of my roles was to collect impact evidence about programmes funded by government. But it was difficult to show evidence of change, especially at a small scale. For example, to assess whether or not a neighbourhood improved as a result of a particular government-funded activity. So it was often a frustrating experience.
Steve: In what way was it frustrating?
Richard: The tendency was to devolve to numerical measurements too quickly. They were often the wrong numbers, and the numbers would be interpreted incorrectly. This still goes on a lot. Just the other day, I was in a discussion about community organising, and one organisation was using levels of church attendance as its primary indicator of success. That’s just one small example, but it illustrates the difficulty: numerical measurements are tempting but can easily become instrumental and narrow.
Enterprise grants and increased trading – a ‘bright line’ measure?
Steve: So, given that, what should a funder do – particularly one that is involved in enterprise grant-making - if they want to assess the impact of the grants they make?
Richard: Well, it helps if you start with clarity about what you are trying to achieve. One advantage of enterprise grant-making is the focus on enterprise, on an increase in trading. This, it seems to me, is a ‘bright line’ measure, capable of providing a clearly defined and objective standard. If you don’t increase trading, even if there are other benefits that can be demonstrated, you have not achieved the impact you wanted.
Steve: Yes, I can see that. But is measuring an increase in trading sufficient?
Richard: No, not entirely. I remember someone quoting the phrase ‘income is vanity, profit is sanity’ and of course that’s a good point, but on the other hand without income you can’t have profit, or wider social benefits for that matter, so measuring an uplift in traded income, and being clear why you are doing that, is a good starting point.
Steve: Yes, I agree, a clear and useful line of enquiry to start with. And are there ways to build on that?
Trading and grant ratios
Richard: When I was working at Power to Change, we were providing funds to the School for Social Entrepreneurs, so that they could award Match Trading grants to community businesses. We were excited by this method of using grant money to incentivise and reward trading success. We were also interested in understanding the wider mix of income, so we measured the ‘trading ratio’ – in other words the level of earned income a community business achieved compared to grant income. We wanted to discover whether the Match Trading model increased the balance in favour of earned income – and in fact we were able to show that it did.
Steve: Was that a straightforward distinction to make?
Richard: There are some subtleties to take account of. Occasionally it can be difficult to determine whether something is truly earned income, or whether in fact it is really a form of grant, but despite this we felt we were able to measure the trading ratio in a reasonably robust way. And one thing we noticed was that for some organisations, as their trading income grew, so did their grant income.
Steve: That doesn’t surprise me. I remember from working with community enterprises at Locality some years back that it was often the case that those who started to generate significant earned income became well known for their can-do and entrepreneurial spirt, and this made them very attractive for some grant-making trusts and foundations, and so their grant income rose fast as well. Do you see that as a good thing?
Richard: I suppose most people would see that as success. But it does raise the question about what is regarded as sustainability: are we aiming to wean people off grants entirely, or are we aiming for an uplift of earned income, sitting alongside continuing grant income?
Steve: Is the latter preferable?
Richard: Generally, I think so. But I think we may be moving towards a more nuanced way of grant-making where it becomes increasingly possible to tailor the level of grant subsidy according to more realistic expectations about earned income. For some types of activity, in some parts of the country, it will never be possible to rely significantly on trading income, so a high level of grant subsidy will continue to be needed. In other cases, the need for continued grant subsidy will be much less. Or sometimes, no grant at all will be needed. As we understand more about these variances, funders will be able to tailor their support better.
The benefits of enterprise grants
Steve: Tell me more about this, and why it matters.
Richard: I see an opportunity here. The funding environment at the moment is incredibly tough – funders are having to turn down good applications left right and centre. Enterprise grants offer a much better way to distribute a limited pot of money. If we can get more funders involved and collect more data, we will be able to see which sectors and activities, and which geographies, need which types of trading ratios, and then we can tailor the enterprise grants we give more efficiently, and the precious pool of grant funding will be spread far and wide, and as many people will benefit as possible.
Steve: Yes, I can see that, and that is an exciting prospect. But won’t the large number of variables make this just too difficult?
Richard: Yes, I accept that there will be many variables. For example, within a particular geography there might need to be different ratios at neighbourhood level to take account of the different local circumstances. The Match Trading model developed by the School for Social Entrepreneurs offers£1 grant for every £1 uplift in trading – perhaps in neighbourhoods where deprivation is highest the grant could be increased to £2, and conversely lowered to 50p in the wealthiest neighbourhoods. I’m not saying that we will be able to fine-tune the ratios to the last decimal place, and our approach should never become too formulaic, or over-reliant on data. A funder who takes the time to walk around an estate, and talk to people there, will probably gain more insight from that, than from any dataset. But good data can add to our insight, and the more that funders are able to share data about enterprise grant-making the more we will learn, sometimes in unexpected ways, and the more we can make good judgements to direct the grants to where they can produce the best results.
Steve: Good judgements rather than set formulas, that sounds right to me. But, whatever the level of grant, do you see efforts to increase trading as universally positive?
Richard: Where trading can contribute to an organisation’s income, even a small proportion, that’s a better way to do business, because of the freedom that it can give. But only where it is well-aligned with the core mission of the organisation.
Steve: And in your experience, is that alignment sometimes missing?
A win-win situation: strong alignment with purpose and successful income generation
Richard: I’ve often heard the concern that too great a focus on trading can lead to a loss of attention on social objectives. And of course, the enterprise activity should never be in opposition to the core mission. But in my experience, this is largely a theoretical worry. People working in the social sector are usually highly motivated to seek positive social outcomes. The win-win situation is where there is clear and strong alignment with social purpose and there is successful income generation at the same time, and that is mostly what I’ve seen.
Steve: But sometimes that isn’t achieved, the alignment isn’t there, or the trading is simply not strong enough. What should happen then?
Richard: One of the benefits of Match Trading, as delivered by the School for Social Entrepreneurs alongside their action learning programmes, is that it encourages people to be more mature and ‘business like’ in the sense that they are willing to confront the brutal facts when an enterprise is not working.
The critical thing is to distinguish between impact measurement and impact assessment … to understand whether we are making a difference and what kind of difference
Steve: Yes, I’m sure that’s right, and I agree that enterprise grant-making can help people reflect and strengthen their practice in the way you’ve described. But tell me, when we think about impact should we be thinking mainly about ‘proving’, in other words producing hard evidence of success for funders, investors, Boards, and so on, or ‘improving’, that is, using the data to encourage the kind of reflection that can lead to better ways of working?
Richard: Both can have their place. The critical thing I believe is to distinguish between impact measurement, and impact assessment. Impact measurement is part of the process, and generating numbers can be valuable. And while there is a natural tendency to select the numbers which best prove whatever argument one wishes to advance, there are sometimes ways to address this – for example, in the case of Match Trading grant-making we were able to run a control group, so that we could be sure that the numerical evidence, on the relatively narrow question of trading uplift, was solid. But beyond impact measurement we also need impact assessment, which is a different thing and requires a different mindset. The aim there is to understand whether we are making a difference and what kind of difference, not to generate a set of numbers that can prove that we are making the difference we had promised at the outset.
The importance of intentionality
Steve: What would be an example of an impact assessment approach?
Richard: At Power to Change we commissioned MyCake to undertake a study of community-run coffee shops. In assessing the impact of these enterprises, it became clear that the question of intentionality was of central importance. If the intention is to compete commercially with the likes of Costa and Starbucks for example, that’s one thing (though those who try to do so are likely to fail). But if the intention is to generate social outcomes, employment opportunities for marginalised groups of people for example, then achieving that while perhaps not making a profit but maybe generating sufficient income to cover costs could legitimately be seen as a very positive outcome.
Steve: So, what makes for good impact assessment?
Richard: Above all you need to encourage curiosity. You need to understand that value can be added in many ways. You can’t delegate judgement, you need to apply judgement to the evidence. You need to take account of the broader context – the operating environment, the stage of development of the organisation, the social objectives. And sometimes you need to be prepared to come clean that something hasn’t worked as well as hoped for, and be willing to face up to that and to learn from it.
Are enterprise grants right for everyone?
Steve: That’s so helpful. But can I now ask you about the relationship between the funder and the funded organisation, and how that might affect how we understand impact. It’s not a straightforward relationship, is it? There is always a power imbalance, and the funded organisation often feels a need to impress the funder. What are the implications of this? Is it possible to have a truly honest conversation about impact with a funded organisation, to genuinely encourage curiosity?
Richard: It’s not easy to do this well. There is an obligation on the funder to be responsible. After all, with enterprise grant-making, there is an implication that the funder knows what is best for the organisation it funds, that an uplift in trading and an enterprise mindset will be good for them. At worst, this could become a form of libertarian paternalism. The truth of course is that enterprise grants are not right for everyone. So, funders should be always asking whether putting effort into increasing trading is actually the right way forward for a particular organisation, and whether the organisation sees it that way, or if not, how can it be supported in other ways, that allows it to flourish on its own terms.
Steve: Yes, that is really important – when funders impose their agenda on a funded organisation that rarely ends well. Anything else you would like funders to be thinking about, when it comes to understanding the impact of enterprise grant making?
Richard: Yes, I would like to see funders explore fresh applications of enterprise grant-making. What would happen, for example, if enterprise grants were made available for organisations which have not yet undertaken any trading at all, is there a way to help organisations have a first go, and how can we learn to do that well?
Steve: Enterprise grant-making is still at an early stage of development – there is a lot still to learn.
Sharing experience between enterprise grant recipients
Richard: Exactly. And one more thing, I would hope that funders could come together to support some kind of association of organisations which have received enterprise grants – to enable them to provide collective feedback, to act as a sounding board for future enterprise grant-making initiatives, and to help with efforts to measure and assess impact. Strengthening the voice of those on the receiving end will be essential, it seems to me, to shape the future direction of enterprise grant-making, in ways that are meaningful for them, and for the communities they serve.
Steve: Yes, I can definitely see the value of that, and I hope that will happen. And thanks so much Richard, we’ve covered a lot and it’s been such a useful discussion.
