How far can SROI take us?Charity Insight Contributor
We must recognise the limitations of social return on investment if we are to make it stronger, says new research.
If it wasn't for the big society, 'impact' might have been the voluntary sector's 2010 buzzword. Voluntary organisations are under increasing pressure to measure and demonstrate the positive effects of their work. Government sources and private funders want to know how to get the most from their money and organisations want to be able to demonstrate their value, especially as economic conditions get tighter. Many tools have been developed to provide a basis for measurement but, among these, Social Return on Investment (SROI) has risen to prominence, promoted by a number of government departments and voluntary sector organisations.
SROI is a type of cost benefit analyses. One of the features of SROI is the production of an SROI ratio - which compares money spent with social outcomes achieved. So, for example, a ratio of 1:3 means that every £1 spent produces £3 of social benefit. This is a great way for organisations to demonstrate their benefit in a language that those outside the sector can identify with. It is immediately easy to conceptualise, and arguably contains some of the force that narrative descriptions alone often lack.
One challenge of course is that social benefits are not always easy to identify or quantify, let alone to translate into monetary value. It relies on the judgement, and to some extent the imagination, of those conducting the SROI. So, the increased wellbeing of 30 people might be translated into a monetary value using the annual NHS cost of treating 30 people with depression. But we know that happier people are also physically healthier, so we might add on an increased benefit of X times fewer GP visits, or X times fewer sick days etc.
Such problems have been well documented and in many ways are not exclusive to SROI. How we value social outcomes represents a problem inherent in all types of impact measurement. But while we recognise that no evaluation tool is without difficulty, we also recognise that SROI's growing popularity affords it special need for analysis. The Third Sector Research Centre's recent report, The ambitions and challenges of SROI aims to help make sure new approaches like this can become as workable as possible.
One of the biggest challenges identified by our research is the gap between what SROI seems realistically able to do at this stage and the ambitions that many people have for it. Those who have promoted SROI have warned against its use as a comparative tool. However, our research shows that embarking on a comprehensive social impact assessment is most often motivated by a need to strengthen competitive advantages. We may therefore assume that SROI is, and will be, used in a comparative context.
While the SROI ratio provides a powerful illustration of impact, it is, in reality, a metaphor. It is an example of what the social benefits they create might look like if we thought of them in monetary terms. It does not necessarily represent real monetary savings (although many of them may do), and does not always involve standardised measurements or values. Real meaning is provided by using the ratio in conjunction with the SROI narrative, which explains the assumptions made and how figures have been arrived at. The point is that the ratio itself is so dependent on the individual assumptions and processes used to create it that it cannot be compared between organisations.
What's more, research shows that once completed it is very easy for the SROI ratio to become divorced from the narrative and become the sole focus of the evaluation results. This makes it hard for audiences to assess claims being made, but also places the focus solely on proving impact rather than evaluating success. One of SROI's weaknesses is its ability to attribute impacts to actions, to tell us how and why change happens. It is certainly important for organisations to be able to demonstrate their overall value, but it is also important for them to be able to assess the value of more specific interventions or methods, so that they and others can learn and improve. Without more research, SROI alone may not provide a strong basis for replicating or scaling up interventions.
Certainly one tool cannot do everything and the aim of TSRC's research is not to simply point out the shortcomings of SROI but to help strengthen it. An assurance process and procedures that aim to standardise the way SROI is calculated have been developed by the SROI Network it these need further investigation. Perhaps most importantly, careful attention needs to be paid to how SROI and its results are used. SROI may prove most useful once we explicitly recognise its limitations.
This article was provided by the Third Sector Research Centre and is based on the report 'The ambitions and challenges of SROI' by Dr Malin Arvidson, Professor Fergus Lyon, Professor Stephen McKay and Dr Domenico Moro.