- What is SROI?
- SROI Analysis
Accounting for Value
This blog will aim to look at similarities and differences by comparing the two from the perspective of each SROI principle. It's also available for download as a document here.
Though SROI does draw from Cost Benefit Analysis (CBA), it developed drawing on two other traditions; sustainability accounting and financial accounting.
This principle is fundamental to the SROI approach, and is followed in all aspects of SROI. It is especially important to involve stakeholders when trying to determine outcomes, or the changes that result from an activity.
CBA can focus on a particular policy issue that is being considered, and doesn’t implicitly require involvement in deciding what outcomes are, though they may be consulted.
Both CBA and SROI focus specifically on change, predominantly change in situation, capacity or behaviour with related changes in wellbeing. The only slight difference is that SROI has an assurance process to ensure completeness of change.
This principle identifies the most notable difference between the two approaches.
CBA is an aspect of welfare economics, so begins from the perspective that all welfare effects will be included. In practice, however, it often focuses on a particular policy outcome with some recognition of unintended consequences. Whilst it is not possible to consider all welfare effects, focusing on a policy objective without stakeholder involvement risks omission of important effects.
SROI on the other hand recognises these limitations and aims to include material outcomes, drawing on financial and sustainability reporting, which hold materiality as a central tenet.
Materiality is not the same as proportionality. Materiality requires a decision to ensure that the outcomes included, both positive and negative, are those that, if omitted, would affect the decisions of the stakeholders. This process requires a judgement on the extent to which stakeholder groups are split into smaller groups which experience different material outcomes – a balance is required between considering every stakeholder as an individual case with a personal outcome profile, and aggregating stakeholders together and so risking a loss of understanding how different smaller groups experience different outcomes.
Proportionality means that the extent or depth of the analysis should be tailored to the relative size, impacts, and risks of the activity under analysis.
This need to consider benchmarks, counterfactual, attribution and displacement is shared by both approaches.
Whilst both approaches use money to value benefits, the valuation technique and/or perspective from which the valuation is taken can differ.
Many CBAs focus on value from the perspective of real, potential or imagined savings to the public sector. This value can either be expressed as an indirect financial value to the taxpayer, or sometimes as a proxy for achieving a stated social policy goal.
Although some SROI analyses do use these types of values, they also aim to value outcomes from the perspective of the stakeholder. This could be by using revealed preference, stated preference, or (more recently) wellbeing valuation techniques, applied as appropriate to the outcome or stakeholder.
This principle should be another area of overlap between SROI and CBA.
SROI follows accounting and some sustainability reporting by requiring appropriate verification of the result. This may well take the form of an external assurance process, but could include a range of other methods of verification (such as going back to the stakeholders and asking their opinion on the results of the report).
The reasoning behind this principle is that an SROI analysis will inevitably include judgements about what is included or excluded, and these judgements need to be reviewed as reasonable or unjust.
CBA does not have an audit process. However, since judgments on what is included and excluded are made and the audience need to know whether these judgements are reasonable, an external assurance of those decisions, acting on behalf of those affected, is required.
CBA tends to be used by the public sector or quasi-public sector, so is often applied with a reasonably high level of rigour.
In contrast, SROI principles can be used at any level of rigour, as long as it is ‘good enough’ for the type of decision it is being used to inform. In this respect it is similar to financial accounting, which aims to provide information that is good/accurate enough predominantly to inform investors, as opposed to social science levels of rigour.
At one end of the spectrum SROI can use similar levels of rigour as CBA, with additional requirements for consideration of materiality and assurance/verification. At the other end of the spectrum a much lower level of rigour could still be good enough for board-level strategic decisions within organisations.
An illustration of this would be the Maryland Scientific Methods Scale. This is a scale that ranges from 1-5, and indicates the level of scientific rigour of a particular study. At the top end of the scale lies Randomised Control Trials (RCTs), which can be used in an SROI context but depends on use and purpose for the SROI analysis.
At the bottom end of the scale would be an assessment of the counterfactual simply by asking beneficiaries what would have happened otherwise when they enter a program, and does not have any other use of control groups etc. This type of rigour would be unlikely to influence policy, but is still useful for designing services.
This different application of rigour in SROI applies to valuation as well; an estimate for a value may be good enough for a particular audience. Similarly, in accounting we see examples of judgement being used in figures such as the bad debt provision.
In summary, whilst CBA and SROI do have areas of overlap, their differences originate from the fact that SROI is additionally informed by financial accounting and sustainability reporting, particularly with respect to materiality, verification of a result, and use of different levels of rigour according to use and audience.
Social Return on Investment (SROI) and Social Impact Bonds (SIBs) are two ideas that are increasingly mentioned in the same breath. SROI is a measurement and accounting framework and SIBs are a way to contract and finance a service. Both require three common ingredients:
While not a necessary ingredient, SROI can contribute to the design, operation and evaluation of SIBs.
*NB the word ‘outcome’ is used here to represent a change in someone’s life – some readers (particularly from the US) may use the word ‘impact’ to mean the same
A Social Impact Bond is a contracting and financing model that involves a number of changes for most contracting environments. The shift towards outcomes-based contracting often poses the most significant change. It is challenging because the quantification and valuation of outcomes is new and difficult.
For many people, SROI is synonymous with the identifying and valuing of social outcomes of a program or organisation. It is a framework that consolidates measurements and values, to compare the value of inputs and outcomes. SROI is attractive for use with SIBs because it explicitly collates both financial and non-financial value.
SROI does not dictate how to measure effect size (change in outcome), or value outcomes, but does offer guidance on different tools that can be used for different purposes. To share other practitioners’ outcomes and values, the SROI Network has established the Global Value Exchange database. SROI assessments can be used to examine the effectiveness of one project, to compare different project designs, to compare different projects or to examine changes over time.
The principles of SROI are consistent with SIBs.
SROI has not been used in the development of Social Impact Bonds to date, but conversations involving both concepts are increasingly occurring across the globe: in the UK; Japan; Australia; and Canada. The diagram below maps some of the opportunities for using SROI in the design, operation and evaluation of a Social Impact Bond.
This post is a blog from Emma Tomkinson, originally printed on her personal blog here. Emma is a Social Impact Analyst working in Australia, and an active member of the SROI Network.
The extent of debate (and often disagreement) about the definition of social impact investment is fascinating – exploring in great depth the nuances and prerequisite principles for investing in a way that seeks both positive social outcomes and financial returns. But this discourse risks being divisive and self-defeating. Does this complexity actually attract or repel new investors from engaging in this exciting market?
Inspired by this documentary on risk takers featuring Elon Musk, the South African born founder of PayPal, Tesla electric cars and SpaceX, I have renewed respect for the visionaries who just get on with showing others how amazing things are possible. General Colin Powell once said “Great leaders are almost always great simplifiers, who can cut through argument, debate, and doubt to offer a solution everybody can understand.”
I don’t think General Powell would approve of how we are explaining social impact investment to the masses – and this possibly reveals why most people don’t know anything about it.
Earlier this year I was delighted to be invited by the World Economic Forum (WEF) to attend a roundtable discussion in London, the topic was to explore how to mainstream impact investing. It was duly attended by the good and great, indeed it was a high quality discussion and I walked away with new friends and fresh insights.
But during the day of discussions, the voice of my business partner Rupert Evenett circulated in my head. He rightly says that most frontline social purpose organisations (from charities to entrepreneurial social enterprises) really don’t care where the money comes from – as long as it is legal and above board – they just need more of it to improve the amount of social good they can deliver. An intentionally provocative point.
Yes of course, the funding must be appropriate – sometimes grant funding, sometimes equity investment and sometimes loan finance. But the point is a simple one, it really doesn’t matter what it’s called and whether it’s branded as social impact investment or not. What these organisations do care deeply about is how easy it is to access funding and whether the terms are appropriate for what they need the money for.
In the wrap-up of the WEF event, I reflected on the day’s discussion and on the fact that it was still not possible for everyone in the room to actually agree what social impact investment is. I made the point that ultimately investment is about risk and return. We should avoid making social impact investment seem more complicated than necessary by waxing lyrically about the complexities of impact measurement.
If we really want to make it go mainstream then we need to describe its characteristics in the language of finance – risk and return. Simple.
The risk of social impact investing is still perceived to be high, but nobody can honestly say what it really is beyond a few anecdotal facts from a handful of deals they know about, or quoting from a report that was written with some inevitable bias. The reality is that we need to improve the time-series data available on deals. “Data unlocks investment capital” as my colleague Rupert likes to say. Transparency will cut through the uncertainty and reveal what the real risks are – and I’m willing to bet that it will be lower than many think.
What about the second dimension of finance – returns? There is certainly great debate about whether social impact investment can deliver market rate returns or whether a trade off between social and financial returns should be expected, or even be a necessary precondition.
Enjoying the opportunity to provoke debate, at the end of the WEF event I suggested to the group that perhaps the creation of positive social outcomes could be seen as a proxy for future long-term financial performance. Or explicit positive externalities to frame it more in the language of finance.
By this I don’t mean maximising financial returns, but I do mean an indicator of broader financial attributes, such as resilience to uncertainty and therefore – being inherently about delivering social value and also probably more likely to be valued by society – able to suggest better ability to resist unpredictable economic shocks or other financial speed wobbles.
The room looked back at me without comment – in fairness I did make this point at the end of a long day.
A few months later I attended a debate about the the European Commission’s Green Paper on long term financing – for example, the financing of major infrastructure projects or SME investment tied up for 10 years. I pointed out that the poor liquidity and by definition long term nature of this market segment closely resembles the social impact investing market.
I asked the speaker (from a large multinational asset manager) how they evaluate their long term investments during the course of the investment period. How do they know whether the investments are doing well or not? Leading to the obvious decision about continuing the investment programme or quitting the deal.
His answer surprised me. He said they increasingly ask what their investee organisation actually do in the real world. In other words, what their social value to society is. Now this is not explicitly a metric of social outcomes as many social impact investors would see it, but it does support my assertion that social impact indicators could – and do – serve as proxy indicators of financial performance.
If we are going to stand a reasonable chance of mainstream investors deploying more capital in the form of social impact investment, then we need to make sure it is easier (not harder) to make it part of their day-to-day asset allocation decisions. This objective is shared by the European Investment Fund with their Social Impact Accelerator – a pilot to explore how more private capital can be leveraged along side public capital in creating a more vibrant and higher volume capital market for social impact investment.
To follow Powell’s guidance, we need to simplify the narrative of social impact investment and tell the story in the terms that capital markets understand – namely risk and return.
And to make it even easier to understand, we should call social impact investment by the name that has the most chance of attracting mainstream investors – investment – and let the rest have complicated nomenclature such as ‘impact agnostic investment’ or ‘socially useless finance’ as Lord Turner refers to it.
With this spirit of intense pragmatism in mind and a simple desire to see a better capitalised ecosystem of organisations that explicitly do good for society, Rupert and I are building EngagedX, a financially orientated index and data platform for
social impact investment. I will be speaking about this project at The SROI Network’s Annual Conference Social Value Matters, or see the presentation here, which explains more about our EngagedX pilot and what we plan on doing next. It is an evolution of the material I used in a lecture to MBA students at Oxford University Saïd Business School. Enjoy – your comments and dissent are welcome!
Less than a year ago, I was delighted to be invited to attend the SROI Network’s annual conference in Liverpool, where I made a tentative presentation around ideas HACT (a UK-based housing ideas and innovation consultancy) were developing in partnership with SROI Network advisor Daniel Fujiwara on new approaches to measuring social value, drawing on Daniel’s Wellbeing Valuation methodology, which has been generating increasing interest in both the UK and a range of OECD member states.
I’m now massively looking forward to this summer’s Social Value Matters, bravely scheduled to take place in Milan, in the two days leading up to England taking on Italy in their opening tie in the 2014 World Cup. It is an event which looks set to highlight the extent to which the SROI Network has succeeded in mainstreaming social value thinking, across public, private and civil society sectors and internationally. In the session I’ll be running with Daniel (now heading the ground-breaking social value consultancy, SImetrica), I’ll hoping to be able to share progress, and benefit from the input of an even wider group of social value thinkers and practitioners as we move from concept towards what we think will be an important and viable new approach to understanding and maximising organisational social value.
Social Value Matters is coming at an important time for HACT and for the wider social value community. As concepts of social impact and return grow in visibility and are more broadly embedded in decision-making at all levels, some of the questions I raised in Liverpool have become more important. In particular, the extent to which conventional SROI approaches are capable of providing support for decision making in larger organisations who may be investing resources across multiple areas of social value, and require strategic-level (as opposed to evaluative) tools to help them with that process
The challenge we have been seeking to address is whether it is possible to develop cost-appropriate approaches to social value measurement and decision making which are capable of helping complex organisations understand and compare the potential value of disparate community focused initiatives using comparable measures and metrics. And to do so in a way that retains the core principles established by the SROI Network and are consistent with and complementary to conventional SROI approaches.
This is particularly important in our core sector of housing. In the UK a housing association (not-for-profit landlords, providing 1 in 8 homes in England and Wales) might typically invest several hundred thousand pounds a year in support of activities ranging from community gardening to youth sports to employability initiatives – and want to be able to show to its residents, governing board and regulator that their investment decisions are underpinned by an understanding of the value they are creating.
In developing our approach, we’ve looked to focus on three key elements:
One of the biggest successes of the SROI Network has been the building of what was WikiVOIS and is now the Global Value Exchange. Access to high quality social value proxies is a critical component of any SROI evaluation, and the Exchange has enabled social value practitioners to quickly find and apply more and more robust values than at any time in the past. It’s a great resource that continues to grow in scale and importance.
But the size and scale of the Exchange is also a weakness when it comes to assessing comparative social returns across multiple areas of social activity. The sheer range of values on the Exchange, generated by different academics and researchers using different methodologies at different times to assess both impacts and then to monetise them make it hard to assess comparative impacts across differing areas of activity where values are derived from very different methodological bases.
That is why we have prioritised work with Daniel Fujiwara to create a single, methodologically consistent set of social value metrics covering the widest range of community investment activities, drawing on his Wellbeing Valuation approach, which involves the econometric analysis of very large national datasets, and produces social impact values at a level of robustness which has seen it recognised and used by the UK government in assessing social value of a range of its own activities (most recently around sports and cultural activities).
Whilst the sheer breadth of the value set means that it cannot always go into the same level of detail as some more thematically-focused valuation research, the fact that all the generated values come from the same methodological base means that valuation reports that use them will inherently be comparable one to another. We’ve made the new value set available (subject to some limited licensing conditions) on the Global Value Exchange, as well as from the HACT/SImetrica Social Value Bank, on which we’ll be publishing future Wellbeing-based values as they are produced.
Conventional SROI approaches provide great in-depth insight into the social value created by relatively focused interventions, but they are less effective as a means of modelling and comparing the prospective social value created across a portfolio of social investments. This isn’t just a matter of comparability of metrics – it is also an issue of cost and timing. It may not be cost-appropriate to commission full SROI reports on all areas of activity, and in any event it may not be possible where activities may be at a planning rather than implementation stage. This can make it challenging to achieve any sort of robust assessment of the comparative social returns of a portfolio of activities needed in order to assure a business it is planning to achieve the optimal social value from its investments.
That is why, after extensive practical testing with partners, we’ve released a lightweight, but powerful Social Value Calculator model that utilises our new metrics set, enabling easy assessment of the potential value generated by social investment activities and portfolios, and the ability to model potential outputs in “sandbox” fashion. And later this year, we’ll be releasing (initially for housing providers and, later, more widely) Value Insight – a next generation online social and economic impact mapping and modelling product that will take modelling, reporting and social value analysis to the next level.
It is important to recognise what our new approach does not do. With a value set based on analysis of responses by individuals to extensive panel interviews across massive samples over many years, it does not place the same emphasis on project-specific stakeholder engagement that sits at the heart of an assured SROI report. And the scope of the analysis, key outcome measures, and ‘story of change’ are necessarily set by delivery agencies, rather than project beneficiaries (although there is provision in our model for limited stakeholder outcome survey work). We’ve also sought to reduce the extent to which calculations of value are subject to sometimes arbitrary adjustment factors, which can have a significant impact on social return calculations whilst being difficult to robustly substantiate, in particular in relation to attribution, deadweight and drop off. Instead, we have opted for a much more measured use of terminology when talking about the contribution of any individual project to the wellbeing to beneficiary groups.
In doing so, we have sought to reduce some of the areas of SROI practice which in our view carry with them the highest levels of cost when compared to quality of insight they generate, whilst significantly improving the extent to which robust and cost-effective impact comparisons can be made across businesses.
Whilst there are clearly some key differences of approach between the HACT/SImetrica model and conventional SROI, it is critically important to us that all of this sits within the context of existing social value and social return practice. The Social Value Bank will be fully accessible to those seeking to use its value for more conventional social return analysis, and the values generated by our Social Value Calculator, are intended to be compatible with and comparable to those generated by SROI evaluations (whilst recognising that these will typically go into more depth than the assessment provided by the calculator model). This provides scope for those organisations who might only be able to afford a conventional SROI on one part of their social investment portfolio to nevertheless access a cost-appropriate and robust assessment of the overall social value they create, helping with both business decision making and project/impact evaluation.
The demand for this sort of model is already evident. Since we launched Measuring the Social Impact of Community Investment: A Guide to using the Wellbeing Valuation Approach and its associated Social Value Bank in March, over 700 organisations have accessed guidance and values, with interest extending well beyond HACT’s core housing market. This makes it (for the time being) the fastest growing new social value product launched in recent years.
We’re hugely excited by the impact of our new social value framework, and its potential as a tool for mainstreaming social value at a strategic level in organisations making choices and delivering impact across a range of differing social priorities. HACT has already secured funding to support the development of further social values compatible with our new approach, working in partnership with Daniel Fujiwara and SImetrica, increasing its relevance and viability as a tool, and will be publishing v2 of our model and values later in 2014-15.
It is early days yet, and we’re still learning, developing and putting things right: we think the model and values are great, but at the moment, it’s still a work in progress rather than a final product. We’re hoping that – as our Social Value Bank grows, and our methodology and model develops - we’ll benefit from developing practice and learning across the SROI field, and contribute to continued significant growth in the social value marketplace. Its why I am very much looking forward to sharing where we’ve got to and benefiting from the great range of speakers and delegates who are going to be in Milan in six week’s time.
To find out more about Social Value Matters, The SROI Network's annual conferece, head to the event website here.
Jeremy Nicholls 13th January 2014
The SROI Network is continuing to grow. There are now nearly 900 members in over 40 countries. There are 8 national SROI networks, from Japan to the UK as part of the international network. Guidance is available in five languages and being translated into a further three this year.
Despite this growth and the growing recognition of the importance of social and environmental issues, there is still nervousness about having one way of measuring social impact. It is true that there are many social impacts which may need different ways of being measured but this is not the same as there being many ways of deciding which impacts should be measured.
Over the last few years of working on social return on investment, I haven’t seen any approaches which bring more than SROI, although there are many approaches which do less. SROI is based on 7 principles which ensure that anyone accounting for and reporting on their impact can answer a number of questions:
Who has changed?
How have they changed?
How much change was there?
What would have happened anyway?
Did anyone else contribute to the changes?
Which changes are important enough to manage and account for?
How valuable are the changes to those who changed?
How much credibility does the analysis have?
Who answered these questions?
Answering these questions is fundamental to an account of social impact. Some approaches answer some of these questions but not all. If any are being missed then there is a higher risk that the information on impact is incorrect.
But I do see a wide range in the level of rigour being applied in answering these questions
How much rigour is needed depends on the people who are going to use the information on social impact. If the information is going to be used to make or influence decisions, and there seems to be little point in accounting for and analysing impact unless the information is going to be used then the more rigour the less risk that the conclusion will be incorrect although sometimes the more time and resource that is required.
The SROI Network provides an assurance framework which seeks to ensure that all these questions are being answered at comparable levels of rigour and at a level which shows that the principles are understood.
Some methods answer some questions at a higher level of rigour but don’t answer other questions at all. Some audiences want a high level of rigour for one question but are less concerned about the others, for example where a high level of rigour in answering “what would have happened anyway” has led to the use of randomised control trials even though the change being examined has been decided on without any stakeholder involvement. This is not that uncommon and could mean that a great deal of time and energy is being spent on assessing an outcome which is either not the material outcome arising from the activity and or is not of any real value to stakeholders compared to other outcomes that are being experienced.
Most organisations are doing something and sometimes a lot of work on a particular question but when they do a quick self-analysis discover that they are doing much less or none on some of the other questions. This is a good guide on where to improve practice since there is always a temptation to get even better in an area where an organisation is already good.
It may not be appropriate or possible for an organisation to answer all of these questions but organisations should be aware of the risks of not answering them and should plan to address them over time.
If the information is going to be used to make different decisions on allocating resources then it needs to be credible to those that are going to use it and our experience is that people always come back to these questions and understanding how they have been answered before they are happy to use the information.
All these questions need answering, but if there is a place to start then it’s on understanding who changes and how they change and ensuring that those affected are involved in reaching this understanding.