SROI Blog

SROI Blog
 

Collective Impact and SROI- Why Scale Matters

GUEST BLOG BY GARRY HAYWOOD, 6TH DECEMBER 2011

In his opening blog, Jeremy uses the anecdote of his son Jack playing under the desk as a device to explore if an ‘integrated development strategy’ should be simple or not. People like simplicity, it’s easy to understand and in situations like Granby/Toxteth it also fits linear expectations, induced from the deficit model, which suggests a causal link between good policy, right levels funding, a good dose of hard work leading to the creation of the desired impact/value: doing A leads to B. If it was really that simple we’d have solved the problems associated with economic and social deprivation a long time ago. The relations between various agents in the two systems we call society and economy are complex, and this operates at all scales from the human-scale to the global-scale.

However, complexity doesn’t have to be complicated. Sending an astronaut into outer space is a very complicated affair, getting vehicles to proceed around a traffic island is a complex affair. However, accepting complexity does mean a change in paradigm, a loss of pseudo-control in exchange for a better understanding of reality. Complexity is the outcome from dynamic interactions of simple processes such as getting traffic to take turns in crossing the junction from contributory roads and becoming part of the rotary flow around the island until they exit. Complexity increases with the level of variation (distinct factors), dependency (connecting factors) and dimension. A traffic island with 3 contributory roads is less complex than one with five entry points.

As complexity increases, the system becomes more non-linear; it becomes increasingly unlikely that A will lead to B. Making small changes to the operation of the system, can effect the whole system or just parts in a multiplicity of ways, or even not at all. This is why, in policy terms, we lose some pseudo-control. Why do I use ‘pseudo’ here? Because quite often the effect of our intervention is out of our control.

For example, all road users will have experienced a busy traffic island when one car comes out from a junction too early/sharp/etc. We are each in control of our car, but we are not in control of the traffic. The eager driver might cause a local effect so only cars at that contributory junction are effected. It may just as easily cause the whole traffic island to become momentarily clogged-up. And it is equally plausible that this single event causes a tail back of one block or even several. This might even cascade into other events a long way from the initiating event. A simple event can have localised or wider system implications and to know how extended the effect is we may have to consider the system at several scales. If we only look at one junction we might not see the effect to the whole traffic island or the extended road network. If we are looking at a system, the less complexity we have the less we might see any effect, yet equally if we have too much complexity the effect might be buried. This suggests there might be an optimum level of scale(s) for certain types of interventions.

It is important to stress this when we’re talking about systems such as a local economy; we need to think of them as complex systems in which there is a dynamic (flow) from all the actual interactions within the local system and without. We will need to consider different scales of the system, to understand different complexities, different levels of variation and connectivity. We might not be able to predict the outcome because the simple interventions we can make, such as helping to start a firm, have to interact in a system with other firms in a complex way. We certainly can’t measure all the interactions in the system. Yet we can still test if our interventions had any impact, desired or unintended. To do this we must understand at what scale(s) we should consider measuring any changes to various systems that we are trying to affect.

The objective of the strategy in Granby/Toxteth was to intervene in the local system of firms so that we could have more firms offering more employment opportunities, while at the same time improving education and training provision for residents so that they might take up those opportunities. We might summarise that we had four simple groups of interventions to achieve our objective: adding more new firms; helping existing firms to grow to take on more employees; improving human capital; improving the match between job opportunities and human capital development. So far, so simple.

Where this becomes complex is the firms that are supported must interact with other firms in the market place. A new firm would add some employment to the area, but it might destroy employment in a competitor firm. Imagine supporting a community shop that wins customers from an existing shop which must now close because it cannot sustain itself on less custom. The intervention has negated itself. Likewise supporting an existing firm to become stronger may make it impossible for new firms to gain traction and become a sustainable enterprise. The same events might just as well work. Starting a new business in an area may make the existing business look for customers from a different market. On a programme wide basis we might expect all these to be true. Consequently, matching the training needs of residents to firms future needs is reliant on getting the firm interventions right, otherwise the mismatch may continue or increase.

When we start out we just don’t know what will happen, only what we hope will happen. That’s what happened with the GTDT strategy. Lots of careful preparation. Lots partnership working. A dash of optimism. Yet, as Jeremy wrote, there was little overall effect from the interventions. Without a full evaluation it is difficult to know why this was, but on reflection I suspect that we didn’t sufficiently consider scale, for both the intervention or the measurement of its outcome.

There is lots to think about when considering scale. Foremost in my mind is the area we were working on. It had an arbitrary border, which was largely a political construct hewn out of administrative boundaries. What if our area was not actually a system but only part of system? For example, consider a circle on a piece of paper. If we zoom out at a certain scale the circle will become a dot and then disappear. If we zoom in we will lose the edge and we will only see space. If the spatial scale of our intervention or measurement is wrong then we may not create any impact or, worse still, we may have some impact but not be able to see it, or even worse, we may see some impact that was generated by other influences and claim it as our own leading us to bias our own methodology.

We certainly didn’t consider the scale of dependency between the SRB Partnership and other areas. The GTDT area is an inner-city area that borders on to the CIty Center. It is possible that any effect our interventions had were limited by the proximity of this more potent force.

So what to do about scale? I’m not entirely sure although I have some thoughts about statistical distributions, particularly those of a power-law nature and also about interventions that are specified to have wider-system effects but the only measurable value is at the human scale. I’ll save developing these thoughts for a future contribution to this discussion. But I’d like to end this blog with a brief return to our traffic island metaphor. If we wanted to regulate the traffic flow onto the island, would we do it with lights at the junctions to the island itself or further back on the approach road? I don’t want to propose an answer here, but let you think about it while highlighting that it is a question of scale.

Add new comment
 

Creating more value with what we have

There are so many different approaches and opinions around ‘accounting for value’ that it’s difficult to see the wood for the trees. In Making it Count, a new publication launched yesterday at the Good Deals conference, Jeremy Nicholls introduces some of the key players and principles in this hugely important area – and suggests how to find a clearer route through the forest.

The last couple of years have seen a rapid increase in interest in social and environmental accounting. Questions of how we know we are making a difference and what sort of difference was it anyway are recognised as fundamental.

This is all to the good. Not so good is that, while organisations and their investors are recognising they probably should be doing something more, they are not sure what. There seems to be a profusion of approaches and arguments over which approach is best.

A UK-based think-tank, the New Economics Foundation, has published a guide listing 20 different quality and social impact frameworks for civil society organisations. Meanwhile, a database called TRASI (Tools and Resources for Assessing Social Impact), developed by the US’ Foundation Center with Mckinsey&Co, lists more than 150 tools and approaches.

Of course, not all these approaches try to do the same thing. It’s also clear that some sort of classification scheme – what academics call a ‘taxonomy’ – is needed. Yet the idea of a taxonomy is perhaps even less engaging to most of us than the idea of measuring impact. So we must be careful that any work in this area does not become a dry academic exercise but is related to a mission and to a story of positive change.

Just look at financial accounting. We all know we have to have financial accounts.

Some of us can read them but most of us are relieved we aren’t tasked with preparing them. And they become more relevant to us when we understand them as a management tool for building better business.

In the civil society sector in the UK, social accounting and accountability have had a long history and, more recently, social return on investment (SROI) has gained support. In the ‘for profit’ arena, organisations such as the International Integrated Reporting Committee (IIRC) and Global Reporting Initiative (GRI) have been centre stage both in the UK and around the globe. More recently GIIRS (Global Impact Investing Rating System) and IRIS (Impact Reporting and Investment Standards) have generated interest amongst ‘impact investors’ – itself a relatively new term.

The challenge must be to arrive at a more standardised approach, one which allows us to communicate the value we create in a reasonable way.

In an attempt to work through this challenge, the SROI Network has argued for an approach to what we call ‘accounting for value’, which answers a series of questions within a framework. The questions become principles and the advantage should be that the debate could move to what is the minimum set of questions that should be answered and then on to how those principles should be applied. This approach remains flexible and recognises that value will be different for different people in different circumstances – so there is no absolute list of indicators – and can be used for different purposes and different audiences by organisations of different sizes.

To some extent this approach has been lost in preconceptions of SROI. So far the brand name and its history have been stronger than the message. And various myths abound (which we address in the ‘myth busting’ section later in this document). Another challenge is that it still seems that we are waiting for a way to know objective answers. SROI argues that applying these principles will require judgements, subjective inevitably, where what are accepted as good judgements will develop over time from the bottom up. No different to financial accounting.

The good news is that over the last couple of years more organisations and their investors have moved from a focus on outputs to outcomes, and are recognising the importance of involving stakeholders in determining outcomes. They also understand better the need to know how valuable these outcomes are, and to consider what might have happened anyway (in other words, how much people’s lives or a particular service might have improved naturally, without a particular intervention taking place).

These questions are surely fundamental to all our endeavours, and we hope that leaders in the social business arena – such as the social investment wholesaler Big Society Capital in particular – will find ways to ensure that those organisations receiving investment can answer these questions.

And the principles can also help us to develop a more dynamic, practical taxonomy. For example, the tool LM3 (Local Multiplier 3, which allows you to calculate the economic contribution your organisation makes to a local area, can be seen as a way of measuring a specific outcome. IRIS and SROI Network have recently documented the relationship between their two approaches. SAN (the Social Audit Network) and the SROI Network have done the same. The principles of including what is important has drawn on work by AccountAbility.

In drawing up the current Guide to SROI, staff from the Green Book team in the UK Treasury, who set out the framework for evaluating all government policies, were involved, and guidance on using SROI in commissioning was also shown to the Treasury’s Office.

All these interrelations are challenging but below the surface there is more and more convergence. It is becoming possible to differentiate, for example, between approaches that focus on accounting for value, approaches that seek to understand the relationship between inputs, activities and outputs, and approaches to develop indicators and tools to measure specific outcomes.

The Making it Count publication – produced in partnership with Buzzacott accountants and SocialEnterpriselive.com – draws together some of the current thinking and practice around accounting for value. It explores some of the challenges and myths, and provides insight into why and how investors, commissioners and intermediary organisations are assessing the value created by the organisations they fund and support.

Importantly, it also spotlights some examples of accounting for value in action – based on a range of case studies written up for the SROI Network.

Making it Count is available to download here. The publication also includes a wide range of links and signposts for further information.

Add new comment
 

Collective Impact and SROI

BY JEREMY NICHOLLS

With all the current interest in collective action in order to create significant and sustainable change, I can’t help thinking back over the UK Governments long and cross party history in investing in collective action to achieve sustainable change; from City Challenge, the Single Regeneration Budget (SRB), Local Strategic Partnerships and Local Economic Partnerships.

In 1994, I was working with Urban Strategy associates and for one of the SRB Partnerships in Liverpool, in Granby and Toxteth. The SRB seemed to have all the requirements for success; a central point, regular communication between those involved, clear goals, shared measurement, processes and metrics. One of the areas we were focusing on was enterprise development; the goal was to increase numbers of enterprises and therefore employment opportunities in the area.

All went well. We gathered all the agencies involved in enterprise support, from the government backed Business Link to the local Yemeni Business association. We worked with them, first to agree a reasonable target for a net increase in enterprise numbers in the area, and then to work out how the organisations would ‘share’ that target, contributing as appropriate to help businesses seeking support. Otherwise, multiple claims of number of enterprises starts supported far exceeded the actual numbers of new businesses.

We agreed system wide targets and organisational targets within this; we set budgets, allocated resources. We agreed a shared measurement system using a spreadsheet model to capture information on businesses supported, and collate this across the different organisations involved to compare actual performance against our shared targets. However in the end, the number of enterprises as a share of those in Liverpool didn’t change significantly, and neither did the number of job opportunities. Something or things hadn’t worked as we had all hoped.

I remember presenting the strategy to the board of the SRB partnership on an evening when I had my three year old son with me – playing under the board table. At one point one of the local business representatives on the board said “this is all very complicated” and a small voice replied from under the table “no it isn’t, it’s easy.”

I suppose it depends on your point of view, but at the time we thought it was pretty simple and that it was the idea of collective action of shared targets that was complicated. Turns out of course it was more complex than we thought. Amongst the reasons, and there are several, were the questions of who we had involved in setting targets, and what was the size of the area over which we were working.

We had missed out private sector business advisors, some private sources of finance (we had one of the high street banks), more informal sources of finance and advice, potential customers, owners of land and property and so on. Small wonder our carefully laid plans were not successful and yet this was not strange at the time, and would not be strange today.

More fundamentally we hadn’t realized that the system we were working in was just too small. The structural issues that were facing businesses were operating over a larger scale than Granby and Toxteth, growing businesses may just move out, competitiveness would be affected by a new shopping centre just outside the area. And this is the bigger question. What size of area should you be working at? Too large and you will never be able to start. Too small and you won't make a difference. We now think that the principles of SROI would help. If we had spoken to stakeholders, owners of businesses, owners of land and property, customers and thought about who else was involved in creating change more clearly, we would have wanted to expand the scale. The problem is that this is so often a given, someone somewhere decides that a project will cover a town, a community, even a region but unless some consideration is given to the scale at which the project needs to operate, it risks being unsuccessful.

I am hoping that my friend and colleague who I was working with at the time, Garry Haywood, will be inspired to help us out with some ideas on setting the right scale for shared action.

Then with the right objectives, people, the central support, the shared management, and regular communication, maybe we can create sustainable social change.


Add new comment
 

A focus on objectives

Objectives! Do we have them, are they any good and are we achieving them? All good questions. And the current enthusiasm for impact; are we investing in it, commissioning for it, or delivering it, is also very encouraging.

However there is a big danger in confusing these two. And yet discussions about impact too often start with the questions like, what are we trying to achieve or what impact do we want to have? This initial focus is dangerous because impact can be much wider than our objectives and because the wider impact could be negative. Ah unintended consequences. Yes but by starting with objectives these unintended consequences become an afterthought, an add on to the main course. It becomes too easy to miss them out or argue that they are not really significant.

Where the discussion about impact has started from the perspective of evaluating our work it is more likely to focus on an organisations objectives, at least as a starting point. Where it starts from the perspective of being accountable for the effects of our work it will be a broader conversation.

To be accountable, the starting point for impact should be what has been the impact of our work – regardless of what our objectives were. This is messier because there will now be many impacts, both positive and negative, and we will need some way of deciding which are impacts are the important ones – without falling back on our objectives as the main filter for that decision.

In a simple example of an organisation that works with people who have been unemployed for a long time and hopes to help them gain work. 30% of those they help do gain employment (and lets leave aside questions of how long this lasts, whether it would have happened anyway and how much was a result of the work of this organisation). What happens to the other 70%? If any of them feel a sense of failure that it hasn’t worked for them and have less confidence than they did, they are at best now further away from getting work than they were. How does this trade off against the 30% that are in work?

How many organisations do you know whose objectives include something about minimizing the risk of negative consequences of their work? 

Comments (1)
 

Not for profit?

I wonder if there is any chance we could stop using ‘not for profit’ to describe a type of organisation. I know people sometimes say that this is short hand for ‘Not for profit distribution’ but leaving aside all the questions this would raise, it still leaves the more fundamental problems unresolved.

Firstly it is not very inspiring to describe organisations by what they are not.

Secondly ‘not for profit’ is the same as ‘for loss’.

Thirdly it is a reason not to address the far more important question which would come from a change of name.

So how about For Social Profit (FSP). Or for those who don’t like the word profit, as in ‘to profit at someone’s expense’, For Social Value (FSV).

These names would help us focus on the main purpose - creating social value – and help us to think about how we can create more value with the resources at our disposal.

Comments (8)
 
Page 1 of 2
 

Menu - Hidden Nav