How to improve your social value/impact measurement

Who are your stakeholders?

Group discussion to list your real stakeholders

Difficulty level: easy

Time: 30 mins

1. Bring together a small group of whomever you can from staff/volunteers/board members/ beneficiaries/wider community.  You may find this easiest to do as part of something else – a regular management meeting or perhaps a planned consultation event.
2. Ask your group “who are the people and organisations we make a difference to?” and list the answers on a flip chart.
3. Prompt for more by asking “who do we affect indirectly?” and “who do we affect in a negative way?”
4. Finally ask which of these groups and organisations you currently talk to or liaise with and which you do not.  Are there any people or groups or organisations you may make a big difference to but who you don’t talk to?  is there anything you would like to do about this?

Talking to stakeholders

Ask questions to understand your stakeholders perspectives

Difficulty level: Moderate

Time: Variable

1.  Identify a list of stakeholders to speak to – make sure that this includes representatives of most of the groups you make a lot of difference to.
2. Ask them “what change have you experienced as a result of your involvement/participation in (or as appropriate) our project/service/organisation?”
3. What changes do you hope to see for you and/or your organisation as a result of our work?

Try to focus all your stakeholders – not just your primary beneficiary group (if you have one) – to focus on the changes they as individuals or an organisation experience, not what they want to see for others.

Listen to what they have to say and note it down, particularly when it doesn’t match in with your goals or those of your funders.

Don’t just consult - involve

Involve your stakeholders in planning your evaluation

Difficulty level: Hard

Time: 2 hrs+

1.  Start by identifying some stakeholders that are interested in evaluation – by asking them.  Pay particular attention to people who experience change as a result of what you do but are often treated as beneficiaries rather than planners or decision makers.
2. Set up a small group discussion to answer three questions:

• What do you think are the most important changes we produce?
• How could we know that these changes are happening?
• How could we involve you in understanding the value we create in the future?

Your story of change

Listing and sorting your outcomes

Difficulty level: Hard

Time: 2-3 hrs

1.  Start by listing the outcomes you deliver – if possible based on discussions with your stakeholders (see earlier section).  Outcomes are changes and you can usually tell if something is an outcome because it starts with:

• More……Less….Longer….Shorter….
• Better…Improved…Reduced…Greater

   2.  Write each outcome on a card and then group them into short, medium and long-term (decide on your own definitions).

   3. Write these up – you have a draft story of change!

Negative outcomes

Group discussion to explore negatives

Difficulty level: Moderate

Time: 1-2 hours

1.  Invite a small group (stakeholder groups, staff advisors or others) to a short brainstorming session.
2. Start by listing your top stakeholder groups (no more than ten) – that is the top ten groups of individuals or organizations that you make most difference to.
3. Taking each in turn try to list at least one negative outcome, disadvantage or potential damage you do or may do to each group.
4. Now consider if there are any people or organizations who you only affect negatively – include the environment in your consideration.
5. Again list any negative changes or outcomes you deliver for these stakeholders.

Where is your evidence

Reviewing what you have

Difficulty level: Moderate

Time: 2-3 hours

1.  Choose the 5-6 most important outcomes or changes you create.
2. For each one set out what evidence you currently have about this outcome, it will probably help to think in terms of:

• Reported evidence;
• Observed evidence;
• Evidence gathered through tools or questionnaires.

How important are your outcomes?

Comparing outcomes

Difficulty level: Medium/Moderate

Time: 1-2 hours

1.  Draw up a list of the most important outcomes you deliver for you key stakeholders – a total of between 6 and 10 is reasonable.
2. In a group session ask participants to allocate “importance points” to each outcome so that for example if you have 10 outcomes each person will allocate one point to their lowest valued outcome and 10 to their highest.
3. Collect the “votes” and produce a prioritised list of the outcomes.
4. Where there are any major differences in ranking between people ask participants in your discussion to explain their reasoning.
5. If you can reach consensus on the overall priority list then do so but the main focus of this exercise is to consider the relative importance of your outcomes.

Using published research to identify saving you deliver

Desk research

Difficulty level: Medium - Hard

Time: 3hrs – few days

1.  Agree a list of key outcomes and stakeholders (for each outcome) you wish to assess.
2. Using Google try a succession of search terms linking your outcome with phrases such as cost savings, cost reduction, fiscal savings.
3. For each outcome review 2-3 pages of Google search results and download any documents or sources that would appear to match well.
4. Read the sources and identify if any provide a value.
5. Assess the validity of the source – that is how well does it match your situation and how much proof does it give.

Trying out the value game

Experimenting with stated preference

Difficulty level: Easy

Time: 1 hour

Look at the website www.valuegame.org and consider whether you could use this approach with some of your stakeholders to understand the importance or value they attach to the outcomes you cause.

This approach is an engaging and practical way of getting people to think about values using a card game based on accepted economic techniques.

What matters to your stakeholders?

Putting yourself in others shoes

Difficulty level: Easy

Time: 30 mins

1. Bring together a small number of your team, or put aside some time at a management meeting.
2. Choose a group of important stakeholders such as your key beneficiary or customer group.
3. List or ‘brainstorm’ all the differences that you make to them.
4. Now ask everyone to vote for the three differences/changes/outcomes that are most relevant to that group – add up the scores to get a relevance rating.
5. Now do the same for significance – this may be similar but there will often be some differences.
6. Finally compare what you think is most relevant and significant for your stakeholder to organisational priorities.

Arguing the toss

Challenging understanding

Difficulty level: Medium

Time: 1-2 hour

1. Draw up a list of the changes or outcomes your organisations delivers for two of your most important stakeholder groups.
2. Arrange for a small number of your team to meet and split into two groups.
3. Ask each group to take one of the stakeholder outcome lists and come up with arguments why each of the outcomes is not significant or relevant to that group.
4. Then ask each group to do the reverse (come up with positive arguments) for the outcome tackled initially by the other group.
5. As a whole group take first one stakeholder group and then the other.  Discuss if your understanding of relevance and significance holds up to the ‘attack’ of the negative thinking.

Talking to your stakeholders

Just asking

Difficulty level: Medium

Time: 2 hours

1. Choose a group of important stakeholders such as your key beneficiary or customer group.
2. Bring them together for a one hour session.
3. Ask them to list or ‘brainstorm’ all the differences that you make to them.
4. Now ask everyone to vote for the three differences/changes/outcomes that are most relevant to them – add up the scores to get a relevance rating.
5. Now do the same for significance – this may be similar but there will often be some differences.
6. Finally compare what you think is most relevant and significant for your stakeholder to what they say.

Who else makes a difference?

Listing other factors

Difficulty level: Easy

Time: 1 hour

1.  Draw together a small group of staff, stakeholders or advisors.
2. Brainstorm a list of all the other people and organisations that affect or contribute to the outcomes you achieve.
3. Agree amongst the group through discussion or simple voting the top five most important contributions to the changes you deliver for your stakeholders from other people and organisations.
4. Note how each of the top five affects your stakeholders.

What would happen if you weren’t there?

Talking to your stakeholders

Difficulty level: Hard

Time: 1 day +

1.  Identify individuals drawn from your key stakeholder groups – the more individuals you identify the more robust this process will be.
2. Contact and arrange to discuss with them face-to-face or over the phone their experience of your service/organisation/project.
3. Ask them to imagine what would have happened if you didn’t exist, or if they had not come into contact with your organisation:

• Would they have experienced similar changes through some other means?
• Would they have simply used an existing alternative service/project?
• Would they have travelled or gone elsewhere?

      4. Once you have established what might have happened if you had             not been there, ask them to estimate how much of the changes              they have experienced as a result of your organisation/project               they would have experienced anyway – this will be very rough.

Where is the evidence?

Triangulating evidence

Difficulty level:Hard

Time: 2-3 days

1.  Explore who else might have contributed to the outcomes you have affected and what would have happened without you as above.
2. Plan for a robust questionnaire or set of interviews with your key stakeholders using closed questions, based on your initial consultations, to assess the extent to which they agree or disagree with the people you have consulted with.
3. Identify sources of local or regional information on changes experienced to your outcomes by people or organisations across a wider area than you operate in.
4. Combine your evidence directly from your stakeholders and your “secondary” data from published sources to conclude how much you have contributed to the difference in the outcomes you have made.

Identifying your sources

Organising and keeping notes

Difficulty level: Moderate

Time: Variable

When you are reporting make sure that you include a full explanation of where your evidence came from.  So this might include:

• Information about the published data source if you have taken it from elsewhere.
• Information about your questionnaire if this has been your approach (for example, how many people completed it, what percentage of the overall number of people who could have completed it and so on).
• If you have used a range of methods to get your evidence and include an explanation of your methodology.

Your goal is that someone reading your account can either replicate your approach for themselves or track down your sources of information without having to come back to you.

Public access to your information

Use the web

Difficulty level: Moderate

Time: Variable

Rather than keeping your reporting to yourself find ways to provide the greatest possible access.  This might include printed information, particularly if you have stakeholders with limited information to the internet.

But in general the most effective way of putting your information in the public domain would be to publish it on your website or someone else’s with clear search terms so that anyone can find it.

Stakeholder review meetings

Feedback to your stakeholders!

Difficulty level: Easy

Time: 2-3 hours

Once you have explored your outcomes, found evidence and reported it – however in-depth or limited this has been, check out your results with your stakeholders.

You could do this simply by circulating a draft of the report and asking for feedback.  Or you could call a meeting and present your results and ask for feedback.  The key will be to share or present your results in a way to encourage challenge and debate rather than simply receive spelling corrections.

Feedback from stakeholders

Feedback to your stakeholders!

Difficulty level: Easy

Time: 2-3 hours

Once you have explored your outcomes, found evidence and reported it – however in-depth or limited this has been, check out your results with your stakeholders.

You could do this simply by circulating a draft of the report and asking for feedback.  Or you could call a meeting and present your results and ask for feedback.  The key will be to share or present your results in a way to encourage challenge and debate rather than simply receive spelling corrections.

Feedback from experts

Ask for feedback!

Difficulty level: Easy

Time: 1 day +

Identify a small panel of experts in your field – you will probably know who they are in any case.

Contact them initially be phone or email to secure their assistance in reviewing your report.

Then send your report to them for comment – but highlight ten key questions rather than simply asking them for overall feedback.  This is much more likely to stimulate a useful response.

Feedback from the SROI Network

Submit your report for assurance

Difficulty level: Hard

Time: 3 months +

This guide is meant for people starting out in SROI, not for those considering submitting a report to the SROI Network to have it assured.

Once you are implementing the principles in full, however you may wish to make use of the Network assurance process – the only way (other than academic peer review) of getting any type of evaluation report externally assured in the UK at present.

You can find out about the assurance process and other aspects of the SROI Network including practitioner training at www.sroinetwork.org