How to measure social impact

I have a keen interest in the not-for-profit sector because its commitment to mission aligns with my personal values and goals. As part of my Executive MBA studies, I completed the Social Impact course that provided me with an insight into complex problems in society and how I can leverage my skills to help tackle them.

One of my biggest learnings from this course relates to demonstrating social impact.

I previously viewed social impact measurement as a distraction from doing actual work imposed by funders and regulators. It was easier for me to focus on outputs rather than long-term outcomes and impact, because they were readily available and straightforward to report on. This course broadened my perspective and helped me distinguish between outputs and lasting social, economic and environmental effects.

Applying course concepts, particularly Theory of Change and Logic Models helped me see the benefits of social impact measurement like learning and personal development, increased accountability, transparency and trust and overall organisational improvement. 

In this blog I’ll share some of the tools that can be used to analyse a particular problem. We will use homelessness as an example and compare how two organisations tackle this problem and demonstrate social impact.

Homelessness in Australia is a politically complex problem. This blog introduces the analysis and potential solutions to this problem from the not-for-profit and hybrid sector perspectives. It compares two organisations from these sectors that are united by the underlying objective of reducing homelessness. Mission Australia and The Big Issue diverge in their funding models, scaling approaches and the way they implement their initiatives, and yet both achieve measurable outcomes and contribute to long term changes in conditions that cause homelessness.

According to the latest census figures, 122,487 people were experiencing homelessness in Australia in 2021 with its rate increasing in many states (ABS 2021). Homelessness is a complex problem (Flatau et al. 2021), it spans multiple interconnected domains: political, social and economic with no simple solutions. Housing supply, tax system and luxury investment (Dawson et al 2022) further exacerbate the problem. Multiple stakeholder groups exist with conflicting values contributing to sluggish wage growth (Stewart et al 2022) and intergenerational inequality (Dawson et al 2022). Although some degree of uncertainty exists (e.g. it is difficult to forecast economic trends, consequences of policy interventions and unforeseen events like pandemics), the nature of the problem and potential solutions are known (e.g. the experience of Nordic countries as per Grimstad et al 2021). This is, therefore, a politically complex problem (Head 2022).

Systems thinking approaches (Meadows and Wright 2009) can be applied to better understand the problem and critically analyse its context and complexity. A problem tree (Figure 1) is used to analyse potential causes and impacts of homelessness from the not-for-profit and hybrid sectors perspectives.

This problem tree serves as a foundation to uncover dynamics and causal relationships in this complex system. For example, the figure below illustrates a self-reinforcing feedback loop that creates a vicious cycle of a profit-price dynamic that drives the increased cost of living (Stanford, 2023) which contributes to housing affordability as one of the causes of homelessness (Muir et al 2018).

Causal loop diagrams can help visualise the problem and facilitate a discussion about potential drivers of these dynamics. For instance, the figure below illustrates an interconnected nature of balancing feedback loops with delayed effects on the rate of homelessness.

The not-for-profit sector organisations like Mission Australia can leverage their resources (e.g. donations) to deliver services in line with their mission of ending homelessness (Mission Australia 2024).

The hybrid sector can have an impact on this problem through innovative business models. The Big Issue is a social enterprise that is ‘creating work opportunities for people experiencing homelessness, marginalisation and disadvantage’ (Big Issue 2024).

Goals and objectives

Mission Australia has a goal to “end homelessness and ensure people and communities in need can thrive” (Mission Australia 2024). The Big Issue aims to ‘enhance financial inclusion and empowerment of individuals to break the cycle of poverty and reduce homelessness” (The Big Issue 2024). The Big Issue objective is broader and is tackling the issue of homelessness by “creating work opportunities for people who are unable to access mainstream work” (The Big Issue 2024), while Mission Australia’s objective has a more explicit focus on ending homelessness.

Simplified Theory of Change models (Connell et al 1998) can be used to better link these goals and objectives to intended outcomes.

The above figure demonstrates alignment between The Big Issue’s goals and activities it undertakes. Mission Australia’s Theory of Change (below) similarly helps visualise the pathway for achieving the stated goal.

These models serve as overviews of both organisations and begin to illuminate specific steps they take to achieve social impact.

Implementation

The analysis of Mission Australia and The Big Issue social impact initiatives suggest that they are sustainable from the funding and scaling perspective.

Both organisations are registered as charities, however, there are significant differences in their operating and funding models. Mission Australia is funded through conventional non-profit means such as donations and grants. It has a range of supporters and reported $348.8m in total operating revenue with 91% going directly to service delivery in 2023 (Mission Australia 2023). Its sustainability is dependent on continuous funding.

The Big Issue also obtains funding from individual donations and corporate and philanthropic partners like Westpac and Australia Post. However, its funding sources are diversified by including the profit from magazine sales that are then reinvested into the organisation to support its mission (The Big Issue 2024). The Big Issue’s sustainable business model is based on its pursuit of a unified strategy for revenue and social value generation (Battilana et al 2021).

From the scaling perspective, both organisations engage in T-shaped scaling (Nardini et al 2022) as outlined in the analysis below.

	Mission Australia	The Big Issue
Deep scaling	Evidence of establishing relationships and embedding in local communities e.g. focusing only on a handful of communities.	Evidence of customised programs for local community e.g. Community Street Soccer Program.
Wide scaling	Evidence of sharing local ideas and lessons learned across communities e.g. 463 programs delivered across Australia in partnership with local community groups and churches.	Evidence of developing an integrated network of vendors in all Australian states and territories, increasing capacity and widening social impact. E.g. 7,570 vendors since inception with $38.3m in their pockets.

These organisations differ, however, in the way they achieve scale at the organisational level. The Big Issue is a smaller organisation and scales primarily through dissemination and affiliation due to lower cost requirements (Kickul and Lyons 2020). It operates a loosely coupled network of locally managed affiliates across several programs like The Big Issue Classroom and Community Street Soccer Program. Mission Australia, on the other hand, scales through branching, allowing it to exercise greater control over local operations (Kickul and Lyons 2020). Being a larger organisation, it balances higher costs of maintaining local facilities and hiring and retaining staff against the benefits of a more direct oversight.

Both approaches have their pros and cons. Scaling through dissemination is effective for smaller organisations like The Big Issue because it allows for wider reach with relatively low funding. However, the operational control in this case is limited. Mission Australia, on the other hand, is a larger charity and therefore has different requirements. Their branching approach has also been effective, despite higher costs, as it allowed for a more consistent service delivery across geographies.

Implementation challenges primarily relate to the fact that every community is different, so it’s essential to tailor services to the local needs. It’s not possible to just lift and shift practices from one place to another and expect them to have the same outcome or impact. For this reason, Mission Australia, explicitly favours “depth in a handful of communities over breadth to multiply their impact” (Mission Australia’s 20-25 Strategy). To be able to tailor their services, they decided to only focus on five selected communities in the current strategic horizon. To better adapt to local needs, Mission Australia is co-designing local initiatives with communities (Mission Australia 2024). The Big Issue should pilot their ideas in local communities first by deeply understanding their needs and then expand geographically across Australia, widening their scale.

Results

This section builds on the analysis of the context of the problem of homelessness and Theory of Change models presented in earlier sections. The research of The Big Issue and Mission Australia’s impact reports suggests that both organisations measure outcomes and long-term social impact, moving beyond a common focus on inputs and outputs (Muir and Bennett 2014). This is evidenced in their respective logic models below.

Social impact is quantified through reduction in homelessness and can lead to significant societal transformation. For both organisations, it’s difficult to differentiate between long-term outcomes and wider impact, which is inferred in both cases as it’s challenging to effectively assign attribution. For the below logic mode, impact is measured through the reduction in demand for community services, another proxy for significant societal transformation.

Contrasting both organisations’ approaches to demonstrating social impact, Mission Australia’s capability appears to be more sophisticated due to its measurement culture (Parker 2023). This is supported by their strategic objective to ‘develop local and long-term measures in each community we select’ (Mission Australia’s 20-25 Strategy) and more detailed qualitative and quantitative data and insight in their report compared to The Big Issue. Mission Australia also have a more explicit focus on outcomes and impact and a corresponding set of high-quality indicators (Neuman 2011) to demonstrate that desired change occurred.

Interestingly, The Big Issue suggests that it returned $210m in social value since 1996 (The Big Issue 2023) demonstrating the Social Return on Investment. This approach has benefits as it can translate complex qualitative impact into an easily digestible dollar value. However, it can also oversimplify the measurement (Zappalá and Lyons, 2009). Mission Australia is not using this method.

When it comes to communicating impact, both organisations do this effectively in their impact reports, providing a succinct overview in simple terms as well as an executive summary which is followed by data analysis and interpretation (Grobb 2013).

Conclusion

While comparing Mission Australia and The Big Issue it’s important to remember that ‘homelessness is everyone’s responsibility’ (Muir et al 2018). Charities and hybrid organisations have their place in a holistic approach to system change to tackle the multifaceted problem of homelessness in Australia.

Both organisations effectively apply leadership and management principles and achieve sustainability from funding and scaling perspectives. They go about this in different ways and both, to some extent, struggle to differentiate between long-term outcomes and impact in their measurement approaches. Their activities, however, contribute to significant societal transformation in line with their objectives of reducing homelessness.

References

1. A Safe Place to Call Home- Mission Australia Homelessness and Stable Housing Impact Report 2023, https://www.missionaustralia.com.au/media-centre/media-releases/mission-australia-s-homelessness-impact-report-reveals-rising-demand-for-services
2. Australian Bureau of Statistics 2021, Estimating Homelessness: Census, ABS, https://www.abs.gov.au/statistics/people/housing/estimating-homelessness-census/latest-release
3. Battilana, J., Lee, M., Walker, J. & Dorsey, C., 2012, ‘In Search of the Hybrid Ideal’, Stanford Social Innovation Review. 10 (3), 51–55
4. Big Issue Group Impact Report 2022 https://www.bigissue.com/group-impact-report-2022/
5. Big Issue Group website 2024 https://www.bigissue.com/about-the-big-issue-group/
6. Bromfield, N. (2023) ‘Critical policy studies’, in Australian Politics and Policy: Senior Edition 2023. Sydney University Press. 
7. COMM5701 Social Impact Moodle eBook AGSM 2024
8. Connell, J.P. and Kubisch, A.C. (1998) ‘Applying a theory of change approach to the evaluation of comprehensive community initiatives: progress, prospects, and problems’, The Aspen Institute [Preprint].
9. Dawson, E., Llyod-Cape, M., D’Rosario, M., Falzon, J., Tonkin, L., 2022. Housing Affordability in Australia
10. Flatau, P., Lester, L., Seivwright, A., Teal, R., Dobrovic, J., Vallesi, S., Hartley, C. and Callis, Z. (2021). Ending homelessness in Australia: An evidence and policy deep dive. Perth: Centre for Social Impact, The University of Western Australia and the University of New South Wales. DOI: 10.25916/ntba-f006
11. Franks, D.M., Vanclay, F., 2013. Social Impact Management Plans: Innovation in corporate and public policy. Environmental Impact Assessment Review 43, 40–48. https://doi.org/10.1016/j.eiar.2013.05.004
12. Grimstad, S., Holst, H. and Scott, A., 2022. Homes for people: How Nordic policies can improve Australia’s housing affordability.
13. Grob, G.F., 2015. Writing for impact, in: Handbook of Practical Program Evaluation. Wiley Online Library, pp.739–764.
14. Head, B.W. 2022, Wicked problems in public policy: understanding and responding to complex challenges. Cham: Springer Nature: https://doi.org/10.1007/978-3-030-94580-0.
15. Kickul, J. and Lyons, T.S., 2020. Understanding social entrepreneurship: The relentless pursuit of mission in an ever changing world. Routledge.
16. Kirkwood, 1998, CW System Behavior and causal loop diagrams’ in System dynamics methods: A quick introduction, accessed from <https://www.public.asu.edu/~kirkwood/sysdyn/SDIntro/ch-1.pdf&gt;.
17. Meadows, D. and Wright, D., 2009, Thinking in Systems: A Primer. Chelsea Green Publishing.
18. Mission Australia 2023 Strategy update https://www.missionaustralia.com.au/about-us/2023-strategy-update
19. Mission Australia Annual Report 2023 https://www.missionaustralia.com.au/annual-report-2023
20. Mission Australia’s 20-25 Strategy https://www.missionaustralia.com.au/our-strategy
21. Muir, K., Bennett, S., 2014. The Compass: Your guide to social impact measurement. The Centre for Social Impact.
22. Muir, K., Martin, C., Liu, E., Kaleveld, L., Flatau, P., Etuk, L., and Pawson, H. 2018. Amplify Insights: Housing Affordability & Homelessness. Centre for Social Impact, UNSW Sydney.
23. Nardini, G., Bublitz, M.G., Butler, C., Croom-Raley, S., Edson Escalas, J., Hansen, J. and Peracchio, L.A., 2022. Scaling social impact: Marketing to grow nonprofit solutions. Journal of Public Policy & Marketing, 41(3), pp.254-276.
24. Parker, R., 2023. Embracing a Measurement Culture in Australia’s For-Purpose Sector. Power to Persuade. URL https://www.powertopersuade.org.au/blog/embracing-a-measurement-culture-in- australias-for-purpose-sector/6/4/2023
25. Ramia, I., Powell, A., Stratton, K., Stokes, C., Meltzer, A., Muir, K., 2021. Roadmap to outcomes measurement. Your step-by-step guide to planning, measuring and communicating social impact (Paper or Report). Centre for Social Impact, Sydney.
26. Rittel, H., & Webber, M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155–169.
27. SECO 2018 ‘A Primer on Results-Based Management’: https://www.seco-cooperation.admin.ch/dam/secocoop/de/dokumente/resultate/monitoring/quali-primer.pdf.download.pdf/Primer%20on%20Results-Based%20Management.pdf
28. Stanford, J., 2023. Profit-Price Spiral: The Truth Behind Australia’s Inflation [WWW Document]. The Australia Institute’s Centre for Future Work. URL https://futurework.org.au/?post_type=tai_cpt_report&p=1331
29. Stewart, A., Stanford, J., Hardy, T., 2022. The Wages Crisis Revisited [WWW Document]. The Australia Institute’s Centre for Future Work. https://futurework.org.au/report/the-wages-crisis-revisited/
30. Stories for Impact. (n.d.). Problem Tree. https://storiesforimpact.com/toolbox/problem-tree/&nbsp;
31. Tackling a wicked problem. Per Capita. https://percapita.org.au/our_work/housing-affordability-in-australia-tackling-a-wicked-problem/
32. Taylor, G (2014). Viability: A priority criterion for the mitigation of climate change and other complex socio-ecological issues. Journal of Futures Studies, 19(1), pp.77-96
33. The Big Issue Social Impact Report 2023 https://thebigissue.org.au/assets/uploads/2023/11/TBI.SocialImpactReport2023.pdf
34. The Big Issue website 2024: https://www.bigissue.com/about-the-big-issue-group/
35. Walker, A., Wilcox, T., Powell, A., Muir, K., 2017. The Navigator: Your guide to leadership for social purpose. The Centre for Social Impact, Sydney, Australia.
36. Weber, O., 2016. Impact Investing, in: Routledge Handbook of Social and Sustainable Finance. Routledge.
37. Zappalà, G., Lyons, M., 2009. Recent approaches to measuring social impact in the Third sector: An overview.