Systems Thinking

Systems Evaluation | Systems Mapping | Systems Leadership

This page is designed to support professional development work around the engagement with systems approaches to intervention development AND the evaluation of interventions that have adopted systems approaches to change.

Critical Questions

  • What is a systems approach to intervention design?

  • What is a system?

  • What is system mapping and what is it's purpose?

  • Discuss the benefits of systems mapping in terms of both process and outcome.

  • Discuss the difference between evaluating a system and evaluating a systems driven intervention. (Mowles, 2014)

  • Once a system has been "mapped", how do we know where to intervene? (e.g. Nobles et al, 2021; Meadows 12 places; Intervention Level Framework; Iceberg Model)

Related Content

Also on this website: Logic Models in Physical Activity Health Promotion

Articles

Alford, C. (2017) How systems mapping can help you build a better theory of change. https://blog.kumu.io/how-systems-mapping-can-help-you-build-a-better-theory-of-change-4c85ae4301a8

Madden, H. and Ohlson, I. (2 June 2020) Systems Mapping - a brief overview of what, why noand how (Part 1) - https://mojdigital.blog.gov.uk/2020/06/02/systems-mapping-a-brief-overview-of-what-why-and-how-part-1/

Acaroglu, L. (2017) Tools for Systems Thinkers: The 6 Fundamental Concepts of Systems Thinking - https://medium.com/disruptive-design/tools-for-systems-thinkers-the-6-fundamental-concepts-of-systems-thinking-379cdac3dc6a

Todres, G./NPC (2021) What we’ve learnt from systems mapping https://www.thinknpc.org/blog/what-weve-learnt-from-systems-mapping/

Systems Thinking & Public Health

Bolton, K. A., Whelan, J., Fraser, P., Bell, C., Allender, S., & Brown, A. D. (2022). The Public Health 12 framework: interpreting the ‘Meadows 12 places to act in a system’ for use in public health. Archives of Public Health, 80(1), 1-8. https://doi.org/10.1186/s13690-022-00835-0

Diez Roux, A. V. (2011). Complex systems thinking and current impasses in health disparities research. American journal of public health, 101(9), 1627-1634.

Midgley, G. (2006). Systemic intervention for public health. American journal of public health, 96(3), 466-472.

Luna Pinzon, A., Stronks, K., Dijkstra, C. et al. The ENCOMPASS framework: a practical guide for the evaluation of public health programmes in complex adaptive systems. Int J Behav Nutr Phys Act 19, 33 (2022). https://doi.org/10.1186/s12966-022-01267-3

Published Papers: Systems Thinking & Mapping

Barbrook-Johnson, P., & Penn, A. (2021). Participatory systems mapping for complex energy policy evaluation. Evaluation, 27(1), 57-79. https://doi.org/10.1177%2F1356389020976153

Elsawah, S., Pierce, S. A., Hamilton, S. H., Van Delden, H., Haase, D., Elmahdi, A., & Jakeman, A. J. (2017). An overview of the system dynamics process for integrated modelling of socio-ecological systems: Lessons on good modelling practice from five case studies. Environmental Modelling & Software, 93, 127-145. https://doi.org/10.1016/j.envsoft.2017.03.001

Foster-Fishman, P. G., Nowell, B., & Yang, H. (2007). Putting the system back into systems change: A framework for understanding and changing organizational and community systems. American journal of community psychology, 39(3-4), 197-215. https://doi.org/10.1007/s10464-007-9109-0

Gates, E. F. (2016). Making sense of the emerging conversation in evaluation about systems thinking and complexity science. Evaluation and Program Planning, 59, 62-73. https://doi.org/10.1016/j.evalprogplan.2016.08.004

Green, L.W., Sim, L., Breiner, H. (2013) (Eds.) Ch 9: Systems and Evaluation: Placing a Systems Approach in Context. In Evaluating Obesity Prevention Efforts: A Plan for Measuring Progress. Committee on Evaluating Progress of Obesity Prevention Effort; Food and Nutrition Board; Institute of Medicine. https://www.ncbi.nlm.nih.gov/books/NBK202498/

Grove, J. T. (2015). Aiming for Utility in ‘Systems‐based Evaluation’: A Research‐based Framework for Practitioners. IDS Bulletin, 46(1), 58-70. https://doi.org/10.1111/1759-5436.12121

Hamilton, S. H., Fu, B., Guillaume, J. H., Badham, J., Elsawah, S., Gober, P., ... & Zare, F. (2019). A framework for characterising and evaluating the effectiveness of environmental modelling. Environmental modelling & software, 118, 83-98. https://doi.org/10.1016/j.envsoft.2019.04.008

Hawe, P., Shiell, A., & Riley, T. (2009). Theorising interventions as events in systems. American journal of community psychology, 43(3-4), 267-276. https://doi.org/10.1007/s10464-009-9229-9

Kelly, R. A., Jakeman, A. J., Barreteau, O., Borsuk, M. E., ElSawah, S., Hamilton, S. H., ... & Voinov, A. A. (2013). Selecting among five common modelling approaches for integrated environmental assessment and management. Environmental modelling & software, 47, 159-181. https://doi.org/10.1016/j.envsoft.2013.05.005

McGill, E., Er, V., Penney, T., Egan, M., White, M., Meier, P., ... & Petticrew, M. (2021). Evaluation of public health interventions from a complex systems perspective: a research methods review. Social Science & Medicine, 113697. https://doi.org/10.1016/j.socscimed.2021.113697

Nobles, J. D., Radley, D., Mytton, O. T., & Whole Systems Obesity programme team. (2021). The Action Scales Model: A conceptual tool to identify key points for action within complex adaptive systems. Perspectives in Public Health, 17579139211006747. https://doi.org/10.1177%2F17579139211006747

Nobles J, Wheeler J, Dunleavy-Harris K, et al. (2021) Ripple Effects Mapping: Capturing the Wider Impacts of Systems Change Efforts in Public Health. Research Square. https://doi.org/10.21203/rs.3.rs-1008063/v1

Table from: Nobles, J. D., Radley, D., Mytton, O. T., & Whole Systems Obesity programme team. (2021). The Action Scales Model: A conceptual tool to identify key points for action within complex adaptive systems. Perspectives in Public Health, 17579139211006747. https://doi.org/10.1177%2F17579139211006747

Mowles, C. (2014). Complex, but not quite complex enough: The turn to the complexity sciences in evaluation scholarship. Evaluation, 20(2), 160-175.

Peters, D. H. (2014). The application of systems thinking in health: why use systems thinking?. Health research policy and systems, 12(1), 1-6. https://doi.org/10.1186/1478-4505-12-51

Pluchinotta, I., Esposito, D., & Camarda, D. (2019). Fuzzy cognitive mapping to support multi-agent decisions in development of urban policymaking. Sustainable Cities and Society, 46, 101402. https://doi.org/10.1016/j.scs.2018.12.030

Renger, R., Atkinson, L., Renger, J., Renger, J., & Hart, G. (2019). The connection between logic models and systems thinking concepts. Evaluation Journal of Australasia, 19(2), 79-87. https://doi.org/10.1177%2F1035719X19853660

Rosas, S., & Knight, E. (2019). Evaluating a complex health promotion intervention: case application of three systems methods. Critical Public Health, 29(3), 337-352. https://doi.org/10.1080/09581596.2018.1455966

Rutter, H., Savona, N., Glonti, K., Bibby, J., Cummins, S., Finegood, D. T., ... & White, M. (2017). The need for a complex systems model of evidence for public health. The lancet, 390(10112), 2602-2604. https://doi.org/10.1016/S0140-6736(17)31267-9

Stave, K., & Hopper, M. (2007, July). What constitutes systems thinking? A proposed taxonomy. In 25th international conference of the system dynamics Society. https://proceedings.systemdynamics.org/2007/proceed/papers/STAVE210.pdf

Stave, K. (2010). Participatory system dynamics modeling for sustainable environmental management: Observations from four cases. Sustainability, 2(9), 2762-2784. https://doi.org/10.3390/su2092762

Image from: Stansfield, J., Cavill, N., Marshall, L., Robson, C., & Rutter, H. (2021). Using complex systems mapping to build a strategic public health response to mental health in England. Journal of Public Mental Health. Vol. 20 No. 4, pp. 286-297. https://doi.org/10.1108/JPMH-10-2020-0140

Stansfield, J., Cavill, N., Marshall, L., Robson, C., & Rutter, H. (2021). Using complex systems mapping to build a strategic public health response to mental health in England. Journal of Public Mental Health. Vol. 20 No. 4, pp. 286-297. https://doi.org/10.1108/JPMH-10-2020-0140

Tourais, P., & Videira, N. (2021). A participatory systems mapping approach for sustainability transitions: Insights from an experience in the tourism sector in Portugal. Environmental Innovation and Societal Transitions, 38, 153-168. https://doi.org/10.1016/j.eist.2021.01.002

Voinov, A., Jenni, K., Gray, S., Kolagani, N., Glynn, P. D., Bommel, P., ... & Smajgl, A. (2018). Tools and methods in participatory modeling: Selecting the right tool for the job. Environmental Modelling & Software, 109, 232-255. https://doi.org/10.1016/j.envsoft.2018.08.028

Wilkinson, H., Hills, D., Penn, A., & Barbrook-Johnson, P. (2021). Building a system-based theory of change using participatory systems mapping. Evaluation, 27(1), 80-101. https://doi.org/10.1177%2F1356389020980493

Systems Thinking: Physical Activity & Public Health

Barnes, A., Black, M., Baxter, S., Furber, A., Strong, M., Beynon, C., ... & Clowes, M. (2019). Understanding public health systems: a participatory systematic review and systems infographic. The Lancet, 394, S22. https://doi.org/10.1016/S0140-6736(19)32819-3

Bird, E.L., Evans, D., Gray, S. et al. (2022) Opportunities to engage health system leaders in whole systems approaches to physical activity in England. BMC Public Health 22, 254. https://doi.org/10.1186/s12889-022-12602-5

Bellew, W., Smith, B. J., Nau, T., Lee, K., Reece, L., & Bauman, A. (2020). Whole of Systems approaches to physical activity policy and practice in Australia: the ASAPa Project overview and initial systems map. Journal of Physical Activity and Health, 17(1), 68-73. https://doi.org/10.1123/jpah.2019-0121

CLOA/SportEngland (2021) Navigating Local Systems Test and Learn approach to system change https://cloa.org.uk/wp-content/uploads/2021/03/Final-Navigating-Local-Systems.pdf

Rutter, H., Cavill, N., Bauman, A., & Bull, F. (2019). Systems approaches to global and national physical activity plans. Bulletin of the World Health Organization, 97(2), 162. https://dx.doi.org/10.2471%2FBLT.18.220533

Pogrmilovic, B. K., O’Sullivan, G., Milton, K., Biddle, S. J., Bauman, A., Bellew, W., ... & Pedisic, Z. (2019). The development of the Comprehensive Analysis of Policy on Physical Activity (CAPPA) framework. International Journal of Behavioral Nutrition and Physical Activity, 16(1), 1-15. https://doi.org/10.1186/s12966-019-0822-5

Rigby, B.P., Dodd-Reynolds, C.J. & Oliver, E.J. (2022). The understanding, application and influence of complexity in national physical activity policy-making. Health Res Policy Sys 20, 59 https://doi.org/10.1186/s12961-022-00864-9

Image from: Rutter, H., Cavill, N., Bauman, A., & Bull, F. (2019). Systems approaches to global and national physical activity plans. Bulletin of the World Health Organization, 97(2), 162. https://dx.doi.org/10.2471%2FBLT.18.220533
Image from: Bellew, W., Smith, B. J., Nau, T., Lee, K., Reece, L., & Bauman, A. (2020). Whole of Systems approaches to physical activity policy and practice in Australia: the ASAPa Project overview and initial systems map. Journal of Physical Activity and Health, 17(1), 68-73. https://doi.org/10.1123/jpah.2019-0121
om: Bellew, W., Smith, B. J., Nau, T., Lee, K., Reece, L., & Bauman, A. (2020). Whole of Systems approaches to physical activity policy and practice in Australia: the ASAPa Project overview and initial systems map. Journal of Physical Activity and Health, 17(1), 68-73. https://doi.org/10.1123/jpah.2019-0121

Systems Thinking: Food and Obesity

Allender, S., Owen, B., Kuhlberg, J., Lowe, J., Nagorcka-Smith, P., Whelan, J., & Bell, C. (2015). A community based systems diagram of obesity causes. PloS one, 10(7), e0129683. https://doi.org/10.1371/journal.pone.0129683

Maitland, N., Wardle, K., Whelan, J., Jalaludin, B., Creighton, D., Johnstone, M., ... & Allender, S. (2021). Tracking implementation within a community-led whole of system approach to address childhood overweight and obesity in south west Sydney, Australia. BMC public health, 21(1), 1-11. https://doi.org/10.1186/s12889-021-11288-5

Swinburn, B. A., Kraak, V. I., Allender, S., Atkins, V. J., Baker, P. I., Bogard, J. R., ... & Dietz, W. H. (2019). The global syndemic of obesity, undernutrition, and climate change: the Lancet Commission report. The lancet, 393(10173), 791-846. https://doi.org/10.1016/S0140-6736(18)32822-8

Waterlander, W. E., Luna Pinzon, A., Verhoeff, A., Den Hertog, K., Altenburg, T., Dijkstra, C., ... & Stronks, K. (2020). A system dynamics and participatory action research approach to promote healthy living and a healthy weight among 10–14-year-old adolescents in Amsterdam: The LIKE programme. International journal of environmental research and public health, 17(14), 4928. https://doi.org/10.3390/ijerph17144928

Related Reading

The Butterfly Effect: Everything You Need to Know About This Powerful Mental Model - https://fs.blog/the-butterfly-effect/

Resources

OMIDYAR Group (2017) Systems Practice Workbook https://oecd-opsi.org/toolkits/systems-practice-workbook/

Penn, AS, Barbrook-Johnson, P (2019) Participatory Systems Mapping: A practical guide. CECAN toolkit. Available at: https://www.cecan.ac.uk/resources/toolkits/

CECAN (2021) The Complexity Evaluation Toolkit. July 2021. Version 1.0. Online at www.cecan.ac.uk
Preskill, H., Gopal, S. Mack, K. and Cook, J. (nd) Evaluating Complexity: Propositions for Improving Practice https://www.fsg.org/resource/evaluating-complexity/
Systems Thinking Toolkit: Putting Systems Thinking into Practice in Your Organization https://www.fsg.org/resource/systems-thinking-toolkit-0/

CECAN (2021) The Complexity Evaluation Toolkit. July 2021. Version 1.0. Online at www.cecan.ac.uk

Egan M, McGill E, Penney T, Anderson de Cuevas R, Er V, Orton L, White M, Lock K, Cummins S, Savona N, Whitehead M, Popay J, Smith R, Meier P, De Vocht F, Marks D, Andreeva M, Rutter H, Petticrew M. (2019) NIHR SPHR Guidance on Systems Approaches to Local Public Health Evaluation Part 1: Introducing systems thinking https://sphr.nihr.ac.uk/wp-content/uploads/2018/08/NIHR-SPHR-SYSTEM-GUIDANCE-PART-1-FINAL_SBnavy.pdf

Egan M, McGill E, Penney T, Anderson de Cuevas R, Er V, Orton L, White M, Lock K, Cummins S, Savona N, Whitehead M, Popay J, Smith R, Meier P, De Vocht F, Marks D, Andreeva M, Rutter H, Petticrew M. (2019) NIHR SPHR Guidance on Systems Approaches to Local Public Health Evaluation. Part 2: What to consider when planning a systems evaluation. London: National Institute for Health Research School for Public Health Research https://sphr.nihr.ac.uk/wp-content/uploads/2018/08/NIHR-SPHR-SYSTEM-GUIDANCE-PART-2-v2-FINALSBnavy.pdf

Kania, J., Kramer, M. and Senge, P (2018) The Water of Systems Change https://www.fsg.org/resource/water_of_systems_change/

NPC (2018) Thinking big: How to use theory of change for systems change https://www.thinknpc.org/resource-hub/thinking-big-how-to-use-theory-of-change-for-systems-change/

Systems Thinking Toolkit: Putting Systems Thinking into Practice in Your Organization https://www.fsg.org/resource/systems-thinking-toolkit-0/

Preskill, H., Gopal, S. Mack, K. and Cook, J. (nd) Evaluating Complexity: Propositions for Improving Practice https://www.fsg.org/resource/evaluating-complexity/

Tamarack Institute. TOOL: The Systems Change Evaluation Canvas https://www.tamarackcommunity.ca/library/systems-change-evaluation-canvas

TASO Run your own theory of change workshop https://taso.org.uk/evidence/resources-hub/toc/

Software & Mapping Tools

PRSM - https://cress.soc.surrey.ac.uk/prsm/ [see Barbrook-Johnson, P., & Penn, A. (2021)]

https://www.diagrams.net/ - diagrams.net: easy to use and edit but doesn't allow automated analysis.

Kumu - https://kumu.io/

Miro - https://miro.com/

Vensim - https://vensim.com/

STICKE from Deakin Uni - https://sticke2.deakin.edu.au/#about

SimpleMind - https://simplemind.eu/

yED (https://www.yworks.com/products/yed)

NetDraw (Borgatti, 2002)

Gephi (Bastian et al., 2009)

Free Online Learning Resources

Open University. Systems thinking. https://www.open.edu/openlearn/science-maths-technology/engineering-technology/systems-thinking-free-courses

John Hopkins University (Coursera). Systems thinking in Public Health. https://www.coursera.org/learn/systems-thinking

Systems Thinking on Twitter

Brian Castellani - @complexcase

Image From: https://medium.com/disruptive-design/tools-for-systems-thinkers-the-6-fundamental-concepts-of-systems-thinking-379cdac3dc6a

Systems Thinking: Books & Chapters

Anderson, V. and Johnson, L. (2015) Systems Thinking Basics: From Concepts to Causal Loops (Pegasus Workbook Series). Pegasus Communications.

Boulton, J. G., Allen, P. M., & Bowman, C. (2015). Embracing complexity: Strategic perspectives for an age of turbulence. OUP Oxford.

Byrne, D., & Callaghan, G. (2013). Complexity theory and the social sciences: The state of the art. Routledge.

Castellani, B., Rajaram, R., Buckwalter, J. G., Ball, M., & Hafferty, F. (2015). Place and health as complex systems: A case study and empirical test. Springer.

Finegood, D., Johnston, L., Steinberg, M., Matteson, C. L., & Deck, P. (2014). Complexity, systems thinking, and health behavior change. In Health Behavior Change in Populations; Kahan, S., Gielen, AC, Fagan, PJ, Lawrence, WG, Eds, 208-236.

Holland, J. H. (2000). Emergence: From chaos to order. OUP Oxford.

Ison, R. (2010). Systems practice: how to act in a climate change world. Springer. [sample chapter]

Meadows, D. H. (2008). Thinking in systems: A primer. Chelsea Green Publishing.

Stroh, D. P. (2015). Systems thinking for social change: A practical guide to solving complex problems, avoiding unintended consequences, and achieving lasting results. Chelsea Green Publishing.

West, G. (2017). Scale: The Universal Laws of Life, Growth, and Death in Organisms. Cities, and Companies. Penguin Publishing Group.

Related Resources

Plastrik, P. and Taylor, M. (2006) NET GAINS: A Handbook for Network Builders Seeking Social Change https://networkimpact.org/downloads/NetGainsHandbookVersion1.pdf

Brian Castellani and Lasse Gerrit's Map of The Complexity Sciences

https://sacswebsite.blogspot.com/2021/09/q-for-2021-version-of-map-of-complexity.html