Challenges of impact assessment in Social Innovation: A qualitative study from two European rural regions

Desafíos de la evaluación de impacto en la Innovación Social: Un estudio cualitativo de dos regiones rurales europeas

  • Marina Novikova ISCTE - Instituto Universitário de Lisboa
Keywords: social innovation, impact assessment, rural regions, local development initiatives, Austria, Portugal


Over recent years, various approaches to assessing the impacts of social innovation (SI) have developed without a uniform method having arisen. There are some issues around how impacts can be assessed, connected with the questions on the nature of impacts, the levels of analysis and effects of a purely positivist approach to impact assessment. While attempting to assess such impacts, various SI initiatives face the diversity of challenges. To this end, the aim of the article is to investigate the experience of said initiatives promoting social innovation related to impact assessment of social innovation. The paper is based on an empirical study conducted with the local development associations and local action groups involved in social innovation projects in two rural regions of Austria and Portugal. The results indicate that, despite recognising the importance of impact assessment regarding social innovation activities and the opportunities it provides, local organisations in question face many challenges in assessing the impacts of social innovation, including conceptual and practical difficulties.



En los últimos años se han desarrollado diversos enfoques para evaluar los impactos de la innovación social (IS) sin que haya surgido un método uniforme. Existen algunas cuestiones en torno a la forma de evaluar los impactos, relacionadas con las preguntas sobre la naturaleza de los impactos, los niveles de análisis y los efectos de un enfoque puramente positivista de la evaluación de impactos. Al intentar evaluar dichos impactos, varias iniciativas de IS se enfrentan a la diversidad de desafíos. Para ello, el objetivo del artículo es investigar la experiencia de dichas iniciativas que promueven la innovación social en relación con la evaluación del impacto de la innovación social. El artículo se basa en un estudio empírico realizado con asociaciones de desarrollo local y grupos de acción local que participan en proyectos de innovación social en dos regiones rurales de Austria y Portugal. Los resultados indican que, a pesar de reconocer la importancia de la evaluación de impacto en relación con las actividades de innovación social y las oportunidades que ofrece, las organizaciones locales en cuestión se enfrentan a muchos retos a la hora de evaluar los impactos de la innovación social, incluyendo dificultades conceptuales y prácticas.

Palabras clave: innovación social; evaluación de impacto; regiones rurales; iniciativas de desarrollo local; Austria; Portugal.




Antadze, N., & Westley, F. R. (2012). Impact metrics for social innovation: Barriers or bridges to radical change? Journal of Social Entrepreneurship, 3(2), 133-150.

Arena, M., Azzone, G., & Bengo, I. (2015). Performance measurement for social enterprises. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 26(2), 649-672.

Arya, A., & Mittendorf, B. (2015). Supply chain consequences of subsidies for corporate social responsibility. Production and Operations Management, 24(8), 1346-1357.

Avelino, F., Wittmayer, J. M., Pel, B., Weaver, P., Dumitru, A., Haxeltine, A., ... & O'Riordan, T. (2019). Transformative social innovation and (dis) empowerment. Technological Forecasting and Social Change, 145, 195-206.

Avelino, F. (2021). Theories of power and social change. Power contestations and their implications for research on social change and innovation. Journal of Political Power, 1-24.

Baturina, D., & Bežovan, G. (2015). Social innovation impact-review. Seventh Framework Programme, Third Sector Impact. Brussels.

BEPA. (2010). Empowering People, Driving Change. Social Innovation in the European Union. Bureau of European Policy Advisers and European Commission.

Bock, B. B. (2016). Rural marginalisation and the role of social innovation; a turn towards nexogenous development and rural reconnection. Sociologia Ruralis, 56(4), 552-573.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101.

Bund, E., Gerhard, U., Hoelscher, M., & Mildenberger, G. (2015). A methodological framework for measuring social innovation. Historical Social Research/Historische Sozialforschung, 48-78.

Burdge, R. J., & Vanclay, F. (1996). Social impact assessment: a contribution to the state of the art series. Impact Assessment, 14(1), 59-86.

Chatzichristos, G., & Nagopoulos, N. (2020). Regional Institutional Arenas for Social Innovation: mixed methods research. Journal of Social Entrepreneurship, 1-23.

Clark, C., Rosenzweig, W., Long, D., & Olsen, S. (2004). Double Bottom Line Project Report: Assessing Social Impact in Double Line Ventures. New York. Available online at: (accessed 21 November).

Cunha, J., & Benneworth, P. (2020). How to measure the impact of social innovation initiatives? International Review on Public and Nonprofit Marketing, 17(1), 59-75.

Di Iacovo, F., Moruzzo, R., Rossignoli, C., & Scarpellini, P. (2014). Transition management and social innovation in rural areas: Lessons from social farming. The Journal of Agricultural Education and Extension, 20(3), 327-347.

Ebrahim, A., & Rangan, V. K. (2010). The limits of nonprofit impact: A contingency framework for measuring social performance (No. 10-099). Harvard Business School.

Ebrahim, A., & Rangan, V. K. (2014). What impact? A framework for measuring the scale and scope of social performance. California management review, 56(3), 118-141.

Epstein, M.J., & Yuthas, K. (2014). Measuring and improving social impacts: A guide for nonprofits, companies, and impact investors. Berrett-Koehler Publishers. Available online: (accessed 23 April 2021).

European Commission. (2014). Proposed approaches to social impact measurement in European Commission legislation and in practice relating to EuSEFs and the EaSI: GECES sub-group on impact measurement. European Commission Publications Office.

Eurostat. (2019). Available online: (accessed 12 December 2019).

Fougère, M., & Meriläinen, E. (2021). Exposing three dark sides of social innovation through critical perspectives on resilience. Industry and Innovation, 28(1), 1-18.

Goldstein, J., Hazy, J. K., & Silberstang, J. (2010). A complexity science model of social innovation in social enterprise. Journal of social entrepreneurship, 1(1), 101-125.

Grieco, C. (2015). Assessing social impact of social enterprises: Does one size really fit all? Springer.

Howaldt, J., Domanski, D., & Kaletka, C. (2016). Social innovation: Towards a new innovation paradigm. RAM. Revista de Administração Mackenzie, 17(6), 20-44.

Howaldt, J., Kaletka, C., Schröder, A., & Zimgiebl, M. (Eds.). (2018). Atlas of social innovation: new practices for a better future. Technische Universität Dortmund, ZWE Sozialforschungsstelle.

Köhler, J., Geels, F. W., Kern, F., Markard, J., Onsongo, E., Wieczorek, A., ... & Wells, P. (2019). An agenda for sustainability transitions research: State of the art and future directions. Environmental innovation and societal transitions, 31, 1-32.

Kolodinsky, R. W., Madden, T. M., Zisk, D. S., & Henkel, E. T. (2010). Attitudes about corporate social responsibility: Business student predictors. Journal of Business Ethics, 91(2), 167-181.

Krlev, G., Bund, E., & Mildenberger, G. (2014). Measuring what matters—Indicators of social innovativeness on the national level. Information Systems Management, 31(3), 200-224.

Liket, K. C., & Maas, K. (2015). Nonprofit organizational effectiveness: Analysis of best practices. Nonprofit and Voluntary Sector Quarterly, 44(2), 268-296.

Liket, K. C., Rey-Garcia, M., & Maas, K. E. (2014). Why aren’t evaluations working and what to do about it: A framework for negotiating meaningful evaluation in nonprofits. American Journal of Evaluation, 35(2), 171-188.

Ludvig, A., Weiss, G., Sarkki, S., Nijnik, M., & Živojinović, I. (2018). Mapping European and forest related policies supporting social innovation for rural settings. Forest Policy and Economics, 97, 146-152.

Maas, K., & Grieco, C. (2017). Distinguishing game changers from boastful charlatans: Which social enterprises measure their impact? Journal of Social Entrepreneurship, 8(1), 110-128.

Maas, K., & Liket, K. (2011). Social impact measurement: Classification of methods. In R. Burritt, S. Schaltegger, M. Bennett, T. Pohjola & M. Csutora (Eds.), Environmental management accounting and supply chain management (pp. 171-202). Springer.

MacCallum, D. (2009). Social innovation and territorial development. Ashgate Publishing.

Moulaert, F., Martinelli, F., Swyngedouw, E., & Gonzalez, S. (2005). Towards alternative model (s) of local innovation. Urban studies, 42(11), 1969-1990.

Moulaert, F. (2013). The international handbook on social innovation: collective action, social learning and transdisciplinary research. Edward Elgar Publishing.

Mulgan, G. (2010). Measuring social value. Stanford Social Innovation Review, 8(3), 38-43.

Neumeier, S. (2012). Why do social innovations in rural development matter and should they be considered more seriously in rural development research? –Proposal for a stronger focus on social innovations in rural development research. Sociologia Ruralis, 52(1), 48-69.

Neumeier, S. (2017). Social innovation in rural development: identifying the key factors of success. The geographical journal, 183(1), 34-46.

Nicholls, A., Simon, J., & Gabriel, M. (2015). Introduction: Dimensions of social innovation. In A. Nicholls, J. Simon & M. Gabriel (Eds.), New frontiers in social innovation research (pp. 1-26). Palgrave Macmillan.

Niccholls, A. (2015). Synthetic grid: a critical framework to inform the development of social innovation metrics. CRESSI Working papers, 14. Oxford, Creating Economic Space for Social Innovation (CRESSI) project.

Nicholls, A. (2018). A general theory of social impact accounting: Materiality, uncertainty and empowerment. Journal of Social Entrepreneurship, 9(2), 132-153.

Nicholls, A., & Dees, J.G. (2015). Social Innovation. In J. D. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences (pp. 355-361). Elsevier.

Nijnik, M., Secco, L., Miller, D., & Melnykovych, M. (2019). Can social innovation make a difference to forest-dependent communities? Forest Policy and Economics, 100, 207-213.

Novikova, M., de Fátima Ferreiro, M., & Stryjakiewicz, T. (2020). Local Development Initiatives as Promoters of Social Innovation: Evidence from Two European Rural Regions. Quaestiones Geographicae, 39(2), 43-53.

Novikova, M. (2021). Transformative Social Innovation in Rural Areas: Insights from a Rural Development Initiative in the Portuguese Region of Baixo Alentejo. European Countryside, 13(1), 71-90.

Noy, C. (2008). Sampling knowledge: The hermeneutics of snowball sampling in qualitative research. International Journal of social research methodology, 11(4), 327-344.

Nyseth, T., & Hamdouch, A. (2019). The transformative power of social innovation in urban planning and local development. Urban Planning, 4(1), 1-6.

Organisation for Economic Cooperation and Development. (2006). The new rural paradigm: Policies and governance. OECD Publishing.

Phills, J. A., Deiglmeier, K., & Miller, D. T. (2008). Rediscovering social innovation. Stanford Social Innovation Review, 6(4), 34-43.

Preskill, H., & Beer, T. (2012). Evaluating social innovation. Available online: (assessed 11 June 2021).

Ravazzoli, E., Dalla Torre, C., Da Re, R., Marini Govigli, V., Secco, L., Górriz-Mifsud, E., ... & Nijnik, M. (2021). Can social innovation make a change in European and Mediterranean marginalized areas? Social innovation impact assessment in agriculture, fisheries, forestry, and rural development. Sustainability, 13(4), 1823.

Schuitmaker, T. J. (2012). Identifying and unravelling persistent problems. Technological Forecasting and Social Change, 79(6), 1021-1031.

Tracey, P., & Stott, N. (2017). Social innovation: a window on alternative ways of organizing and innovating. Innovation, 19(1), 51-60.

Vanclay, F., Baines, J. T., & Taylor, C. N. (2013). Principles for ethical research involving humans: ethical professional practice in impact assessment Part I. Impact assessment and project appraisal, 31(4), 243-253.

Vanclay, F. (2003). International principles for social impact assessment. Impact assessment and project appraisal, 21(1), 5-12.

Vanclay, F. (2020). Reflections on Social Impact Assessment in the 21st century. Impact Assessment and Project Appraisal, 38(2), 126-131.

Van der Have, R.P. and Rubalcaba, L., 2016. Social innovation research: An emerging area of innovation studies? Research Policy, 45(9), 1923-1935.

Westley, F., & Antadze, N. (2010). Making a difference: Strategies for scaling social innovation for greater impact. Innovation Journal, 15(2), 1-19.

Wittmayer, J. M., Backhaus, J., Avelino, F., Pel, B., Strasser, T., Kunze, I., & Zuijderwijk, L. (2019). Narratives of change: How social innovation initiatives construct societal transformation. Futures, 112, 102433.

Ziegler, R. (2017). Social innovation as a collaborative concept. Innovation: The European Journal of Social Science Research, 30(4), 388-405.

How to Cite
Novikova, M. (2021). Challenges of impact assessment in Social Innovation: A qualitative study from two European rural regions. European Public & Social Innovation Review, 6(2), 35-50. Retrieved from
Research articles