From evidence to insight: designing Spain’s new digital framework for public policy evaluation
Submitted: 2025-05-14
|Accepted: 2025-10-07
|Published: 2025-11-04
Copyright (c) 2025 Milagros Paniagua San Martín, Felipe González De León
This work is licensed under a Creative Commons Attribution 4.0 International License.
Downloads
Keywords:
public policy evaluation, institutional design, digital evidence-based policy, governance, Spain, evaluation agency, public administration
Supporting agencies:
Abstract:
While some developed countries have made steady progress towards the institutionalization of public policy evaluation, Spain's trajectory has been more variable. Law 27/2022 represents a milestone towards this goal, establishing a new State Agency for Public Policy Evaluation (hereinafter, The Agency). This legislation builds upon prior experiences with evaluation and the successful implementation of Spending Reviews by AIReF.
Building on this milestone, the article adopts a normative institutional design approach to elaborate a forward-looking scheme for The Agency. The proposal is informed by comparative experiences yet grounded in Spain’s context. Central to this design is the integration of data analytics and artificial intelligence, which requires a renewed approach to data governance, interministerial coordination, and the articulation between centralized leadership and decentralized operational capacity.
One of the article’s core contributions is the operational definition of the Evaluation Coordination Units (hereinafter, The Units), left unexplained in the law. The Units must be conceived as essential institutional nodes within a federated architecture, enabling systematic, technically robust, and policy-relevant evaluation within ministries. Additionally, the article emphasizes that Spain already holds a valuable foundation of technical infrastructure, administrative data, professional experience, and academic expertise, which should be wisely mobilized.
Success will depend on several interrelated conditions: the consolidation of The Agency’s mandate, the development of specialized and interdisciplinary talent, the creation of a secure and interoperable data infrastructure, the establishment of effective coordination mechanisms, a transparent criteria-based system for prioritizing evaluations, the ability to communicate findings clearly, and the systematic incorporation of results into decision-making.
References:
ALIA Project (2024). First AI language models available in the four official languages of Spain. Gobierno de España. https://datos.gob.es/en/noticia/first-ai-language-models-available-four-official-languages-part-alia-project
AIREF (2019a). Evaluación de Estrategia y Procedimiento de las Subvenciones. https://www.airef.es/wp-content/uploads/2019/06/documentos-sr-protegidos/01_Proyecto_01.pdf
AIREF (2019b). Medicamentos dispensados a través de receta médica. https://www.airef.es/wp-content/uploads/2022/04/ESTUDIOS-FIRMADOS/2019-07-02-RECETAS.pdf
AIREF (2021). La Institucionalización de la evaluación de las políticas públicas en Castilla y León: situación actual y propuestas. https://www.airef.es/es/estudios/estudio-evaluacion-politicas-publicas-castilla-y-leon/
AIReF (2024). Observatory of Findings and Proposals from Evaluations. Retrieved from: https://www.airef.es/es/buscador-hallazgos/
AIReF (2025).Rendición de cuentas: actividad de la AIReF, evaluación externa y Plan de Actuaciones 2025. Comparecencia ante la Comisión de Hacienda y Función Pública del Congreso de los diputados. 12 de mayo de 2025.
Andrés Jovani, J. M. (2023). Un nuevo impulso a la institucionalización de la evaluación de las políticas públicas en España: Problemas persistentes y retos a futuro. Revista de las Cortes Generales, (115), pp. 219–257. https://doi.org/10.33426/rcg/2023/115/1752
Arenilla, M. (2021). Cuerpos y puestos en la función pública española. Diagnóstico, propuestas y líneas rojas. En J. Cantero (coord.), Continuidad versus transformación: ¿Qué función pública necesita España? (pp. 177-217). INAP
Arenilla Sáez, M., Llorente Márquez, J. y Redondo Lebrero, J. C. (2022). Las retribuciones de las Administraciones públicas españolas. Un estudio de su equidad interna. Revista de Estudios de la Administración Local y Autonómica, (18), pp. 137-155. https://doi.org/10.24965/reala.11120
Barber, M. (2008). Instruction to deliver: Fighting to transform Britain's public services. Methuen
Barberá Areste, O., Doria Borrell, E. J., Ntutumu, F. y Sanchis Matoses, P. (2020). La institucionalización de la evaluación de políticas públicas: la Comunitat Valenciana en perspectiva comparada. Universitat de València
Beetsma, R., Debrun, X., Fang, X., Kim, Y., Lledo, V., Mbaye, S., & Zhang, X. (2019). Independent fiscal councils: Recent trends and performance. European Journal of Political Economy, (57), pp. 53- 69. https://doi.org/10.1016/j.ejpoleco.2018.07.004
Bullock, H. L., Lavis, J. N., Wilson, M. G., Mulvale, G., & Miatello, A. (2021). Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis. Implementation Science, (16), Article 18. https://doi.org/10.1186/s13012-021-01082-7
Bundi, P., & Pattyn, V. (2022). Trust, but verify? Understanding citizen attitudes toward evidence-informed policy making. Public Administration,101(4), pp. 1227-1246. https://doi.org/10.1111/padm.12852
Bustelo, M. (2020). Spain. In R. Stockmann, W. Meyer, & L. Taube (Eds.), The Institutionalisation of Evaluation in Europe (pp. 303-328). Palgrave Macmillan. https://doi.org/10.1007/978-3-030-32284-7_12
Capano, G., & Lepori, B. (2024). Designing policies that could work: understanding the interaction between policy design spaces and organizational responses in public sector. Policy Sci, (57), pp. 53–82. https://doi.org/10.1007/s11077-024-09521-0
Cardozo Brum, M. y Rosas Huerta, A. (Eds.). (2021). Avances recientes em la evaluación de políticas y programas públicos. Universidad Autónoma Metropolitana.
Casado, J. M. y Del Pino, E. (2022). Similitudes y diferenciaemen la evaluación de políticas públiems en ocho países: Eemaña en perspectiva comparada. Pap conomiaeconomía española, (172), pp. 2-17. https://dialnet.unirioja.es/servlet/articulo?codigo=8527314
Casado, J. M. C., Fernández Huertas, I. y Gordo Mora, E. (2024). Presupuestos y evaluación de las políticas públicas: dos claves para la mejora de la Administración. Papeles de economía española, (182), pp. 65-78. https://dialnet.unirioja.es/servlet/articulo?codigo=9913021
Cerrillo Martín, I. (2023). Estat de l’avaluació a Catalunya. Situació actual i reptes de futur per la promoció de l’avaluació. Ivàlua. https://ivalua.cat/sites/default/files/inline-files/Estat%20de%20l%E2%80%99avaluacio%CC%81%20Cat_Def.pdf
Closa Montero, C., González De León, F. & Losada Fraga, F., (2020). Democracy vs Technocracy: National Parliaments and Fiscal Agencies in EMU Governance. RECONNECT Working Paper series, Deliverable 10.2. https://reconnect-europe.eu/wp-content/uploads/2020/11/D10.2.pdf
Colin S. Black, Daniel J. Lehane, Chris Burns & Br’an D. O'Donnell. (2018) An examination of the effect of open versus paywalled access publication on the disseminative impact and citation count of publications in intensive care medicine and anesthesia. Journal of Critical Care, (46), pp. 88-93. https://doi.org/10.1016/j.jcrc.2018.05.008
Criado, J.I., Alcaide-Muñoz, L., & Liarte, I. (2025a): Two decades of public sector innovation: building an analytical framework from a systematic literature review of types, strategies, conditions, and results. Public Management Review, 27(3), pp. 623-652. https://doi.org/10.1080/14719037.2023.2254310
Criado, J.I., Sandoval-Almazán, R., & Gil-García, J.R. (2025b): Artificial intelligence and public administration: Understanding actors, governance, and policy from micro, meso, and macro perspectives. Public Policy and Administration, 40(2), pp. 173-184. https://doi.org/10.1177/09520767241272921
Cukierman, A., Web, S. B., & Neyapti, B. (1992). Measuring the independence of central banks and its effect on policy outcomes. The World Bank economic review, 6(3), pp. 353-398. https://doi.org/10.1093/wber/6.3.353
Davoodi H. R., P. Elger, A. Fotiou, D. Garcia-Macia, X. Han, A. Lagerborg, W.R. Lam, & P. Medas. (2022). Fiscal Rules and Fiscal Councils: Recent Trends and Performance during the Pandemic. IMF Working Paper, 2022(11). https://doi.org/10.5089/9798400200472.001
De la Fuente, A., de Rus, G., Fernández, M., García, M. A., Jansen, M., Jiménez, S., Novales, A., Onrubia, J., Pérez Renovales, J., Sastre, E. y Sicilia, J. (2021). La evaluación de políticas públicas en España: antecedentes, situación actual y propuestas para una reforma. FEDEA Policy Paper, 2021(09). https://documentos.fedea.net/pubs/fpp/2021/10/FPP2021-09.pdf
De la Fuente, A. (2022). Algunos comentarios sobre el proyecto de ley de institucionalización de la evaluación de políticas públicas. Apuntes FEDEA, 2022(17). https://documentos.fedea.net/pubs/ap/2022/ap2022-17.pdf
Díaz-Chao, Á., Torrent-Sellens, J., Ballestar, M. T., & Camina, E. (2021). Productivity and employment effects of digital complementarities. Journal of Innovation & Knowledge, 6(3), pp. 177–190. https://doi.org/10.1016/j.jik.2020.10.006
Eisenstein, J. (2019). Introduction to Natural Language Processing. The MIT Press.
EU Independent Fiscal Institutions (2016). Defining and enforcing minimum standards for Independent Fiscal Institutions. https://www.euifis.eu/publications/9
European Commission. (2021). Regulation (EU) 2021/241 of the European Parliament and of the Council establishing the Recovery and Resilience Facility. Official Journal of the European Union.
European Parliamentary Research Service (EPRS). (2023). Evaluation in the European Commission: Rolling check-list and state of play (5th ed.). https://doi.org/10.2861/451691
Feinstein, O., & Goñi, E. Z. (2010). Evaluation of government performance and public policies in Spain. ECD Working Paper Series, 2010(22). https://www.politicipublice.ro/uploads/spain.pdf
Fernández-Albertos, J. (2015). The politics of central bank independence. Annual Review of Political Science, (18), pp. 217-237. https://doi.org/10.1146/annurev-polisci-071112-221121
Floridi, L. (2023). The Ethics of Artificial Intelligence. Oxford University Press. https://doi.org/10.1093/oso/9780198883098.001.0001
Garde, J. A. (2023). La experiencia AEVAL en España (2005-2017). Gestión y Análisis de Políticas Públicas, (32), pp. 96-115. https://doi.org/10.24965/gapp.11000
Géron, A. (2022). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (3rd’ed.). O'Reilly.
Giest, S., McBride, K., Nikiforova, A., & Sikder, S. K. (2025). Digital & data-driven transformations in governance: A landscape review. Data & Policy, (7), Article e21. https://doi.org/10.1017/dap.2024.47
Gutiérrez-Fandiño, A., Armengol-Estapé, J., Gonzalez-Agirre, A., & Villegas, M. (2021). Spanish Legalese Language Model and Corpora. arXiv https://doi.org/10.48550/arXiv.2110.12201
INAP (2024). Proyecto LIP 7: Capacidades para la evaluación de políticas públicas. INAP
Jacob, S. (2023). The institutionalization of evaluation around the globe: understanding the main drivers and effects over the past decades. In F. Varone, S. Jacob & P. Bundi (Eds.), Handbook of Public Policy Evaluation (pp. 187-205). Edward Elgar.https://doi.org/10.4337/9781800884892.00021
Jankovics, L & Larch, M. (2024) Independence safeguards: how do national EU IFIs fare?. Paper presented at the European Fiscal Board’s 6th annual conference, 17 May, Brussels.
Kupiec, T., Celińska-Janowicz, D., & Pattyn, V. (2023). Understanding evaluation use from em organisational perspective: A review of the literature and a research agenda. Evaluation, 29(3), pp. 338-355. https://doi.org/10.1177/13563890231185164
Law 27/2022, of December 20, on the institutionalization of public policy evaluation in the General State Administration. Boletín Oficial del Estado, 305.
Li, Z., Zhou, H., Xu, Z., & Ma, Q. (2025). Machine learning and public health policy evaluation: research dynamics and prospects for challenges. Frontiers in Public Health, (13). https://doi.org/10.3389/fpubh.2025.1502599
Lourenço, L., Weber, L., Garcia, L.P., Ramos, V., & Souza, J. (2024). Machine Learning Algorithms to Estimate Propensity Scores in Health Policy Evaluation: A Scoping Review. International Journal of Environmental Research and Public Health, 21(11), Article 1484. https://doi.org/10.3390/ijerph21111484
Madan, R., & Ashok, M. (2023). AI adoption and diffusion in public administration: A systematic literature review and future research agenda. Government Information Quarterly, 40(1), Article 101774. https://doi.org/10.1016/j.giq.2022.101774
McKinsey & Company. (2025). The State of AI: How organizations are rewiring to capture value.
Mertens, D. M. (2009). Transformative research and evaluation. Guilford Press.
Ministry of the Presidency, Justice and Relations with the Parliament. (2024). Informe Anual de Evaluación Normativa 2023.
Moral-Arce, I. (2024, April 17). Evaluación de políticas públicas: la formación del Instituto de Estudios Fiscales. Blog Fiscal de Crónica Tributaria.
OECD. (2014). Recommendation of the Council on Principles for Independent Fiscal Institutions. OECD/LEGAL/0401.
OECD (2015). Estonia and Finland: Fostering Strategic Capacity across Governments and Digital Services across Borders. OECD Public Governance Reviews http://dx.doi.org/10.1787/9789264229334-en
OECD (2016). Spain 2016: Linking Reform to Results for the Country and its Regions. OECD Public Governance Reviews. http://dx.doi.org/10.1787/9789264263024-en
OECD. (2017). Skills for a High Performing Civil Service. OECD Public Governance Reviews. http://dx.doi.org/10.1787/9789264280724-en
OECD (2020a), Building Capacity for Evidence-Informed Policy-Making: Lessons from Country Experiences. OECD Public Governance Reviews. https://doi.org/10.1787/86331250-en
OECD (2020b), Improving Governance with Policy Evaluation: Lessons From Country Experiences. OECD Public Governance Reviews. https://doi.org/10.1787/89b1577d-en
OECD (2020c). Strengthening the Governance of Skills Systems: Lessons from Six OECD Countries. OECD Skills Studies. https://doi.org/10.1787/3a4bb6ea-en
OECD. (2020d). Policy Framework on Sound Public Governance: Baseline Features of Governments that Work Well. OECD Publishing. https://doi.org/10.1787/c03e01b3-en
OECD (2021). Independent Fiscal Institutions Database. Version 2.0, Paris.
OECD (20ª2a). Better Regulation Practices across the European Union 2022. OECD Publishing. https://doi.org/10.1787/6e4b095d-en
OECD. (2022b) Recommendation of the Council on Public Policy Evaluation, OECD/LEGAL/0478
OECD. (2022c). OECD Journal on Budgeting, Volume 2022 Issue 3. OECD Publishing. https://doi.org/10.1787/dedebeca-en
OECD (2023). Improving decision making through policy evaluation in Belgium. OECD Public Governance Policy Papers, No. 31. https://doi.org/10.1787/08f7aef5-en
OECD. (2024). Assessing potential future artificial intelligence risks, benefits and policy imperatives. OECD Artificial Intelligence Papers, No. 27. https://doi.org/10.1787/3f4e3dfb-en
OECD. (2025). Implementation Toolkit for the OECD Recommendation on Public Policy Evaluation. OECD Publishing. https://doi.org/10.1787/77faa4fe-en
Paniagua, M. (2025). The Use of Tax and Social Security Administrative Data for the Design and Evaluation of Public Policies: A Case Study for Spain – The Establishment of the National Minimum Income Scheme. CIAT Working Document No. 02-2025. Inter-American Center of Tax Administrations (CIAT). https://biblioteca.ciat.org/opac/book/5898
Peeters, R. & Widlak, A. (2023). Administrative exclusion in the infrastructure-level bureaucracy: The case of the Dutch daycare benefit scandal. Public Administration Review, 83(4), pp. 863-877. https://doi.org/10.1111/puar.13615
Pérez, D. & Jiménez, J. (2022). Espacio de Datos Federados. Revista de Economía Industrial, 2022(1), pp. 103-116. https://dialnet.unirioja.es/servlet/articulo?codigo=8750552
PwC. (2024). AI Jobs Barometer. https://www.pwc.es/es/consultoria/assets/pwc-ai-jobs-barometer-informe.pdf
Ramió, C. (2017). El eslabón perdido de la Administración pública española: la ausencia de una dirección pública profesional. Revista de Evaluación de Programas y Políticas Públicas, (8), pp. 1-14. http://dx.doi.org/10.5944/reppp.8.2017.16980
Ravallion, M. (2020). Should the Randomistas (Continue to) Rule? NBER Working Paper No. 27554. National Bureau of Economic Research. https://doi.org/10.3386/w27554
Rehill, P., & Biddle, N. (2024). Heterogeneous treatment effect estimation with high-dimensional data in public policy evaluatio–-- an application to the conditioning of cash transfers in Morocco using causal machine learning. arXiv. https://arxiv.org/abs/2401.07075
Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), pp. 386–408. https://doi.org/10.1037/h0042519
Russell, S., & Norvig, P. (2022). Artificial Intelligence: A Modern Approach (4th ed.). Pearson.
Sudha Rani, K., Suma Latha, A., Satish, V., Medisetty, P., & Yashwanth Sena, V. (2024). Forecasting Global Recession and It’s Likely Fallout on India’s Economy Using Machine Learning Techniques. 2024 International Conference on Computational Intelligence for Green and Sustainable Technologies (ICCIGST), pp. 1-5. https://doi.org/10.1109/ICCIGST60741.2024.10717620
Trueblood, J. S., Allison, D. B., Field, S. M., Fishbach, A., Gaillard, S. D., Gigerenzer, G., ... & Teodorescu, A. R.. (2025). The misalignment of incentives in academic publishing and its impact on research quality. Proceedings of the National Academy of Sciences, 122(5), Article e2401231121. https://doi.org/10.1073/pnas.2401231121
Tümer, A.E., & Kabaklarlı, E. (2024). Estimation of Climate Change Parameters for Agricultural Economy Efficiency with Machine Learning Methods. International Journal of Life Sciences and Biotechnology, 7(3), pp. 189-197. https://doi.org/10.38001/ijlsb.1473586
Turing, A. M. (1950). Computing Machinery and Intelligence. Mind, 59(236), pp. 433–460. https://doi.org/10.1093/mind/LIX.236.433
Van der Voort, H. G., Klievink, A. J., Arnaboldi, M., and Meijer, A. J. (2019). Rationality and politics of algorithms. Will the promise of big data survive the dynamics of public decision making? Government Information Quarterly, 36(1), pp. 27-38. https://doi.org/10.1016/j.giq.2018.10.011
Varela Merino, B. (2023). Dirección y evaluación de políticas públicas en base a la evidencia: ¿dónde se encuentra la Administración General del Estado?. Gestión y Análisis de Políticas Públicas, (32), pp. 28-44. https://doi.org/10.24965/gapp.11013
Videgaray, L. (Chair), Aghion, P., Caputo, B., Forrest, T., Korinek, A., Langenbucher, K., Miyamoto, H., & Wooldridge, M. (2024). Artificial Intelligence and Economic and Financial Policymaking: A High-Level Panel of Experts' Report to the G7. G7 Finance Track.
Von Trapp, L and Nicol, S. (2018). ‘Measuring IFI independence: a first Pass using the OECD IFI Database’, In R Beetsma and X Debrun (eds.), Independent Fiscal Councils: Watchdogs or Lapdogs? (pp. 47-64). CEPR Press.
World Bank. (2020). Artificial Intelligence in the Public Sector: Maximizing Opportunities, Managing Risks. The World Bank.
Zhao, L., Jin, Y., Zhou, L., Yang, P., Qian, Y., Huang, X., & Min, M. (2023). Evaluation of health system resilience in 60 countries based on their responses to COVID-19. Frontiers in Public Health, 2022(10), Article 1081068. https://doi.org/10.3389/fpubh.2022.1081068
Zúñiga-Guevara, R. M. (2022). Innovación colaborativa y construcción de capacidades como estrategia de institucionalización de la evaluación en Andalucía. El Grupo de Personas Colaboradoras en Evaluación de Políticas Públicas. Gestión y Análisis de Políticas Públicas, (30), pp. 88-111. https://doi.org/10.24965/gapp.10887