The EU Recovery and Resilience Facility falls short against performance-based funding standards

The mere completion of a project does not guarantee a positive social impact. As Kristalina Georgieva, European Commission vice-president when the ‘EU Budget Focused on Results’ initiative was launched in 2015, put it, “we can build a road with 0 percent error rate but if it goes nowhere, it is still a road to nowhere, and it is a 100 percent waste of our taxpayers’ money”. She argued for measuring the impact of spending instead in terms of how much it boosts household income, raises standards and creates jobs (European Commission, 2017).

Public funding that is contingent on measurable results, rather than on financial resources spent, is commonly known as “performance-based funding” (OECD, 2007). The European Union Regulation 1 Regulation (EU) 2021/241 available at https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32021R0241. that established the bloc’s post-COVID-19 economic recovery fund, the Recovery and Resilience Facility (RRF), appeared to embrace this approach, by stating that the RRF is “performance-based” and “funding is based on the achievement of results”.

Surprisingly, however, the specific definitions in the RRF Regulation do not require the achievement of results in the commonly understood sense. The European Commission guidelines for the preparation of recovery plans by EU national authorities – in order to access RRF funding – appear to explicitly discourage result indicators. Perhaps for this reason, there are major differences in the use of result indicators in national recovery plans (NRPs). Some NRPs include many results indicators but others include hardly any. For those that support the logic of performance-based funding (as we do), this is a problem.

Performance-based funding

The European Court of Auditors (ECA, 2021) distinguishes between four classes of funding indicator:

• Input: The financial, human, material, administrative or regulatory means used to implement a project or programme.
• Output: Something produced or achieved by a project, such as the delivery of a training course or the construction of a road.
• Result: The immediate effect of a project or programme on its completion, such as the improved employability of course participants or improved mobility following the construction of a new road.
• Impact: The wider long-term consequences of a completed project or programme, such as socio-economic benefits for the population as a whole.

When assessing performance-based financing in EU cohesion policy, ECA (2021) defined performance budgeting as “the systemic use of information about outputs, results and/or impacts to inform, influence and/or determine the allocation of public funds.”

Other institutions define performance-based financing (or performance-based budgeting) similarly, including World Bank economists (Moynihan and Beazley, 2016; Gurazada et al, 2022) and USAID 2 “Performance-based financing (PBF), often referred to as pay-for-performance (P4P) or results-based financing (RBF), refers to payment to a government, organization, or individual conditioned on taking measurable actions toward or achieving desired goals.” See https://www.fpfinancingroadmap.org/learning/specific-topics/performance…. .

Performance-based budgeting in the sense defined by ECA can be rationalised as a solution to two types of moral hazard problems:

• Disbursing funds conditional on project outputs, as opposed to project costs, addresses moral hazard related to project implementation. If funding for a construction project merely takes the form of reimbursement for the cost of construction materials, it would create incentives to waste (or steal) these materials. In contrast, conditioning funding on outputs (eg the completion of a tunnel) encourages the completion of the project in a cost-minimising way. Conditioning funding on inputs expected ex ante (eg physical units) rather than costs ex post would also be consistent with cost minimisation, while enabling the partial disbursement of funds before the project is completed 3 Alternatively, indicators of key implementation steps (defined by ECA, 2021, as “intermediate stage, complementing milestones, in progress towards the achievement of output targets”) could be used to facilitate interim payments. .  
• The use of results (and possibly impact) indicators addresses moral hazard related to project selection. An entity applying for funding normally has private information about the social usefulness of the project – for example, whether a road project is likely to help the local economy. Results (and possibly impact) indicators address this problem; they rule out ‘roads to nowhere’. As Moynihan and Beazley (2016) put it, “All performance budgeting efforts have a common goal – to focus the mindset and behavior of public officials on policy priorities and results.”

A possible objection to the use impact indicators is that they create excessive risk for the executing authority. Even a well-selected and well-executed project may have an underwhelming impact due to bad luck. For example, some transport infrastructure projects completed in the mid-to-late 2000s underperformed after demand fell short of expectations because of the global financial crisis. However, this argument does not apply to results indicators as defined by ECA, because these are related to the immediate usability of an output, rather than its broader social and economic impact.

Increased focus on performance and results in the EU budget

Though EU financing of projects under the Multiannual Financial Framework (MFF) – the EU’s seven-year budget – remained predominantly cost-based, several instruments were introduced to enhance performance (see ECA, 2019, for a summary). Steps towards greater performance focus included the EU budget review by the European Commission (2010) 4 “Spending on the right policies is only worthwhile if it secures the desired results. Spending programmes must have a real impact, with the investment feeding through into action – action which is measured in terms of real impact, rather than in terms of the inputs involved.” , the launch of the ‘EU Budget Focused on Results’ initiative in 2015 (European Commission, 2017), and the 2018 revision of the financial regulation of the EU budget, which required the ex-ante definition of programme objectives and the monitoring of progress with performance indicators 5 “The concept of performance as regards the budget should be clarified. Performance should be linked to the direct application of the principle of sound financial management. The principle of sound financial management should also be defined, and a link should be established between objectives set and performance indicators, results and economy, efficiency and effectiveness in the use of appropriations.” . More recently, the European Parliament’s Committee on Budgetary Control has called for promotion of “a profound culture of ‘result orientation’ aimed at optimising the use of funds” (European Parliament, 2023).

The Recovery and Resilience Facility is not results-oriented

Notwithstanding common definitions of performance-based funding and the increased performance focus in the EU budget, the RRF is in fact not a results-focused instrument. While the preamble of the RRF Regulation hints at performance-based funding principles, its articles and guidance on implementation do not require reporting on actual results.

The preamble of the RRF (Regulation (EU) 2021/241) refers to “the performance-based nature of the Facility” and states that “For reasons of efficiency and simplification in the financial management of the Facility, Union financial support for recovery and resilience plans should take the form of financing based on the achievement of results measured by reference to milestones and targets indicated in the approved recovery and resilience plans.”

However, Article (2) of the regulation defines ‘milestones and targets’ as “measures of progress towards the achievement of a reform or an investment”. The expression “measures of progress towards” thus indicates a process, not necessarily the achievement of results in the sense of the ECA and OECD definitions.

Moreover, as also noted by European Court of Auditors (ECA, 2022), European Commission guidance on the preparation of national recovery and resilience plans called for input and output indicators, but appears to discourage both impact and results indicators. According to the Commission guidance (European Commission, 2021):

“Milestones and targets should be clear and realistic, and the proposed indicators relevant, acceptable and robust. They can reflect different stages of the implementation of reforms and investments, either based on input indicators (e.g. resources provided, which can be financial, human, administrative) or preferably output indicators (e.g. number of workers trained, numbers of renovated schools).

Overall, it is important that milestones and targets remain within the control of the Member State and are not conditional on external factors such as the macroeconomic outlook or the evolution of the labour market. Impact indicators (e.g. decrease in the number of unfilled vacancies in the IT sector) should be avoided given the unpredictability of such indicators and their dependence on other factors outside the control of the Member State.”

While the point that milestones and targets should not be subject to excessive risks outside the control of EU governments is well taken, this guidance raises two issues. First, it fails to acknowledge the distinction between (wider) impact indicators and (immediate) results indicators, and hence it does not encourage EU countries to look for the latter. Second, the example of an ‘impact indicator’ used in the guidance actually constitutes a results indicator according to the ECA definition. If a project involves training coders, and the project is well chosen (for example, based on vacancies for coders), then a drop in the number of unfilled vacancies for coders is entirely appropriate as a results indicator. In the taxonomy proposed above, it would address the second type of moral hazard (namely, related to project selection), while exposing the EU country to minimal risk outside its control (see here for further examples that show that designing results indicators meeting this condition is feasible).

A potential justification for the omission of results indicators could be that, given the involvement of the Commission in the development of national recovery plans, moral hazard related to project selection could have been eliminated. In that case, input and output indicators would be sufficient. However, while the direct involvement of the Commission should certainly reduce the risk of inappropriate project selection, it probably does not eliminate it. The Commission had to negotiate 27 plans, which were often very long and complex, and the time available for negotiations was relatively short. It is unlikely that the Commission had the capacity to evaluate each project in each plan thoroughly 6 One indication of this is that the Commission’s involvement has not always been able to prevent unambitious projects. Corti and de la Ossa (2023) showed that the Italian recovery plan target of assisting 300,000 jobseekers by December 2022 was just half of the national target of 600,000 jobseekers, and reported similar anomalies for a programme on early childhood education. Darvas and Welslau (2023) identified cases in which the deadline for reaching targets coincided with, or even preceded, the submission date of the plans, implying the completion of targets without any additional efforts after the approval of the plan. . Relying at least partly on results indicators could incentivise countries to focus on the social value of projects.

In addition, focusing the RRF purely on process (input and output indicators) is at odds with the EU’s vocal championing of results-based approaches. By making explicit the focus on added social value, the use of result indicators could contribute to meeting public expectations about a performance-based instrument. The use of results indicators and monitoring would also address criticism of the inability of national authorities to invest EU money in desirable projects, and the lack of capacity of EU institutions for identifying inappropriate projects. By demonstrating that EU funding is indeed expected to yield positive results, such scepticism could be dispelled.

What kind of performance indicators are included in the national recovery and resilience plans?

In Darvas and Welslau (2023), we analysed the targets set out by the five largest EU countries and Romania, and put them into three categories: inputs, outputs and results (according to ECA definitions). Impact indicators were excluded, and were indeed absent from national recovery plans since such impacts often become observable only several years after projects are completed. Here, we extend the analysis to include Finland (Table 1).

France, Germany and the Netherlands adopted very few results indicators, either in absolute numbers or as a share of total indicators. Input indicators account for more than half of all indicators for Germany and almost half for France, and more than a third for the Netherlands and Spain.

The numbers of results indicators and their shares in total indicators are both much larger in the Finnish, Italian and Romanian plans than in the other four plans. Spain is somewhere halfway between the groups of France/Germany/Netherlands and Finland/Italy/Romania.

Thus, the French, German and Dutch plans essentially correspond to the process-focused definition of milestones and targets in the RRF Regulation, while the Finnish, Italian and Romanian plans have meaningful result orientations.

The low numbers and shares of result indicators in the French and German plans cannot be explained by less access to the RRF (Table 2). France (€37 billion in grants) and Germany (€28 billion in grants) have obtained more RRF funding than Finland (€1.8 billion) and Romania (€12 billion in grants and €15 billion in loans), yet Finland and Romania included many more results indicators and fewer input indicators than France and Germany (both in terms of the number of targets and the share of all targets).

A further disparity relates to the number of target indicators per RRF funding amount: despite receiving less funding from the RRF than France and Germany, Romania has more target indicators (254) than those two countries (105 and 75, respectively). This discrepancy is even more pronounced when comparing Finland and France, with France receiving twenty times the amount of funding, yet having less than twice as many target indicators as Finland.

Recommendations

Any future instrument using a ‘performance-based’ approach should define clearly what is meant by performance. We suggest that this term be reserved for instruments that focus on achieving results, in line with the EU jargon and the general use of this expression by other institutions. Since achieving results may take time and involve some risk of not meeting goals after project completion, input and output indicators should still be included in a performance-based framework. Results indicators, however, should have a prominent role.

The significant differences in the numbers of target indicators per RRF funding amount call for efforts to harmonise indicators and evaluate the ambition of the national plans of EU countries. There are already common output and result indicators for regional policy linked to policy objectives (see Annex 1 of Regulation (EU) 2021/1058 on the European Regional Development Fund and on the Cohesion Fund ⁷ Available at https://eur-lex.europa.eu/eli/reg/2021/1058. ), which can help in designing results indicators in a harmonised way across countries. However, complete harmonisation is unlikely to be achieved, and the difficulty of achieving targets varies. For example, a single target selected by one country may prove to be more challenging to reach than ten targets chosen by another. Therefore, it is crucial to exercise sound judgment when evaluating or amending the targets proposed by EU governments. Efforts to compare the levels of ambition and expected achievements of different countries, however, remain crucial.

References

Corti, F. and T. Ruiz de la Ossa (2023) ‘The Recovery and Resilience Facility: What are we really monitoring with a performance-based approach?, CEPS Explainer, https://www.ceps.eu/ceps-publications/the-recovery-and-resilience-facility-2/

Darvas, Z. and L. Welslau (2023) First lessons from the Recovery and Resilience Facility for the EU economic governance framework, In-depth analysis requested by the European Parliament's Committee on Economic and Monetary Affairs, available at https://www.bruegel.org/report/first-lessons-recovery-and-resilience-facility-eu-economic-governance-framework

ECA (2019) ’Delivering performance in Cohesion’, Briefing Paper 08/2019, European Court of Auditors, available at https://www.eca.europa.eu/en/Pages/DocItem.aspx?did=50385 

ECA (2021) ‘Performance-based financing in Cohesion policy: worthy ambitions, but obstacles remained in the 2014-2020 period’, Special Report 24/2021, European Court of Auditors, available at https://www.eca.europa.eu/en/Pages/DocItem.aspx?did=%7bF0370F47-0A28-47A3-816D-6E908AC98FA1%7d

ECA (2022) ‘The Commission’s assessment of national recovery and resilience plans – Overall appropriate but implementation risks remain’, Special Report 21/2022, European Court of Auditors, available at https://www.eca.europa.eu/en/Pages/DocItem.aspx?did=61946

European Commission (2010) ‘The EU Budget Review’, COM(2010) 700, available at https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0700:FIN:EN:PDF

European Commission (2017) EU Budget Focused on Results, Summary of the 1st annual conference held on 22 September 2015, De Gasperi Hall, European Commission, Brussels, Publications Office of the European Union, available at https://data.europa.eu/doi/10.2761/900943

European Commission (2018) ‘Regulation (EU, Euratom) 2018/1046 on the financial rules applicable to the general budget of the Union, OJ L 193/1, available at https://commission.europa.eu/publications/eu-financial-regulation_en

European Commission (2021) ‘Commission staff working document guidance to member states recovery and resilience plans – Part 1’, SWD(2021) 12 final, available at https://data.consilium.europa.eu/doc/document/ST-5538-2021-INIT/en/pdf

European Parliament (2023) ‘Report on the Council position on the draft general budget of the European Union for the financial year 2023’, A9-0241/2022, available at https://www.europarl.europa.eu/doceo/document/A-9-2022-0241_EN.html

Gurazada, S., M. Piatti and T. Poulsen (2022) ‘Performance Based Financing: Inspiring new approaches to Public Financial Management in Health and Education’, World Bank Blogs, 5 May, available at https://blogs.worldbank.org/governance/performance-based-financing-inspiring-new-approaches-public-financial-management-health

Moynihan, D. and I. Beazley (2016) Toward next-generation performance budgeting: Lessons from the experiences of seven reforming countries, World Bank Publications, available at https://openknowledge.worldbank.org/entities/publication/85474368-2eef-5873-bda7-03666bc94763

OECD (2007) Performance Budgeting in OECD Countries, Organisation for Economic Co-operation and Development, available at https://www.oecd.org/gov/budgeting/performancebudgetinginoecdcountries.htm