New CAP Delivery Model, Old Issues

The proposed Common Agricultural Policy (CAP) for the period 2021-2027 will be more flexible and, presumably, more effective. To provide for sufficient ambition and prevent a race to the bottom, national strategic plans will be introduced with quantitative targets covering both policy pillars. This article argues that since formal requirements and the evaluation model are weak on actual long-term impact, substantial improvements are unlikely. To test this, programming rules are experimentally evaluated on the implementation of CAP 2014-2020 in Slovenia. The experiment shows that while measures and resources broadly correspond to policy objectives, the specific relevance of measures is generally weak and has potential effects dispersed among several objectives, resulting in high costs for individual objectives at best. Without the effective inclusion of an impact assessment, the outcome will rely on the capacity and benevolence of national governance systems.

In the last two decades, CAP implementation at the member state level has become more flexible in order to be closer to different situations in the growing EU membership. The legislative proposals for the CAP 2021-2027 aim to make a further step in this direction by switching from a compliance-based to a performance-based framework. While objectives and measures will still be defined at the community level, member states will be able to – based on their specific needs – set national priorities and accommodate measures. The critical role will be played by the strategic plans, which will explain those changes and will involve quantified targets based on a common list of indicators (European Commission, 2018, 12-15). The plans will have to be approved by the Commission before payments will be granted to assure the proper level of ambition and budgetary responsibility as well as respect for community principles such as a common market and World Trade Organization rules (European Commission, 2018, 11).

This article aims to test the new ‘delivery model’ of the CAP. Specifically, it aims to explore whether the overall requirements and incentives are sufficient for the member states to take actions that will result in a more effective policy. Since national strategic plans are not yet available, the new rules are tested on the 2015-2020 data for the case of Slovenia, a country that demonstrated substantial flexibility in maximising its objectives, which is inter alia reflected in its high share (52%) of rural development funds in the given period (OECD, 2017). Since these are already subject to programming rules, this makes Slovenia a good example to test the impact of the new delivery model.

The article first deals with the specifics and novelties of the new delivery model and what implications these might have. Subsequently, the experimental case study is elaborated and the main results are presented. Finally, the relevance of the results for the CAP post-2020 and EU governance are discussed, taking into account other relevant research.

What difference does the new model make?

Policy cycle as a prescriptive tool

The new CAP delivery model is based on the programme logic (McLaughlin and Jordan, 1999) and prescriptive use of the policy cycle (Cairney, 2012), which should ensure (a) a proper definition of objectives of policy intervention based on public needs, (b) a logical linkage between objectives and means (measures and resources available in the given contexts) and (c) an evaluation of intervention effects, as shown in Figure 1. The latter then also enables the backward loop to inform the next policy cycle. This system is used to make sure that decisions are in line with principles of responsible public intervention when financing is approved. Additionally, elements, such as a common list of indicators and quantitative targets, serve as a tool to ensure that each step of the implementation can be easily checked against the facts, i.e. is evidence-based. This is specifically important in the context of decentralised decision-making and dispersed responsibility (Cairney, 2016), which are typical in EU governance.

Until now, programme logic has been systematically applied in the Pillar II of the CAP, i.e. through the Rural Development Programmes (RDPs). National strategic plans, which are in the centre of the new performance-based delivery model, are envisaged to cover both pillars (income support, market-related instruments and rural development policy) as well as (to a certain extent) national measures. The plans will include assessment of needs, intervention strategy with quantitative targets and milestones based on the common EU indicators for each specific objective, financial plan, description of common elements, and monitoring and evaluation (European Commission, 2018, 95-102).

Member states will be required to report annually on the output and results of the intervention. In the case of more than 25% deviation from a milestone, the Commission is entitled to ask for a specific action plan and could, should a member state fail to respond appropriately, even withhold payments. Member states that meet at least 90% of their target values are also entitled to draw on the remaining 5% of the annual Pillar II allocation (the so-called performance bonus).

New model, old policy

There are some elements which indicate that the new model will not deliver on the high promises. The more general issue is the level playing field. The proposal differentiates nine specific objectives, divided into three groups (economic, environmental and social), plus a horizontal objective with a different level of community engagement instruments and funds that indicate a level playing field (European Commission, 41). Still, it remains open to what extent individual objectives treated as commodities or as goods should be pursued on the EU level and how to do so in order to balance responsibility and solidarity. Many researchers are very restrictive on this; Tangermann proposed a community level approach towards common market-related issues (Horseman, 2018), whereas environmental objectives would be (co)financed nationally or regionally.1

Furthermore, policy instruments (Title III of the proposed regulation) more or less stay the same along with existing problems of public intervention such as weak targeting, which results in weak transparency and distribution logic (Swinnen, 2015). For example, Pillar I direct payments have controversial effects on income due to allocation to landowners and capitalisation in land prices (Ciaian et al., 2016) as well as on other objectives, addressing, e.g. environmental issues (de facto support for production including certain intensive practices) and generation renewal (raising costs of entering farm business).

Specifics and novelties of the new delivery model

There are also more specific issues related to the delivery model, such as measuring the actual impact. The proposal involves three types of common indicators: impact (actual change), result (effects) and output indicators (implementation of instruments, finance). However, assessment of needs (i.e. context indicators), which is essential for the proper definition of objectives, is not part of the Commission’s evaluation framework. This also holds for impact since an ex post evaluation is planned for 2031, whereas the mid-term evaluation will only cover some objectives, which means that it will affect neither the present nor the future policy cycle. Meanwhile, most of the result indicators are in fact output or short-term outcome indicators (McLaughlin and Jordan, 1999) since they are mostly based on the number of beneficiaries and areas of agricultural land involved in measures. As shown by the evaluation of the current RDP system (European Court of Auditors, 2017), such an indicator framework increases the risk that the policy will lead to the maximisation of instruments as opposed to promoting efficiency and effectiveness and will prevent making trade-offs between objectives.

In recognition of these constraints, many specific objectives, such as environmental and generational renewal, are based on specific strategies and analyses. Article 92 thus refers to the principle of ‘no backsliding’ on the environment. However, this principle is defined quite vaguely, stating that the overall contribution to the achievement of the environmental objectives should be higher than in the 2014-2020 period. However, precise definition and quantification of environmental objectives is missing2 as well as explicit linkages with environmental laws (Hart et al., 2018). Furthermore, a strategic environmental assessment will not be part of the evaluation framework.

In general, the experience has shown that loosening the CAP framework has led to a less ambitious approach by the member states, potentially leading to a so-called race to the bottom (Erjavec et al., 2018a). The proposed new CAP model does not change the incentives but rather promotes less ambition to satisfy result-based targets. Even the system of awards could, due to a weak evaluation framework, lead to a less ambitious approach to reduce the risk of not being granted a performance bonus (Ibid).

Administrative capacity

An important governance element is the issue of administrative capacity. Member states have little experience with the programming of Pillar I. The ‘greening’ of the direct payment system, introduced in 2014-2020, which involved certain flexibility and required some planning at the national level, was deemed too complicated by some member states and later largely failed in terms of the environmental objectives it aimed to pursue (Gocht et al., 2016). Eco-schemes, which will upgrade the ‘greening’ component in Pillar I, are more flexible but, potentially, also more complex. Thus, the programming of Pillar I will be a substantial administrative challenge for many member states.

Moreover, stakeholder involvement commitments are rather weak (e.g. strategic plans will only involve a summary of the stakeholder consultation process as per Article 94), which means possible issues with transparency, lobby groups and accountability. Finally, the proposal does not explicitly explain how simplification, which is one of the key ambitions of the new framework, will be achieved. In fact, simplification only appears in strategic plan evaluation criteria (Article 106), which is strange given the fact that this should be in the interest of member states.

An ex ante test of programming

Our research approach was based on an ex ante test of the new CAP delivery model using the case of Slovenia. Because strategic plans for the new period are not yet available, we used data from 2015-2020, a period during which similar overall policy objectives and measures as well as programming principles were applied in Slovenia.3

Based on the analysis of the national strategic and programme documents (Official Gazette of the Republic of Slovenia, 2011), 49 national operational objectives were identified and grouped into 18 specific4 and four general objectives (broadly consistent with nine specific objectives set in the CAP post-2020 legislative proposals, see Table 1) to enable further analyses. Furthermore, 46 EU and nationally funded agricultural policy measures were implemented in 2016 and 2017, including some measures related to forestry (see Erjavec et al., 2018b).

Using the methodology for public policy evaluation (Rossi et al., 1999), we first (a) evaluated the intervention logic of each of the identified specific objectives. In the second part, a Delphi method (Linstone and Turoff, 2002) was used for a qualitative group evaluation, following the approach used by the Commission in similar cases (Chartier and Cronin, 2016).5 We asked participants to answer two questions: (b) what is the relevance and (c) what are the potential effects of each instrument for each specific objective? These questions specifically referred to the instrument design and implementation phase. Additionally, we asked participants to evaluate the cross-effects of instruments to evaluate coherence between the measures (see Table 2 and Figure 1).

Finally, budget weights, based on average annual expenditures for nationally funded measures in the period 2014-2016, and budgetary allocations to EU (co)funded measures in the period 2014-2020 were used to identify the priority of individual policy objectives. Budgetary allocation to the cross compliance was calculated as a 20% share of the budget for the direct payment schemes (minimum sanction in case of violation), whereas 12% of technical assistance expenditure in RDP 2014-2020 was used for the National Rural Network measure. To estimate the ‘allocation’ of the budget to individual objectives and their consequent priorities within the agricultural policy, the assessment of the relevance of each measure was multiplied with the quotient of the measure’s annual budget and the sum of all assessments of the measure’s relevance.

Key results

Intervention logic

The average grade for the elaboration of the intervention logic of each objective was 2.22, which means that substantial gaps were identified. These mainly referred to unsound argumentation stemming from a lack of reference to robust analyses and indicators. Indicators – where they were present – were mostly linked to results and outputs as opposed to impact, whereas limited reference to the past policy cycles (programming periods) and evaluations in particular were present.

The overall grade was the lowest in the economic group (1.86), largely due to poor substantiation of how income-related objectives were linked to the Pillar I measures, which comprise the most important financial support mechanisms of the CAP. In the environmental group, argumentation was somewhat better (2.4), mainly due to the inclusion of environmental policy requirements. Within the group of social objectives (2.0), there was substantial fluctuation of grades, with the biggest issue being the lack of data. In the area of knowledge and communication, the average elaboration grade (3.0) was a bit higher due to some well-designed national measures.

Relevance and potential effects

Only 6.2% of the Slovenian agricultural policy measures were estimated to comprehensively address specific objectives (Table 3). Examples include direct payments in the field of income-related objectives, investment support in terms of boosting competitiveness, quality schemes for agricultural products supporting value chains, agri-environment measures’ environmental objectives and advisory service knowledge transfer. A further 18.6% addressed specific objectives importantly, 16.7% weakly and 24.2% indirectly.

The economic group was addressed by the largest number of instruments (86.3%), especially the objective of income situation. However, targeting was mostly indirect or weak (53.7%). Environmental and social objectives were addressed by about half of all measures, where the share of measures that importantly or comprehensively address these objectives were higher in the case of the environment. Knowledge and communication were addressed by 60.7% of the measures, of which most only indirectly, though several were also comprehensive.

On the other hand, as shown in Table 4, 29.6% of instruments were considered to have no notable effect on particular specific objectives, while the potential effects of 8% of instruments were estimated to be important, of 29.2% weak and of 33.1% weak indirect. In the group of economic objectives, 10.6% of measures were expected to have an important impact, especially on the income situation, e.g. direct payment schemes, investment support as well as Less Favoured Area (LFA) measures, agri-environmental measures, animal welfare and organic farming support, which might indicate the problem of the distributive logic behind those measures. In the group of environmental and social objectives, on average, potential effects were expected to be weak since no more than 4.8% and 4.3% of the measures were estimated to have important direct effects on achieving objectives in these fields.

Relevance versus potential effects

As shown in Table 5, the difference between the relevance and potential effects of particular measures was found to be the strongest in the economic group (21.1%), which is mostly due to non-income objectives and structural measures, especially those of Pillar II. With regards to the environment, the difference was only slightly lower (19.1%) due to biodiversity and climate objectives and agri-environmental and organic farming measures. In the social group, the difference was negative due to the strong indirect measures, while the effect of specific measures, such as LEADER, were considered relatively weak. A positive difference in the case of knowledge and communication objectives was due to the role of measures like FADN (farm accountancy data network).

Adding financial weights demonstrated that of all resources, economic objectives were addressed by 60% and 53.9% (targeting versus impact), with income ranked highest, environmental by 22.3% and 23.3%, social by 10.9% and 14.6% and knowledge and information by 6.6% and 8.2% (see Table 6).

In general terms, there was a linkage between objectives and policy instruments since finances were broadly aligned with relevance and expected effects. However, at the same time, the gap between the relevance and expected effects of the measures was strongest in the group of (economic) objectives, which played a key role in terms of formal objectives and funding. This was a result of many weak relevant measures and weak potential effects of key measures. Effects were reversed with environmental and social objectives, where the scope of funds, allocated via weakly targeted measures primarily addressing other objectives, brought certain effects; the efficiency, however, was called into question since more specific measures had weak effects.

Coherence

The share of measures with complementary or synergy effects is highest (51.6%) in the economic group (see Table 7). The smallest share of instruments with positive effects (17.3%) and the highest share of those with negative effects can be found in the environmental group (especially Pillar I). In the social group, there is comparably more complementarity and synergies. In the group of knowledge and communication, the share of synergies is, given the nature of these measures, rather low.

Discussion: Dispersed effects and weak efficiency

Results of our study demonstrate that the programming framework behind the CAP 2014-2020 in the case of Slovenia provided for only general linkage between the objectives and policy instruments. Moreover, the potential effects in non-economic groups were, paradoxically, even higher than the overall instrument relevance, which could imply that in the implementation phase, national authorities took advantage of flexibility such as setting particular criteria to align measures with specific environmental, social, etc. objectives. This would support the idea behind the new CAP delivery model.

The detailed view of results, however, demonstrates that potential relevance of most of the instruments is one average weak, especially in the economic group. Moreover, the potential effects of key measures in all four groups are also weak. This results in the relatively strong dispersion or ‘secondary’ effects, which in the areas addressed by relatively few measures, such as social objectives, produces a higher score on potential effects compared to their relevance. This is exacerbated further by adding financial weights, demonstrating poor targeting vis-à-vis dispersed potential effects of financially strong (economic) measures. While formally this means that some objectives are reached, the efficiency is highly questionable. It also shows negative effects, especially in the area of environment, and a lack of synergies, e.g. regarding the role of knowledge.

How applicable are these findings outside of Slovenia and for CAP post-2020? When evaluating the CAP 2014-2020 programming period, the European court of auditors argued that interventions target too many objectives that were too general (European Court of Auditors, 2017, 21) and highlighted a weak linkage between the objectives and interventions (European Court of Auditors, 2018). This problem was not only evident for the Pillar I instruments, but also for the Pillar II (Ibid). Consistency between the document-based evaluation and the group evaluation confirm the importance of formal intervention logic for actual quality of intervention. Ex ante studies of the CAP 2014-2020 argued that the reform would not bring any major changes (Gocht et al., 2016), and even indicated that intervention logic was reversed, i.e. that objectives were set to legitimise an existing intervention based on distribution logic (Erjavec et al., 2015), whereas ex post impact assessments of CAP 2014-2020 confirm the main arguments of this research, including the positive but costly ‘secondary’ effects, e.g. of environmental measures on employment (Garrone et al., 2019).

The programme logic of the new CAP will resemble the current RDPs, which means that most of the shortcomings of the current planning system will probably remain. This was specifically demonstrated in the case of Slovenia, which had one of the highest shares of the Pillar II budget. Moreover, a document-based evaluation specifically emphasised the role of formal environmental policy-based commitments as well as the absence of impact indicators and impact assessments. It also showed an overall lack of quality data and competences of the policy planners and indicated a lack of transparency and interest logic, questioning the overall stakeholder consultation process. All of these elements directly refer to the specific issues of the proposed new delivery model as was pointed out in the analysis of the proposed legislation.

Conclusion: Decentralising responsibility

Results show that in the absence of a strong policy impact evaluation and its inclusion into the programming of future policy programmes, substantial improvements in policy effectiveness cannot be expected and will at best rely on the benevolence and administrative capacity of governance systems within member states. Thus, the intention of proposed CAP post-2020 remains questionable. For Matthews, the proposed new CAP delivery model should be interpreted in the context of the criticism of centralised decision-making (Erjavec et al., 2018a), Brexit and less funds available as well as the need to bring responsibility for policymaking closer to member states in order to prevent scapegoating. Thus, while the new CAP will not change much in terms of overall policy efficiency, the strategic plans will at least help the Commission to share the responsibility more equally with member states.

* Research received funding from Slovenian Research Agency and Ministry of Agriculture, Forestry and Food as a part of Targeted Research Project V4-1608 “Impacts and perspectives of CAP on Slovenian agriculture and farming”.

• 1 Cofinancing of main instruments was involved in some of the earlier versions of Commission’s proposal but was later removed.
• 2 An example is a methodology of measuring contribution to climate objectives (Article 87), according to which this criteria is met by 40% of basic income support and complementary income schemes, 100% of new eco-schemes, all environmental RDP measures and 40% of measures on LFAs in spite of weak and even controversial contribution of these to climate and environment (European Commission, 2018, 91-92).
• 3 At the start of implementation of CAP instruments in 2014-2015, the Commission published technical guidelines to define intervention logic in terms of objectives, trends and measures. It made a clear difference between monitoring and evaluation and linked the latter with what works, in what conditions and why not. It argued that expected versus actual results in all phases are key for strategic and practical planning. Document also provided a list of impact, result and context indicators (European Commission 2015, 9-10, 56). This laid grounds for CAP 2021-2027.
• 4 Food security, Income situation, Stable incomes, Efficient use of resources, Accessibility of resources, Value chains, Food safety, Climate change, Biodiversity conservation, Soil protection, Water protection, Animal welfare, Employment in rural areas, Quality of life, Social inclusion, Knowledge generation, Knowledge transfer, Information and awareness.
• 5 Group evaluation involved 18 individuals: 10 from academia and expert organisations (8 with background in agricultural economics, 1 in forestry and 1 in social sciences), 6 from ministry of agriculture and 2 from chamber of agriculture. Two rounds of evaluation were held: evaluation in sub-groups (15 and 16 February 2018) and consolidation of grades (1 March 2018).