Essays on Exploring Innovations to Enhance the Quality of European Impact Assessments

Abstract

Since the European Communities were established in 1957, more than 100.000 legislative acts have been enacted. However, a lesser-known aspect of the European Union’s decision-making process, pertains to the requirement, instituted since 2002, for all legislative interventions bearing significant implications for the economy, environment, or society to undergo Regulatory Impact Assessments (RIAs or IAs) before enactment. These assessments serve as key instruments for ensuring transparency and accountability in the decision-making process, equipping decision-makers with the empirical evidence essential for informed decisions. By delineating the necessity to intervene, the array of available intervention modalities and the potential impact of their intervention, RIAs empower decision-makers while also informing the public of the factual foundation underpinning decisions. Given the expectation that Impact Assessments lead to a better regulatory environment, numerous initiatives have been developed worldwide to uphold IA quality standards. According to the Organisation for Economic Co-operation and Development (hereafter OECD), the EU stands out among these efforts, with its Better Regulation Agenda serving as a best practice example. Nevertheless, a leadership position does not indicate a flawless system and critics across public, academic, and private sectors voice increasing concerns surrounding the way Impact Assessments are being conducted. Academic literature on the topic of the quality of the EU’s IAs is existent , albeit limited. Notably, there are very few studies that explore potential avenues for the improvement of the IA rules and procedures, with existing ones often outdated, and failing to take into account the latest advancements in Better Regulation. Furthermore, the focus of these studies tend to be narrow, centring on specific topics such as the use of the (new) behavioural approaches in the IAs, role and design of the public participation, political influence, etc. Consequently, a more comprehensive view in terms of improvements in the IA process is necessary to complement the current body of literature. This thesis endeavour to address these gaps by identifying potential avenues of improvement in the quality of the European Impact Assessment rules and procedures. It undertakes a dual approach, firstly examining the regulatory framework governing the Impact vi Assessment, to identify gaps and areas of improvement, and secondly, delving into the practical facet of these rules, assessing case studies to see how the practice is affected by the identified deficient rules. The primary objective is to examine whether, and to what extent can the IA process of the EU be refined. It achieves this by means of three research papers, answering seven research questions. The first paper attempts to identify the universal indicators of quality in Impact Assessments, extending beyond the confines of the EU. It synthesizes insights drawn from academic discourse, the results of the Stocktaking Exercises conducted by the Court of Auditors, and the opinions of the Regulatory Scrutiny Board, the Commission’s independent body for quality. To ensure relevance and universality, only elements common to at least two of these sources are retained. As a result, thirty-seven indicators are delineated and organised into six clusters: evidence-backed choices, the transparency of the process, the selection and quantification of the impacts, the methods used in the assessment of the impacts, subsidiarity concerns, and general indicators of overall quality. These thirty-seven indictors serve as guiding principles for Impact Assessment practitioners, guiding their efforts during the development of the Impact Assessment. Following the identification of the denominators of quality in IA, we proceed to evaluate one hundred Impact Assessments conducted by the European Commission spanning the period 2016-2021. This examination aims to gauge their adherence to these indicators. It is important to note that both the framework and the analysis of the IAs have been performed prior to or immediately after the latest update of the Better Regulation Guidelines, and it does not include new elements, such as the strategic foresight, One-in One-out, competitiveness checks, etc. Through this analysis we affirm the views of the OECD, recognizing the EU Commission as a frontrunner in terms of IA development. Nonetheless, the analysis uncovers several shortcomings, two of which are addressed in the chapters three and chapter four. These include a tendency to overlook the integration of intangible impacts in the assessment of the impacts – despite the fact they are initially long-listed, they are rarely included in the more comprehensive analyses. Additionally, shortcomings in the application of the vii principle of subsidiarity are identified, a concern echoed by the national parliaments and by external stakeholders alike. The findings of this paper yield several applications: for civil servants of the EU and beyond, they can serve as a tool for raising awareness regarding the most salient issues addressed in the Impact Assessments. Additionally, they can facilitate an efficient resource allocation, by directing the work of external consultants towards the directions that are most necessary. Lastly, they can serve as a learning tool, enabling the civil servants to learn how to best develop impact assessment, and identify the key elements to eb addressed. The second application extends to the researchers, who now have a foundational framework for a more comprehensive and universally adaptable framework that can be used for any Impact Assessment and for supporting the public authorities in both their development and appraisal of Impact Assessments. Lastly, the general public can utilise the output of our paper as a blueprint for a more objective, capable and informed assessment of the decision-making process, ultimately fostering increased participation therein. The second paper, presented in Chapter three, addresses one of the shortcomings identified in the application of the proposed framework to the one hundred IAs of the commission in chapter one – the non-integration of intangible impacts in the impact analysis. The starting point is the use of public consultations as sources of evidence on which Impact Assessments are based upon. These consultations consist of two components: the informational material presented to the respondents, and the survey itself. The information presented to the public, with the purpose of educating them on the topic under scrutiny, is, unavoidably, framed in a specific way, influencing respondents’ perceptions, potentially impacting their responses. Subsequently, these responses are embedded in the Impact Assessment to help identify the problem, determine potential solutions, or they might reflect the view of the public with respect to the importance of the impacts of the potential interventions. The second premise on which the paper is based upon is one of the findings identified in chapter one: that intangible impacts are rarely included in the assessment of the impacts. It posits that, if the impact assessment is informed by the responses to the public consultations - responses which might be biased by the information received - , the reason for the non- viii inclusion of such intangible impacts might be supported by the responses provided by the participants to the public consultation. In essence, the paper aims to explore whether the framing of the information in the public consultation affects the perceived importance of the various impacts. This is done by means of a vignette experiment, where we simulate a public consultation on the topic of the Consumer Credit Directive (CCD). We ran the experiment across three countries (Belgium, Germany and Romania), and we included a total of 587 participants which were split in three groups in each country: one control group and two treatment groups. In this way, an equal number of participants from each country participated in the control groups and in the two treatment groups. This has allowed us to test treatment effects (the effect of participating into a treatment group as opposed to a control group), and country effects (differences in the responses to the same treatment group, but from different countries – for example, differences in the participants to the first treatment group from Romania and the participants to the first treatment group in Belgium) in addition to testing the importance of the intangible impacts. Initially, all participants were engaged in the same survey gathering information on their level of education, familiarity with the topic and assessing trust in various institutions. The second step involved facing them with different vignettes – or different information – based on their assigned group. Therefore, the control group received the same information that was published with the real public consultation on CCD, whereas the treated participants received additional information – the first treatment group received additional information on the importance of trust on the topic of consumer credits and how this is diminished by the current legal (or lack thereof) framework, the effects of having low trust, and how can this lack of trust be increased by an EU intervention. The second treatment group covered the same information, and, in addition, it provided data, quantifying the impacts of a lack of trust and explaining the impact an increase in trust will have overall (such as a reduction in costs, for example). In the third step, after receiving diverging information, the respondents have been surveyed regarding the importance of the selected impacts (trust and consumer protection being the intangible impacts, and consumer costs and costs for industry being the more tangible ones). This allowed an examination of the differences in the ix importance awarded to the different impacts after receiving differing information. Lastly, the participants answered questions on their personal characteristics. The results show that framing of the information has limited effect on the importance awarded to the different impacts, but this effect is nonetheless discernible. Notably, by examining the country effects, we noticed that this effect was visible and positive in Romania and Belgium – namely, the treated participants tended to attribute grater importance to the intangible impacts – and it was visible but negative in Germany where the treated participants awarded less importance to these impacts. This underscores systematic differences in the manner citizens from different countries respond to information framing. Additionally, another relevant finding pertains to the importance of the intangible impacts: regardless of the country or survey of the participants, the intangible impacts are at least as important as the tangible ones. The implications of this paper are two-fold: firstly, we have noticed that the way information is framed at the public consultation phase influences the importance assigned to impacts by different countries. In the context of the impact assessment, this means that the choice of impacts, or the importance awarded to these impacts (for example within a Multi-Criteria Analysis), may be swayed by how this information is framed. Consequently, there is a need for increased objectivity in drafting the public consultations. Secondly, we may deduce that the “one-size-fits-all” character of the EU public consultations to capture the views of the entire EU population. It is generally acknowledged that countries like Germany, Austria and Belgium have much higher participation rates in public consultations compared to the newer EU member states. Our experiment has demonstrated differences in responses, with participants from Germany for example, responding markedly different compared to the participants from the other countries. By relying therefore on the responses of the participants from the countries with high participation rates, which might diverge from the rest of Europe, there is a risk that IAs do not reflect the view of the entire population. To address this, one suggestion would be that more publicity is being put in place for the public consultation among the states with lower x participation rates, as well as among the societal groups that present less interest in such activities. The third paper explores the application of the principle of subsidiarity, one of the elements identified as deficient in our analysis of the one hundred IAs conducted in the second chapter. In essence, the subsidiarity principle ensures that decisions are made at the most appropriate level - whether at the EU or the Member State level – for each issue at hand. It consists of two tests – a necessity test and an EU added value test. The necessity test explores whether the identified problem cannot be sufficiently tackled by the Member States together with the lower level of administrations. The EU added value test comes as a second level test, theoretically employed when the first one is passed. It assesses whether EU intervention would provide added value compared to interventions at a lower level in instances when intervention is proven to be necessary at EU level. Presently, the application of the subsidiarity principle in the context of the Impact Assessment is guided by the outcomes of the Report on Task Force on Subsidiarity, Proportionality and “Doing Less More Efficiently” deployed by President Juncker in 2017, and is based on a subsidiarity grid that requires fulfilment of the two tests through a set of twelve questions. The criticisms surrounding the two tests stem mostly from the national parliaments and are focused on several topics: the easiness with which EU intervention can be justified by means of harmonisation, contribution to internal market or economies of scale, and the fact that the incapacity of the Member States to tackle problems by themselves is driven by the definition of the objectives, definition oftentimes made in terms of harmonisation, or the achievement of internal market goals. Moreover, there are instances in which multiple objectives are defined, among which, at least one of them can be better achieved at EU level. The application of the principle, therefore, creates uncertainty in the context of the Impact Assessment as well. The paper inquires thus, whether the application of the principle can be improved, and if affirmative, it explores avenues for enhancement. A comprehensive literature review revealed several elements crucial for the assessment of subsidiarity, which are not currently covered by the grid, including the salience of the issue, or issues of accountability or heterogeneity of preferences. Building xi upon these findings, a new framework is developed, encompassing novel arguments against which the subsidiarity principle can be evaluated. A cross-check against 43 reasoned opinions from national parliaments, covering 14 initiatives, confirms that our framework grasps such concerns as well. Another innovative aspect introduced by this paper is the discussion surrounding the dichotomy between the concepts of steering vs ordering legislation, representing the two ends of an intervention spectrum. In essence, steering regulations aim at particular foreseen results and are comprised of commands, measures or specific directions. Typically, they create distributional effects, and, as a consequence, benefit specific groups. Conversely, ordering regulations are aimed at providing order to society, creating win-win situations. They set ground rules by addressing market failures and they do not serve the interests of any group in particular. Economics and public choice scholars argue that ordering regulations are best tackled at the lower political levels due to the controversies created by the distributional effects, requiring better accountability. Ordering regulations on the other hand, are easier justified to be addressed at the higher levels, such as the supranational level of the EU, as their aim is to provide order to the society, and through a richer pool of resources this can be easier done. However, determining the specific type of an intervention is not straightforward, as it is almost impossible for an initiative to be of a purely steering or a purely ordering character. Our main argument herewith is that determining the type of intervention can streamline the application of the subsidiarity principle in the context of the IA. When intervention leans towards an ordering character, the subsidiarity test can be more lenient, as intervention at EU level is more straightforward. In cases where the intervention exhibits a steering character, but doubts persist regarding the ability of the lower levels to intervene, a stricter subsidiarity test should be employed. This approach could be also complemented by a stricter application of the the principle of proportionality, ensuring that if EU intervention is necessary, a rigorous proportionality test is employed. This would guarantee that the intervention is strictly limited to what is essential necessary, preventing an overreach. xii The significance of this paper lies in it shift of the focus in the subsidiarity test, moving away from EU added value to conducting a comparative assessment of EU added value against the benefits of leaving the decision with the Member States, even if necessity has been proven. The original intent of this thesis was to identify all areas of improvement of the Impact Assessment and create a blueprint to enable the civil servants to constantly improve their IA practices based on these identified elements. However, it quickly became apparent that such an objective is exceedingly ambitious, and should extend beyond the period of a typical doctoral study – it becomes a lifetime project. It is necessary to acknowledge, thus, the limitations faced by the current thesis, which can be addressed by further research. One of the recurring limitations is the limited number of case studies upon which the conclusions of all the chapters are based. Chapter two assesses one hundred IAs against the proposed framework from the period 2016-2021; chapter three focuses the simulated consultation on the Consumer Credit Directive, while the fourth chapter exemplifies the application of the new proposed framework through one case study – the media act. It would be useful thus, for future endeavours to consider additional or alternative cases. Second, the thesis does not cover the new requirements under the newest Better Regulation package. At the beginning of writing this thesis these requirements had yet to be adopted, and by the time they could have been addressed, they were still at the beginning of their adoption in the Impact Assessment framework. Consequently, there was limited opportunity to identify deficient practices or areas of improvement. Any further research, should thus, cover elements such as the One-in, one-out, strategic foresight, etc. Lastly, as only two elements that have been identified in need of improvement in Chapter two have been addressed in depth, suggestions for improvement or best practices could be formulated for all elements.