Title: Modelling for EU Policy support: Impact Assessments
Authors: ACS SZVETLANAOSTLAENDER NICOLELISTORTI GIULIAHRADEC JIRIHARDY MATTHEWSMITS PAULHORDIJK LEENDERT
Publisher: Publications Office of the European Union
Publication Year: 2019
JRC N°: JRC117250
ISBN: 978-92-76-09671-9 (online)
ISSN: 1831-9424 (online)
Other Identifiers: EUR 29832 EN
OP KJ-NA-29832-EN-N (online)
URI: http://publications.jrc.ec.europa.eu/repository/handle/JRC117250
DOI: 10.2760/748720
Type: EUR - Scientific and Technical Research Reports
Abstract: The objective of this work is to systematically analyse how models are used in support to the policy formulation phase of the EU policy cycle. We focus on European Commission (EC) Impact Assessments (IAs). The main framework of the EU regulatory policy, the Better Regulation (BR) Agenda (European Commission 2015), sets a clear commitment to a transparent and sound use of evidence for all EU policy making activities. The Better Regulation Guidelines (European Commission 2017), which complement the Agenda to provide concrete guidance throughout the policy cycle, recommend to quantify costs and benefits to the extent possible to support the policy formulation phase. In doing so, the EC makes extensive use of models. A better understanding of these models and how they are used can then contribute to a sound use of evidence in support to EU policies. The Commission's Competence Centre on Modelling (CC-MOD) promotes a responsible, coherent and transparent use of models at the EC. As part of its activities, this analysis systematically investigates how models are used in support to the policy formulation phase, by looking at EC IAs which are publicly available. A total of 1063 IAs carried out in the years 2003 to 2018 have been investigated to examine the frequency and characteristics of model use, by using text mining techniques complemented by manual post processing. The research is facilitated by and feeds back into MIDAS, the Commission-wide Modelling Inventory and Knowledge Management System developed and managed by CC-MOD (Ostlaender et al. 2019), which directly contributes to enhanced transparency and traceability of models used to support policies. Our results show that models are used in 16% of the total IAs (173 out of 1063 IAs), with a positive trend over time, starting with only two IAs using models in 2004, to around 25-30% from 2015 onwards. We identified 123 different models contributing to IAs. These models have been developed or run by the EC or by third parties. More than half (53%, or 65) of these models were used only once, which leads to considerations related to the efficiency of model use and reuse in the EC, as well as on the scope for improved coordination of model related products and services. On the other hand, some models do dominate: the top 10 models contributed to 10 or more IAs, and were used in 66% (or 114) of the total number of IAs using models. Included among the top 10 models are also those used for the development of the series of EU Reference Scenarios (1), led by DG ENER, DG MOVE and DG CLIMA. Indeed, the consistent use of the same series of baselines in the areas of energy, transport and climate goes in the direction of increased consistency across policy areas and fields of analysis, though there is still room for strengthening the coherence in baselines across the whole EC (Marques et al. (2017). At the same time, given the dominance of these models to support the policy formulation phase, it is essential that they undergo careful quality scrutiny and that maximum transparency and traceability of results is ensured. Policy areas with the highest number of IAs using models are environment (including climate), internal market, transport and energy. However, this could also reflect other factors, such as the frequency at which IAs are carried out in the various policy areas. In addition, it should be remembered that models can also be used in other policy relevant studies, or for internal notes and analyses which remain unpublished. Finally, results show that 94% (116) of the total number of models that were used in IAs were used for ex ante assessment of policy options. This is to be expected, since indeed the assessment of policy options is an extremely relevant task for quantitative analysis in IAs. As mentioned, most of the top 10 models were also used for baselines. Transparency is crucial to understand how models work and to validate their behaviour, to encourage their sound and widespread use in support to policy making contributing also to a more effective and efficient use of resources for model development and use within the EC. The requirement introduced by the BR Agenda in 2017 for an annex on ‘Analytical models used in the preparation of the impact assessment’ contributed to better model descriptions and an increased transparency of the methodology used. There is, however, still room for improvement in better documenting models as the individual model descriptions in IAs are still of different quality. The information and reports generated by MIDAS on models and model use can be used in this respect. We also identified some major challenges related to referencing in IAs, such as lack of harmonized and adequate references or outdated hyperlinks, which made the quantitative evidence untraceable in several IAs. Since 2017, the BR foresee that, when IA analysis relies on modelling or the use of analytical methods, the model should be documented in the corporate modelling inventory MIDAS (European Commission 2017). This represents a major step forward in terms of transparency. At the same time, it is also clear that further action is needed to use and promote best practices to ensure transparency and accessibility over time of the evidence base in support to IAs. In addition to the BR guidelines, the JRC, as the science and knowledge in house service of the Commission, can provide additional assistance and support to the Policy DGs. To conclude, the results of our analysis contribute to the BR Agenda by highlighting aspects of transparency, coherence, traceability and accountability in the use of evidence for EU policy making.
JRC Directorate:Joint Research Centre Corporate Activities

Files in This Item:
File Description SizeFormat 
kjna29832enn.pdf2.02 MBAdobe PDFView/Open


Items in repository are protected by copyright, with all rights reserved, unless otherwise indicated.