Europe’s energy information problem

Executive summary

Comprehensive information on energy-related topics, such as take-up of heat pumps, industrial natural gas consumption and prices, and battery connections to the electricity grid, is either not available in Europe in a timely manner, or not available at the level of granularity, reliability and consistency needed for informed decision-making. Certain information is simply not collected, while other information is collected but is not comparable or consistent across Europe, or is hard to access. European policy targets, such as greenhouse-gas emission reduction pathways, are evaluated using models for which input assumptions and parameters are not public knowledge.

It should be a European Union priority to improve this situation. Doing so will enable better decision-making by policymakers and companies. This is especially relevant when Europe faces the triple challenge of decarbonisation, ensuring security of energy supply and growing its internationally competitive energy-consuming industries.

The EU status quo is that good energy information is provided by a mix of institutions, agencies, national bodies, industrial associations and non-governmental organisations, but with substantial room for improvement. Lessons can be learned from the United States Energy Information Administration and the European Environment Agency, which was established to better coordinate climate data.

Improving energy information will involve difficult political decisions. A process should be started to evaluate the options and to measure the continued cost of inaction.

Europe suffers from an under-provision of energy information. Though energy policy choices are complex, and can have major economic and social impacts, comprehensive information is not available in a timely manner or at the level of granularity, reliability and consistency needed for informed decision-making. 

Clear information is necessary to evaluate current energy systems, to assess the impact of planned policies, to plan infrastructure and to identify policy priorities. Making information open and available helps to build trust and facilitate broader participation in political processes while limiting the ability of special interest groups to lobby for suboptimal approaches. The European energy system is evolving rapidly and filling this information gap is today more important than ever.

Better information will help inform choices over the billions of euros being spent on power generation, heating and cooling equipment, vehicles, transportation infrastructure, energy networks, energy storage and factories. These investments are highly interdependent and synchronising them in terms of timing and geography helps minimise costs and reduce inefficiencies. This can be done only with system-wide information. The European energy fuel bill ¹ With a liberal use of the word ‘fuel’ including electricity, uranium products, oil and natural gas. is huge and mostly goes on imports; this will change dramatically as the energy transition progresses. Conversations about the implications of this need to be informed with clear information.

The 2022 energy crisis showed that European policymakers lack reliable information to identify and act on evolving energy-security threats. Data on precise imports of Russian natural gas to individual European countries and the ability of their countries to cope without that gas was not available in an easily accessible or timely manner. Assessing the implications of a complete Russian gas cut-off is complex, yet with the benefit of hindsight, a more constructive public debate would have been possible with clearer information and transparency on the main assumptions ² For example, the German federal network agency, Bundesnetzagentur, dramatically overestimated the potential shortfall of gas in Germany because of pessimistic assumptions about how much gas could be imported from neighbouring countries. .

Meanwhile, industrial policy has become more relevant as governments tackle a series of challenges including the green transition and making supply chains more resilient. Industrial policy involves government decisions to redirect productive forces in the economy to preferred sectors. Like a company making an investment, deploying effective industrial policy requires a government to be as well informed as possible on underlying economic factors.

This Policy Brief makes recommendations on how the European energy information problem can be addressed. We categorise the types of information missing in Europe and then describe what is available via the US Energy Information Administration (EIA), before identifying the institutions that already provide helpful European energy data. These institutions are important, but are neither comparable to the EIA nor sufficient for European needs. We conclude by discussing different options for Europe to close its information gap and the trade-offs this entails. 

We categorise Europe’s energy information shortcomings into six ‘gaps’, meaning there is an under-provision of energy information but improvements are possible (Table 1).

Table 1: Europe’s energy information gaps 

^{Source: Bruegel. Note: bcm = billion cubic metres. DG ENER = European Commission energy directorate-general.}

The first shortcoming is that certain primary information is not collected systematically – for example, information on investments in manufacturing of clean-technologies such as batteries (which will be crucial in the future energy system and potentially important for security of supply) and clean-tech demand, or on investments in electricity grids. More energy-efficient buildings are crucial for climate targets, yet European data on household renovation rates is either missing or unclear (Keliauskaitè et al, 2024).

Second, certain information is collected at national level but is not comparable at European level. This problem is exacerbated by countries collecting and publishing information in different ways and in different places.

A third problem is the consistency of information. Several European Union bodies collect and publish data under different regulations. These institutions occasionally publish different data for what appears to be the same variable. For example, Eurostat and the European Commission’s energy directorate-general (DG ENER) published significantly different numbers for natural gas imports to the EU in 2022 (334 billion cubic metres vs 406 bcm) ³ Query to Eurostat database nrg_cb_gas for imports to EU27_2020 returns 405,506 million cubic metres in 2022. Figure 11 in DG ENER (2023) reports that EU imports of natural gas were 334,000 million cubic metres. . Clearer methods for the validation, querying and updating of data are important to improve trust.

Fourth, barriers restrict the accessibility of information. Easy information access for as wide an audience as possible should be a core principle of energy-data provision. Collecting and organising data in a comprehensible manner is complex but can be done according to best practices. However, this is complicated by European energy data provision being spread across multiple providers and online systems ⁴ There is no standardised application programming interface (API) for accessing European energy data. A user must access electricity and gas flows, for example, from different platforms and using different methods. A consistent API does exist for all the available Eurostat databases. .

When information requires not only data collection but also processing, challenges arise when this additional processing is done by non-partisan actors. This may create trust issues relating to objectivity and reliability. For example, ENTSO-E is the public association of European electricity transmission system operators. It is responsible for producing forward-looking plans that identify required investment in European electricity transmission systems. Results are heavily influenced by modelling choices about future scenarios, such as future technology costs. The same is true for ENTSO-G, the equivalent association for gas, and the European gas transmission system.

Finally, models are necessary to assess system-wide issues such as the implications of national energy and climate plans and network development programmes. Models process assumptions on the state of the current system and its evolution, as well as how changes to individual parameters will affect the rest of the system. This is an attempt to approximate reality and as such results only make sense if one understands the specifics of how models have been constructed. 

Unfortunately, much energy modelling in Europe is performed with proprietary data and code. This results in a lack of transparency on how models arrive at final results, and reduces the possibility of stakeholder engagement. Since 1990, the PRIMES model has been increasingly used by the European Commission as a reference tool for assessing major energy and climate policy decisions (E3 Modelling, 2018). PRIMES is owned by a United Kingdom consultancy and the data, code and assumptions underpinning the model are not made publicly available. This makes it impossible for external stakeholders to properly evaluate European Commission modelling for important issues such as the 2030, 2040 and 2050 climate targets. Part of the issue is that certain datasets needed to run the model are only available from commercial data providers and public distribution is not permitted. This is the case, for example, for much data on commodity or fuel prices from energy exchanges, which is often only available on a commercial basis. 

Figure 1: Energy information provision in the EU

^{Source: Bruegel. Note: opinions on data quality can be subjective; this figure reflects the authors’ views about the current situation based upon their experience in energy policy research, and as such should be taken as illustrative. Red indicates ‘poor’ availability, grey ‘partial’ availability and ‘green’ good availability. Feedback is welcomed. DG ENER = the European Commission’s energy directorate-general; ACER = EU Agency for the Cooperation of Energy Regulators; ACEA = European Automobile Manufacturers’ Association; AGSI = Aggregated Gas Storage Inventory. ICPEI = Important Projects of Common European Interest; ETS = emissions trading system.}

During the 1970s, oil prices rose dramatically, leading to the first global energy crisis. This had a transformative impact on how policymakers thought about energy and related geopolitical issues. In response, the US government created the Department of Energy, with a “central, comprehensive, and unified energy data and information programme” – Energy Information Administration (EIA) ⁵ The EIA was established by the 1977 US Department of Energy Organization Act; see https://www.govinfo.gov/content/pkg/STATUTE-91/pdf/STATUTE-91-Pg565.pdf. .

The EIA is mandated to “collect, evaluate, assemble, analyse, and disseminate data and information” on energy reserves, production, demand and technology. Such information should be made promptly available “in a form and manner easily adaptable for public use”. The EIA is widely considered a successful public initiative and, as of the time of writing, remains an important information provider for both US and global energy market participants.

The EIA collects most data through surveys that energy companies are legally required to complete and submit. These surveys request information on company operations as they pertain to energy, including by source and geographic area. Collected information is used to provide a consistent and statistically accurate understanding of energy industries. 

This allows the EIA to publish a wide variety of information that helps market participants and policymakers understand and explore energy-related issues. Examples of data include market prices for energy commodities and reports assessing the state of energy markets on a monthly, quarterly and annual basis. The EIA also operates a transparent and open-source National Energy Modelling System (NEMS), which is used to produce an Annual Energy Outlook ⁶ See https://www.eia.gov/outlooks/aeo/. There was no Annual Energy Outlook in 2024 as forecasting updates were made. See Laura Sanicola, ‘EIA to skip 2024 energy outlook, update models for new technologies’, Reuters, 26 July 2023, https://www.reuters.com/article/business/energy/eia-to-skip-2024-energy….  that helps policymakers identify risks and investors to foresee opportunities. NEMS is used to analyse policies.

The EIA is politically independent and has its own prerogatives in prioritising which forms of information to collect and make available. The EIA’s head, or administrator – appointed by the US President – is not required to obtain the approval of any government officials for information priority decisions, and has operated autonomously of any political influence. The main form of accountability is that the US Congress must periodically renew the EIA’s funding of approximately $150 million per annum.

The EU has no equivalent to the EIA. Instead, a mix of institutions, agencies, national bodies, industrial associations and non-governmental organisations provide helpful energy data, but in an uncoordinated manner.

We note three main differences between the EU and US approach. First, the US gives the initiative for developing new data sources to the EIA, while in the EU the process for collecting new sources of data is subject to a long, political and bureaucratic process. Second, in the US, information is centralised and access is provided in a standardised way at one location. In the EU, information is scattered across multiple providers. Finally, the EIA operates an open-source model for policy analysis. There is no comparable institutional capacity in the EU, notwithstanding important contributions from the open-source modelling community.

4.1 Reporting obligations

Much of the existing European information stems from reporting obligations set out in EU laws. There are over 700 energy-reporting obligations (Figure 2) ⁷ Note that these reporting obligations are not always necessarily new. In certain cases, they replace reporting obligations from earlier versions of legislation or consolidate them, as was done for example with the Regulation on the Governance of the Energy Union (Regulation (EU) 2018/1999). . These obligations fall on national authorities, the European Commission and electricity and gas transmission system operators. Figure 3 shows the proportion of the 700 reporting obligations by reporting party. 

One problem is that the definition of data to be reported is often not aligned across different regulations that define the scope and nature of what should be reported. This is likely behind Eurostat (with information received from national statistical agencies) and DG ENER (with information derived from ENTSO-G) arriving at different numbers for natural gas imports (Table 1). 

Figure 2: EU energy reporting obligations by year of adoption and source

^{Source: Bruegel based on Governance Team/DG ENER/European Commission based on Eurlex.}

Figure 3: EU energy reporting obligations by responsible entity

^{Source: Bruegel based on Governance Team/DG ENER/European Commission, based on Eurlex.}

The European Commission has committed to reduce these reporting obligations by a quarter (European Commission, 2024). Initiatives to improve the provision of energy information should also eliminate any unnecessary administrative burden. However, there is a balance to be struck between reducing administrative burden and ensuring access to essential information that can improve policy and investment outcomes.

4.2 The main data holders

European Commission directorates-general

Eurostat, the EU’s statistical office, is a European Commission directorate-general. Its role was expanded in the 1980s through the creation of a European Statistical System which provides guidelines for the types of data that should be provided. Eurostat does not collect data directly itself but receives inputs from national statistical institutes and maintains standardisation and consistency between national institutes. Eurostat does not have the mandate to collect energy data at the scale or granularity of the EIA.

The European Commission energy directorate-general (DG ENER) is primarily focussed on policy development and implementation. While DG ENER collects and publishes ⁸ Including a weekly oil bulletin, quarterly gas and electricity market reports and an annual statistical energy pocketbook.  a limited amount of helpful information, it is largely reliant on data from third parties, notably the national energy and climate plans of EU countries, which are a rich source of information on national energy and climate policy priorities.

Another directorate-general, the Joint Research Centre (JRC), is the Commission’s science and knowledge service. The JRC collects and generates data through research to support the Commission’s overall policy agenda, and is allowed a certain degree of flexibility to respond to policy developments. The JRC has developed the POTEnCIA model to assess impacts of different energy and climate policies on the EU’s energy system ⁹ The model builds upon the Integrated Database of the European Energy System (IDEES), also developed by the JRC. IDEES is a database of highly disaggregated energy-economy-emissions data for all EU countries across different sectors. .

Institutions established in support of the European Energy Union 

In 2009, the EU adopted rules – known as the Third Energy Package – aimed at further liberalising energy markets and deepening integration. Among other measures, the package created two public associations, ENTSO-E and ENTSO-G, responsible for transmission systems in electricity and gas markets, respectively. The associations act as European convening points for national electricity and gas system operators. They are responsible for ensuring smooth operation of European energy markets and facilitating cross-border network integration.

The EU obliges companies to publish hourly data on electricity and gas flows, including generation, transmission and demand. This data is hosted on ‘transparency platforms’ operated by ENTSO-E and ENTSO-G ¹⁰ The ENTSO-E transparency platform is available at https://transparency.entsoe.eu/; the ENTSO-G transparency platform is available at https://transparency.entsog.eu/#/map. . This is commendable but it is a limitation that data is stored on separate websites with somewhat difficult processes for accessing the data. Hirth et al (2018) identified issues with data quality, noting that market participants rely on commercial data for this reason.

The associations are also responsible for producing ten-year network development plans (TYNDPs), which detail the infrastructure investments and projects of common interest needed to deepen integration of EU energy markets. 

The Third Energy Package also created the Agency for the Cooperation of Energy Regulators (ACER). ACER collects data on wholesale energy markets across Europe in order to monitor for market abuse and assess operational efficiency. ACER provides a collection of annual reports and certain datasets and offers expertise that aids the European Commission’s decision-making process (Jevnaker and Saerbeck, 2019). One identified channel through which this operates is to reduce the information asymmetry relative to national governments, which can count on their own public administrations.

ACER also objectively evaluates the ENTSO-E and ENTSO-G TYNDPs, with the intention to provide oversight on the grid investment needs reported by these industry associations. In response to the ENTSO’s 2024 draft TYNDPs, ACER raised concerns that the plans were misaligned with climate targets and suffered from a lack of transparency ¹¹ See ACER press release of 1 October 2024, ‘ACER calls for improvements in ENTSOs’ 2024 draft TYNDP scenarios to comply with its Framework Guidelines’, https://www.acer.europa.eu/news-and-events/news/acer-calls-improvements…. . ACER also operates the REMIT database, which records financial transactions in wholesale energy markets and is intended to protect energy markets from abuse ¹² See ACER, ‘About REMIT’, undated, https://www.acer.europa.eu/remit/about-remit. .

Industrial associations and private companies

Several industrial associations provide useful sector-specific information. Examples include Gas Infrastructure Europe, which provides information on volumes of stored natural gas, the European Heat Pump Association, which provide the most detailed heat-pump statistics available, and WindEurope and SolarPowerEurope, which provide a wealth of information on the wind and solar sectors, respectively. These associations promote the interests of their members. While certain data is made publicly available, paywalls restrict some information only to members.

Companies also provide data. For example, Enagas reports Spanish natural gas demand on a daily basis ¹³ See Enagas natural gas demand webpage, https://www.enagas.es/en/technical-management-system/energy-data/demand….  and Nord Pool reports regional energy prices ¹⁴ See, for example, Nord Pool day ahead prices, https://data.nordpoolgroup.com/auction/day-ahead/prices. . Companies such as Bloomberg and S&P Global are established to collect, coordinate and profit from data provision. The granularity of available data and high subscription prices that can be charged indicate significant market value.

National agencies

All EU countries have agencies and government departments concerned with energy information. Some make national energy data available in a reasonably granular manner while others do not. Agencies have a range of structures and reporting responsibilities. For example, the Danish Energy Agency operates within the Ministry of Climate, Energy and Utilities, while the German Energy Agency has been established as a for-profit organisation ¹⁵ See Deutsche Energie-Agentur, https://www.dena.de/en/about-dena/. . National regulatory authorities also provide some data through regular market reports.

Non-governmental associations

Think tanks and research institutes help translate complex data sources into digestible formats, making the information accessible to policymakers and citizens. Noteworthy examples include the Ember electricity data dashboard ¹⁶ The Ember electricity dashboard is available at https://ember-energy.org/data/electricity-data-explorer/. and the European liquified national gas (LNG) tracker from the Institute for Energy Economics and Financial Analysis (IEEFA) ¹⁷ The IEEFA European LNG tracker is available at https://ieefa.org/european-lng-tracker-september-2024-update. .

Substantial contributions are also made by the open-source modelling community. Such initiatives pull together datasets on the generation, transmission and consumption of energy in Europe to develop models from which clear insights can be derived. A good example is the PyPSA-Eur initiative which is an open-source model of the European energy system ¹⁸ A description of the model and the underlying code and databases are available at https://github.com/PyPSA/pypsa-eur. . The European Commission has used the model as tool for certain analyses (see for example Thomassen et al, 2024).

The International Energy Agency

Like the EIA, the International Energy Agency (IEA) was established in 1974 in response to the first oil crisis. Its mandate was to ensure security of oil supply for OECD countries. This brought responsibility for coordination planning among its member states (which was done through the creation of a coordinated system of emergency oil stocks) and data provision to better understand global energy markets. Over time, the IEA has expanded its scope to a broader focus on security of energy supply beyond oil and – since 2015 – a strong focus on the global energy transition. Membership has also broadened over time, and it has been paralleled with a set of ‘association countries’, including Brazil, China, India and South Africa.

The IEA is an unmatched reference source for European policymakers on the state of international energy markets, global energy scenarios and international energy policy comparisons ¹⁹ This also allows it to be a forum for high-quality technical and political exchanges on energy topics. . But it is not set up to provide data and analysis underpinning EU energy policy. Its governance (a voting system partly based on historic oil consumption gives the US and Japan a strong role), budget ($62 million in 2022 ²⁰ In 2022, the IEA reported its operational budget as $61.8 million; see https://www.iea.org/about/structure.  for all its global activities) and mandate make it unsuited to reliably provide the energy data and modelling needed to underpin the complex EU energy policymaking process, which needs to take into account the peculiarities of each EU country. 

The lack of consistent and timely energy information is a substantial drag on the quality of EU energy policy discussions and decisions. Setting up a public European energy information ‘anchor point’ that is transparent, open and impartial would be a major improvement. It could contribute to: 

• The collection and public provision of energy information;

• Objective and transparent analysis of this information;

• Where appropriate, effective integration into policy processes.

5.1 Provision of energy data

The first job would be the comprehensive provision of high-quality public energy information. This requires the ‘anchor point’ to be given a mandate to collect and coordinate transparent access to energy data, according to European policy priorities. Part of this is delivered already through existing European institutions and agencies, but not sufficiently.

5.2 Modelling

The second job would be to carry out meaningful, relevant and open modelling exercises. Much analysis will continue to be provided by consultants, academia, companies and other interested parties. But certain highly policy-relevant discussions and processes would benefit from a widely-accepted reference point. This would be the case for long-term energy and climate scenarios, energy system planning, network development and energy market design questions.

Developing a transparent, open and reusable European reference model, or perhaps multiple models, would reduce the complexity of public debate, allowing broader stakeholder participation and improving outcomes ²¹ A roadmap for EU institutions to transition from proprietary energy-system models to open-source models has been called for. See Sarah Brown, Elisabeth Cremona and Beatrice Petrovich, ‘Ember’s four key asks for the new EU Energy Commissioner’, Ember, 18 September 2024, https://ember-energy.org/latest-insights/embers-four-key-asks-for-the-n…. . Stakeholders should know and agree in principle to most of the assumptions and dynamics. Important lessons can be learned from the US and the EIA’s NEMS open-source model (section 3).

Modelling exercises will only be accepted as reference points when results are trusted. Developing trust requires sufficient modelling capacities, well-balanced governance structures and a high level of transparency. Particularly important is the ability for others to reproduce historical results produced by models. That is not currently the case with the default PRIMES model used for European Commission impact assessments.

When modelling results are crucial inputs in major policy decisions, the more important the governance of the modelling exercise is. An agent or agency must evaluate relevant policy questions, quality standards and acceptable assumptions and scenarios. Hence, a well-governed stakeholder process is crucial.

Transparency and openness of the model (code), parameters and data used is essential to build lasting trust. Such openness helps both in spotting any inaccuracies and in allowing stakeholders to challenge and productively debate underlying assumptions. Finally, transparency allows proper peer review and ex-post evaluation, and is needed to develop institutional knowledge (ensure independence from individual modellers) to ensure modelling processes improve over time.

5.3 Policy integration

The third task for the ‘anchor point’ would relate to the extent to which information is integrated into policy processes. If information is relevant and trusted, it should feed policy processes by informing stakeholders and fostering a more constructive debate. This should be the primary litmus test for any process to improve European energy information.

In certain cases, it is desirable for information to have a more direct route into policy processes. This is the case when a policy measure is explicitly triggered by the receipt of a particular piece of information. For the EU, an example would be a member country missing a target they agreed to in the Council of the EU. Typically, the European Commission would then initiate some form of proceedings. For example, during the energy crisis, the EU passed minimum national-level natural-gas storage targets. The Commission was responsible for monitoring storage levels with objective criteria, and for warning and engaging with member states that were missing targets ²² Specifically, if a member state fell more than 2 percent behind intermediate filling targets, see European Commission, ‘Questions and Answers on the new EU rules on gas storage’, 23 March 2022, https://ec.europa.eu/commission/presscorner/detail/en/qanda_22_1937. .

Developing these European competences requires an agency or agencies to be given the authority to collect, process and analyse energy information. To be effective, any institution given such authority must operate on the basis of public trust and engagement. This requires a mandate that enshrines principles of transparency, objectivity and political neutrality. Easy access to information is also crucial. A European agency operating in this way might require funding in the range of €100 million per annum ²³ Annual funding of the EIA is around $150 million while annual funding of the European Environment Agency is closer to €55 million. . Beyond these essential ingredients, the exact institutional design presents governance and methodological trade-offs that require political debate and choices.

6.1 National or European level?

The first trade-off relates to the principle of subsidiarity – to what extent should these information services be provided at European level?

A first option is that all operations happen at European level. This would involve issuing a European institution or institutions with the authority to collect and process data from companies and households. The economies of scale associated with data collection, processing and publication ²⁴ Not least from the application of any machine support, including maintaining a website. mean that this is likely to be most efficient. It would reduce bureaucracy and administrative burden by reducing the volume of work replicated across 27 independent data-collection systems. This would entail a similar approach to the US, where the EIA has the mandate to collect data across states. 

However, the political priorities of the European Council have typically been for greater degrees of national control over data, as reflected in the mandates of Eurostat and the European Environment Agency. Both are hub-and-spoke models with the central institution collaborating with national statistical agencies to ensure accuracy and comparability of data across countries. This requires fostering the ability of all national statistical institutes to collect new data in a consistent way.

The need for data intermediation to ensure cross-country comparability means there is an implied lower bound to the degree of Europeanisation. A hybrid solution might therefore be conceived with national and sectoral focal points responsible for collecting data, but coordinated by a European agency. This European agency would be responsible for ensuring comparability and consistency across independent data sources. In practical terms, this would involve the creation of a standardised application programming interface (API) in a similar manner to Google Earth Engine, which offers a standardised API for accessing satellite data from different providers ²⁵ The Google Earth Engine (https://developers.google.com/earth-engine/datasets/) platform offers a standardised method (API) for accessing geospatial data that is collected from different satellites and providers. This makes it much easier for researchers to access data from different sources. .

The European agency could also be responsible for some processing of primary data, for instance by modelling the impacts of measures detailed in National Energy and Climate plans or producing periodic reports dissecting progress towards particular energy targets. This would help translate complex and largely undigestible data sources into policy-helpful information.

6.2 A new agency or expansion of an existing institution?

A second trade-off is whether solving Europe’s energy information problem requires the creation of a new independent agency or can be achieved through expansion of existing institutions. This depends on the scale and scope of information provision.

Replicating the EIA approach would entail the creation of a new independent European energy information authority. This would have broad responsibility for collecting information, including by collaborating with existing institutions, and for transparent analysis of this information. There is no obvious existing European institution that could take on both of these responsibilities comprehensively. Funding of perhaps €100 million per year (see above) would also be far beyond the budget of any current similar EU institution.

If instead, only piecemeal political agreements can be reached to improve collection and analysis of some data, then additional layers and responsibilities could be given to existing institutions. A first principle must be to update all mandates to enshrine the principle of providing information in a clear and timely manner to the public, including through the creation of standardised APIs. 

In a decentralised-agency scenario, a large EU legislative push will be required in the next two or three years to update mandates. Improved data collection could be achieved with a fresh legal mandate for Eurostat. The scale of missing energy data is such that this would likely require a whole new component, for instance ‘energy for 2040’. Fresh data is needed on decarbonisation of the building stock, on the capacity and utilisation of storage and demand side flexibility in electricity grids, on renewable curtailment, on industrial energy demand composition, on clean technology manufacturing and availability and more.

This would require EU-level decisions on precisely what data should be collected and the passing of more regulations. The Council of the EU might issue the European Commission with a mandate to evaluate and propose a comprehensive new data package. Eurostat would then take on these extra competences and discuss with national statistical agencies methodologies for collecting the data.

ACER might receive additional funding and an enhanced and expanded mandate to monitor security of supply across electricity and gas markets. Another institution would be given a mandate to develop and maintain an open-source, state-of-the-art reference model for the European energy system. Some form of institution would still be required to ensure transparent and easy access to public data, especially the creation of a user-friendly website. 

The attractiveness of the decentralised agency scenario is that it can work through existing legal and political structures. The downsides are that it would be slow and bureaucratic and would likely have to be reviewed in five or ten years as the scope and nature of the required energy data evolves.

6.3 Who decides what information to collect?

The EIA has a broad mandate to provide energy information but retains significant autonomy over its work programme and in deciding what data to collect. The institution most knowledgeable about energy information is thus able to adjust its approach in a flexible manner with the goal of keeping the US government best informed. The risk with such an approach is that it is subject to ideological capture by the agency. Checks-and-balances to ensure agency objectivity and political and ideological neutrality are important.

The current approach in Europe is different. Institutions that collect information have little-to-no autonomy to pursue new sources of information, having instead their mandates detailed in politically-negotiated regulations. The approach ensures that all EU countries have absolute sign-off on any new data provision, while the downside is that the process for obtaining new information is slow and cumbersome. For our decentralised agency scenario, this is a major drawback. A bureaucratic policy process at EU level would be required to determine which new data should be collected, and this process would have to be repeated often as the available and necessary data points evolve.

In a new independent agency scenario, an important question would be the degree to which the agency is granted the autonomy to interpret its information provision mandate.

6.4 What type of data should be collected? 

Data provision involves a trade-off between speed and precision. A strong focus on quality requires ensuring that information is as close as possible to perfect before publishing. This reduces the need for revisions but limits the ability to inform people and policy in a timely fashion. Principles exist for publishing data quickly, with attached disclaimers that it is subject to revision. This is a practice employed by, for example, ENTSO-G, the European Environment Agency and the National Grid in the United Kingdom. This trade-off might be conceived of as publishing ‘amber’ data when it is almost but not quite ready, versus waiting to publish perfect ‘green’ data. There is utility in both approaches.

A second question is the extent to which new information requires the establishment of new primary data sources, versus adding layers of analysis to data that is available currently but is uncoordinated. New primary data sources require granting of a legal mandate to an institution to question companies and households on their energy activities. This provides a rich new source of data but faces administrative and political hurdles.

Alternatively, new and compelling information can be produced by reshaping existing data. For example, the JRC’s EDGAR database (Emissions Database for Global Atmospheric Research ²⁶ Available at https://edgar.jrc.ec.europa.eu/. ) provides a granular overview of global emissions by compiling data from a range of sources (national inventories, industrial processes and satellite data). Zazzera et al (2025) combined multiple sources to provide detailed data on energy-intensive industries in Italy, with such information rarely available through open sources. The Ember electricity data dashboard compiles data from most countries into one consistent and comparable dataset. Proprietary information can also be bundled, as with the European LNG index by ACER based on market data. 

Ultimately it is likely that a successful European energy information strategy would be fed with both fresh primary data and analysis of existing data. The rapidly growing volume of available data suggests that being open to bottom-up data sources is important. For example, satellite data continues to improve in accuracy, while large language models continue to increase the depth and granularity of information that can be extracted from large text corpuses.

6.5 The involvement of non-EU countries 

We have focussed on the challenge and potential solutions from an EU perspective – the impetus for such an initiative is most likely to come from within the EU. However, neighbouring countries (including the UK, Norway and Ukraine) play pivotal roles in the European energy system. Openness to a wider membership or an alternative form of participation for third countries is therefore important. 

The European Environment Agency, for instance, includes members from the European Economic Area (Norway, Iceland and Liechtenstein as well as EU members). Switzerland and Turkey also participate.

In 2022, the EU faced a crisis over its ability to manage without Russian natural gas. The quality and depth of political discussion aimed at resolving this was undermined by a lack of clear information on physical and financial natural gas flows. The EU faces the challenge of transitioning to a climate-neutral energy system and doing so at an affordable and competitive cost. The challenge is made harder by growing international economic competition and fears of trade wars.

To confront and manage these challenges, public and private participants in the European energy system need clearer information. First and foremost, this requires creating standardised and easy access to the large quantity of European energy information that already exists but is scattered across multiple providers and online systems.

As the energy transition progresses, information requirements will evolve. For example, information on electricity demand flexibility and elasticities, such as the willingness of industrial demand or electric vehicle charging to shift throughout the day, are crucial for managing a cost-efficient electricity system. Europe needs a clearer institutional roadmap for identifying and swiftly investigating new energy-information sources when they become crucial for political discourse. This is not a recipe for greater bureaucracy, but a tool for reducing the complexity of political discourse and potential for lobbying.

Finally, much primary information is complex and requires transformation before it can effectively feed political conversation. Any such transformation must be performed as objectively and transparently as possible. To the greatest degree possible, it should be possible for external stakeholders to copy and validate such procedures. An objective and transparent institution is required for such a role. If done correctly, this could help establish effective ‘anchor points’ for political discussions.

Without adequate energy information, European energy policymaking will continue to be less efficient than it could be. The economic costs of better collection and coordination of European energy information are not large and under reasonable assumptions will be substantially outweighed by the benefits. A reform of European energy information can be designed to harmonise, minimise and in certain cases reduce the regulatory burden.

Exactly how to solve the energy-information challenge is a political question with many trade-offs. Particularly important are decisions over delegating responsibilities between the European and national levels, and whether expanding existing institutions will be enough or if a new independent institution, echoing the US EIA, is required. These trade-offs should be evaluated against the primary objective of establishing an accessible, transparent and trusted information point for European energy discussions.

DG ENER (2023) Quarterly report: On European gas markets, volume 17 (issue 1, covering first quarter of 2024), European Commission Directorate-General for Energy, Market Observatory for Energy, available at https://energy.ec.europa.eu/document/download/c26261bd-aa50-462b-912a-9c4d2a712ae4_en

E3 Modelling (2018) Primes Model – Version 2018, Detailed model description, available at https://e3modelling.com/wp-content/uploads/2018/10/The-PRIMES-MODEL-2018.pdf

European Commission (2024) ‘Report on the Review of the Regulation on the Governance of the Energy Union and Climate Action’, COM(2024) 550 final, available at: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52024DC0550

Hirth, L., J, Mühlenpfordt and M. Bulkeley (2018) ‘The ENTSO-E Transparency Platform – A review of Europe’s most ambitious electricity data platform’, Applied Energy 225: 1054-1067, available at https://doi.org/10.1016/j.apenergy.2018.04.048

Jevnaker, T. and B. Saerbeck (2019) ‘EU Agencies and the Energy Union: Providing Useful Information to the Commission?’ Politics and Governance 7(1): 60-69, available at https://www.cogitatiopress.com/politicsandgovernance/article/view/1781/1053

Keliauskaitė, U., B. McWilliams, G. Sgaravatti and S. Tagliapietra (2024) ‘How to finance the European Union’s building decarbonisation plan’, Policy Brief 12/2024, Bruegel, available at https://www.bruegel.org/sites/default/files/2024-10/PB%2012%202024_1.pdf

Thomassen, G., A, Fuhrmanek, R. Cadenovic, D. Pozo Camara and S. Vitiello (2024) Redispatch and Congestion Management, Publications Office of the European Union, available at https://dx.doi.org/10.2760/853898

Zazzera, E.B., M.G. Prina, R. Marchetti, S. Misconel, G. Manzolini and W. Sparber (2025) ‘Bridging the industrial data gap: Top-down approach from national statistics to site-level energy consumption data’, Data in Brief 59: 111365, available at https://www.sciencedirect.com/science/article/pii/S2352340925000976