A service of the

Download article as PDF

The European power market is undergoing significant changes. The EU has set an ambitious goal of reducing its greenhouse gas emissions by 40 per cent by 2030. This will require significant investment in renewable energy sources such as wind and photovoltaics as well as measured policies to deal with the fluctuating capacity offered by these renewables. The integration of national power systems into a single European system would provide huge benefits in smoothing such fluctuations, enabling the EU power market to further increase its reliance on renewables. Current power generators and government regulators will inevitably face challenges adapting to the new market environment, but experience from other countries and regions could provide useful guidance.

Designing a Market for Low-Carbon Electricity

The EU has set out plans to move to an entirely carbon-free power sector by 2050. Such a technological transformation will require considerable new investment. Part of the existing capacity will have to be replaced with less carbon-intensive and with more “flexible” capacity. This transition is at the heart of ongoing EU discussions on market design, on the reform of the EU Emissions Trading System (ETS) and on the 2030 climate and energy framework. As a result, the market will have to address these policy priorities. Some argue that this will require a regulatory overhaul of the EU’s electricity (and gas) market(s).

This article will test this argument. It will discuss possible shortcomings of the existing market design and examine multiple proposals to reform the market. In particular, the article will draw a distinction between market failures, which need to be tackled through potential policy interventions, and normal market results, which should be accepted even if undesirable for some market participants. For a more detailed analysis, see Genoese and Egenhofer.1

Causes and consequences of recent price declines

From 2008 to 2014, a strong decline in market prices for electricity was observed in many EU member states. Wholesale prices, i.e. the price at power exchanges, dropped by 50 per cent in France and Germany, by 40 per cent in Italy and by 35 per cent in Spain (see Figure 1).

Figure 1
Wholesale power prices in Germany, France, Italy, Spain
Wholesale price (€/MWh)

Sources: EPEX SPOT: Market data, day-ahead auction, www.epexspot.com; OMI-Polo Español: Market results, www.omie.es; Gestore dei Mercati Energetici: Results MGP, www.mercatoelettrico.org.


This development is the result of (i) decreasing coal prices;2 (ii) an oversupply of carbon allowances, resulting in a decrease of carbon allowance prices;3 and (iii) overcapacity of power production plants, which puts downward pressure on wholesale prices. Overcapacity was, in turn, caused by lower-than-expected electricity demand, overinvestments, the injection of new capacity through dedicated policy instruments,4 e.g. for renewable energy, and continuing improvements in the field of coupling national electricity markets.5 While there is general consensus on these three causes, it is more difficult to assess which of them has had the strongest impact on the decline of wholesale prices.

Lower prices for coal and carbon allowances reduce the (variable) production costs of coal-fired power stations. Whether this has an impact on the market price depends on whether coal is the price-setting technology in the market, i.e. whether it is the last unit needed to satisfy demand. The national energy mix has a significant impact on the structure of wholesale prices. Given the significant differences across the EU, it is difficult to draw conclusions valid for all member states. Still, a closer look at the German case reveals some interesting findings.

In 2008 the variable production costs of an average coal station and a brand-new gas unit were quite similar, roughly amounting to €53 per megawatt hour (MWh).6 With market prices that were above this value two-thirds of the time, it is safe to assume that gas was the price-setting technology in most hours. This has changed since then. In 2014 the variable production costs of an average coal-fired unit were roughly €27 per MWh, significantly below those of a brand-new gas station (€41 per MWh). With market prices above €41 per MWh only 23 per cent of the time, it can be concluded that gas was not the price-setting technology in the majority of hours for that year. Thus, the decrease in coal and carbon allowance prices has had an impact on wholesale prices.

In general, varying prices for energy carriers such as gas and coal should be considered a normal market development. Market participants have sufficient possibilities to hedge against the volatility of coal or gas prices.

Variations in the price for carbon allowances should also be seen as a normal element of the EU Emissions Trading System, which is a volume-based instrument. However, many argue that the main cause of the current oversupply of carbon allowances lies in the ETS’s lack of supply-side flexibility. Since the supply of carbon allowances was fixed ex ante, the drop in demand (resulting from the 2008-09 economic crisis) led to a decrease in carbon allowance prices. The impact of deploying renewable energy sources on the demand for carbon allowances has been limited. This is because the ETS cap was designed to be consistent with the planned contribution of renewables in decreasing emissions. Thus, only surpassing the targeted contribution would reduce the demand for carbon allowances, which was not the case as of 2013. Flexibility on the supply side would be needed in order to avoid a carbon lock-in.7 To address this, the EU is very close to adopting the so-called Market Stability Reserve to match the demand variability with a mechanism that should allow for adapting supply to demand, thus allowing volume-based control on the carbon price.

A third cause for the decrease in wholesale prices is overcapacity. The 2008-09 economic crisis was accompanied by an unparalleled drop in electricity demand. In 2008 analysts were expecting an annual growth rate of 1.5 per cent.8 However, at the end of 2012, electricity demand in the EU27 was actually three per cent lower than in 2008. Thus, the divergence between projected and realised values amounted to nine to ten per cent. In absolute terms, this “gap” roughly corresponds to the entire electricity consumption of the United Kingdom in the year 2013. Despite the fact that the EU economy has started to recover (in terms of GDP growth), electricity demand has been rather stagnant over the last three years. Thus, part of the decline appears to be structural, for example as a result of the low economic growth and aggressive energy efficiency policies. A short-term effect of the lower-than-expected demand for electricity is that units with relatively high (variable) production costs are no longer needed to cover demand, thus lowering wholesale prices.

At the same time, we have also observed a massive deployment of renewables, as mandated by the Renewables Directive (2009/28/EC), which set targets for the use of renewable energy in each EU member state. National governments subsequently implemented subsidy systems for renewables in order to ensure that their domestic targets are met. Unless other plants are retired, such an injection of new capacity through dedicated policy instruments has a similar impact on conventional generators as a decline in electricity demand.9 On the one hand, one can argue that the impact of this deployment was largely to be anticipated, at the latest since the adoption of the 2020 climate and energy package in 2009. On the other hand, the power sector is, as of 2013, contributing to the overall 20 per cent target to a greater extent than anticipated in 2010 – counterbalancing the lower-than-expected contribution of renewables in heating, cooling and transport.

Consequences for generators

As a result, compared to 2008, conventional generators sold electricity at a lower price and sold less electricity in 2014. The latter is caused by the contraction in market share. Electricity generated from conventional sources has either been replaced by electricity from renewable sources or is simply not needed anymore because of lower electricity demand.

For generators, selling electricity at a lower wholesale price can – but does not necessarily always – lead to smaller profit margins.10 As outlined in the previous section, lower prices for coal or gas are one possible cause for lower wholesale prices. In this case, lower wholesale prices are accompanied by lower production costs for some generation technologies. The exact impact on a generator’s profits depends on the generator’s individual fuel mix; this should be considered as a normal market risk and consequently does not justify a policy intervention. The situation is different with regard to the price of carbon, however, as the lower price is to a great extent a result of an ETS design that is not fit for its purpose.

A consequence of dedicated support policies for renewables is that these effectively reduce the demand for other sources of power generation. Therefore, the impact on a conventional generator’s profit margin is always negative in the short term. Moreover, some power plants become unprofitable to run and are consequently mothballed or decommissioned, especially where no barriers to market exit exist.11 However, in order for overcapacity to be temporary, excess conventional capacity must be allowed to be retired.

The relevant design question for the short term is whether the current market rules ensure that (1) capacity can be retired, (2) the “right” capacity is retired12 and (3) sufficient capacity stays online. Following the market mechanics and given the current spread between coal and gas prices, as well as present carbon allowance prices, gas-fired power stations are more affected by this development than coal-fired plants. This is a result of the fact that the variable production costs of gas-based stations are higher – despite being less carbon-intensive – than coal. Some market participants consider this an undesirable development, since a thermal generating mix dominated by carbon-intensive fuels is inconsistent with the goal of meeting established policy objectives that extend through 2030 and beyond.

The current blueprint for EU market design: the target model

The “target model” serves as the blueprint for the integration of electricity markets in the EU. At the heart of the current framework is an energy-only market, explicitly remunerating the energy delivered. It is sometimes critically observed that there is no explicit remuneration for being available (or, speaking in economic terms, to cover fixed costs). However, as will be shown later, there is an implicit remuneration for availability through the pricing mechanism.

A central example of such an energy-only market in a liberalised power system is the so-called day-ahead market, an auction which is held the day before physical delivery. In this auction, the intersection of demand bids and supply offers reveals a uniform market clearing price, meaning that each successful supply offer is rewarded with the same price, irrespective of its offering price. In a competitive market, this offering price is linked to the unit’s variable production costs and therefore differs from technology to technology due to different fuel costs, carbon intensities and fuel conversion efficiencies. Consequently, availability is remunerated implicitly in those cases where a unit’s variable production costs are below the market clearing price. This differential is typically referred to as gross margin (see Figure 2). In economic terms, it is needed to cover fixed operation and maintenance (O&M) costs, as well as to recover investment costs.

Figure 2
Simplified cost-production curve

Source: Authors’ illustration.

When this margin is not sufficient to cover fixed O&M costs, production units are likely to be retired, either temporarily (i.e. mothballed) or permanently, from the market. Retiring a plant before the end of its economic lifetime or, more generally, before it can generate the expected return on investment makes it a stranded asset.

The current EU framework foresees various other markets to be implemented by member states, namely (i) an intraday market, (ii) a balancing power market and (iii) a forward market. The first two represent short-term marketplaces used to balance consumption and generation closer to real time, which is crucial for an electricity system, because an imbalance can lead to a blackout. Forward markets offer the possibility to trade long-term contracts for physical delivery or financial hedging against the prices of short-term markets. Currently, the commitment periods available for such contracts seldom go far beyond one year, and the liquidity of forward contracts with a delivery date of more than three years in the future is negligible.

The framework also foresees that national markets will gradually be coupled. This remains an ongoing process for the intra-day and balancing market but has mostly been completed for the day-ahead market.

Investments in energy-only markets

Energy-only markets attract investments in new capacity in a number of ways, both through the direct and indirect reliance on price signals from energy and balancing markets. Indirect reliance refers to market participants entering into commercial arrangements with each other in order to hedge their exposure to the price and volume risk.13 Historically, unexpected policy interventions, mistaken demand expectations, and long lead times for planning and constructing new capacity have led to boom and bust cycles, i.e. times of overcapacity alternating with times of scarce capacity.

Critical challenges

Today, the full costs of low-carbon technologies are above wholesale market prices. This raises two questions. The first question is how low-carbon investments are going to be triggered in the future. Secondly, even if this gap between full costs and market prices is closed – as a result of a further decrease of technology costs or an increase in carbon, coal and gas prices – the question of efficient financing remains. Some argue that the cost structure of some low-carbon technologies requires a different investment trigger than the wholesale price. Technologies such as wind and solar have high upfront costs and close-to-zero variable production costs. This means that capital costs have a stronger impact on the total costs of such technologies than they have on coal or gas. At the same time, there are two side effects of using dedicated support policies. First, when the market is already well supplied and demand is not growing, they add to surplus capacity, reducing the demand for electricity generated from existing conventional sources. Second, there is more fluctuation of demand for electricity from conventional sources, because renewable generation depends to some extent on weather conditions.

The decreasing hours of operation of conventional power plants creates a need for a different mix of conventional generation technologies, as these differ not only in their variable production costs but also in their fixed and investment costs. So-called “base-load” capacity is used to cover the minimum continuous level of electricity demand, as it has relatively low variable costs but high fixed and investment costs. Consequently, when conventional power plants responsible for base-load capacity reduce their hours of operation, some of this will be replaced with capacity that has lower fixed costs (“mid-merit” and “peak-load” capacity).

The business environment for peak-load capacity is generally considered challenging, because investment costs have to be recovered from a low number of hours of operation. With the increasing reliance on renewables, (i) more peak-load units will be needed, and (ii) the exact amount required will be subject to greater uncertainty due to the weather-dependent availability of renewables. Moreover, in the competition for the remaining market share of mid-merit technologies, the more flexible sources with their higher production costs are currently losing out, and the economic argument for them is becoming more challenging.


Market participants generally agree that there is a lack of proper implementation of the existing framework. To address this, the following steps will be necessary:

  • fully and properly implement the current market design (standardisation of products, harmonisation and relaxation of price caps, improved price formation in energy and balancing markets, coupling of intraday and balancing markets);
  • improve the functioning of short-term markets to allow for a level playing field with regard to demand, conventional supply and renewable supply;
  • expose every generator to the same obligations and risks;
  • strengthen the EU ETS in order to provide more predictable long-term signals as well as a market-exit signal for carbon-intensive capacity (although different views exist on how to best achieve this);
  • remove market-exit barriers.

These measures will address the existing shortcomings to an extent. Coupling markets and harmonising price caps will lead to the better use of cross-border resources, thus increasing market efficiency. Removing price distortions, exposing all market participants to the same risks and removing market-exit barriers will improve price formation in energy and balancing markets. In order for these measures to be effective, however, market rules will need to be set – and implemented – at an EU-wide or at least regional level. Market-driven investments require a stable and predictable long-term framework. The EU challenge will be to provide confidence that these rules are in fact stable and that market outcomes, especially in terms of wholesale prices and security of supply levels, will not eventually trigger a public intervention. In order to achieve this, it is important that – irrespective of which approach is chosen – this framework is based on evidence and, ideally, enjoys the support of as broad a group of stakeholders as possible.

Important open questions

There is typically some controversy over two critical points: i) how to treat overcapacity and ii) the potential need for capacity mechanisms – i.e. an explicit remuneration for being available – to support investments.

These controversies are central to the debate of the market design fit for the transition to a low-carbon economy, i.e. a stable and predictable long-term framework conducive to investment. On the first point, overcapacity, some market participants argue that policies are needed to accelerate the retirement of capacity, mainly base-load capacity. Others hold that the market eventually will achieve this on its own, provided there are no exit barriers. Similarly, there is no agreement on the second point – the need for capacity mechanisms, i.e. explicitly remunerating availability or the delivery of energy in times of system stress.

From the perspective of the EU’s internal market, however, there is political urgency to settle these two controversies. Some member states have started putting national measures in place, for example in the fields of capacity markets, long-term contracts or approaches to address overcapacity. At this point, it is essential to understand how dissimilar national choices can co-exist and what level of standardisation and harmonisation is required. Otherwise, the likelihood of market fragmentation increases.

  • 1 F. Genoese, C. Egenhofer: Reforming the market design of EU electricity markets – addressing the challenges of a low-carbon power sector, CEPS Task Force Report, July 2015.
  • 2 The German import border price for hard coal can be used as an indicator for this decline. From 2008 to 2014, it decreased by 35 per cent (from €14 per megawatt hour thermal to €9 per megawatt hour thermal). A major cause of this price drop is the reduced demand for coal in the US power sector, where coal was mostly replaced with less expensive unconventional gas.
  • 3 The average price for carbon allowances was €6 per tonne in 2014, down from roughly €23 per tonne in 2008 (-73 per cent).
  • 4 This is sometimes referred to as “merit-order effect”.
  • 5 Market coupling leads to the more efficient use of cross-border resources. Provided that there is sufficient interconnection and that demand peaks do not occur simultaneously, this can put downward pressure on prices.
  • 6 Underlying assumptions: (i) conversion efficiencies of 41 per cent for coal (which corresponds to the mean value of a German plant) and 60 per cent for gas (which corresponds to a brand-new combined cycle gas turbine station); (ii) energy carrier prices based on German import border prices.
  • 7 In the absence of a clear economic signal for decarbonisation, this term refers to a possible scenario of overinvesting in carbon-intensive technologies in the short term. Given the rather long economic lifetime of assets in the power sector, this scenario entails relatively high emissions in the medium term (i.e. a carbon lock-in) unless the installations remain idle or are decommissioned before their lifetime is exhausted, which is considered a waste of economic resources.
  • 8 P. Capros, L. Mantzos, V. Papandreou, N. Tasios: EU-27 Energy Baseline Scenario to 2030. Update 2007, report for the Directorate-General for Energy and Transport of the European Commission, 2008.
  • 9 The same considerations can be applied to the UK announcement of financing the deployment of nuclear power using a dedicated policy instrument.
  • 10 We use the term “profit margin” as a synonym for “gross margin”, which is defined as the differential between the market price and the variable production costs of a power plant. Thus, it is used to cover fixed maintenance costs and recover investment costs.
  • 11 In some EU member states, regulators need to approve the closure (and even mothballing) of power plants. These market-exit restrictions are usually justified on the grounds of these power plants being needed for system stability or for security reasons.
  • 12 In order to restore market equilibrium, it is essential that there is a proper mix of base-load, mid-merit and peak-load capacity. These technologies differ in their ratios between fixed and variable costs.
  • 13 See also P. Joskow: Competitive Electricity Markets and Investments in New Generating Capacity, in: D. Helm (ed.): The New Energy Paradigm, Oxford 2007, Oxford University Press.

Power Market Design: Lessons from Experience

The dialogue around EU power market design seems lately to have taken on increased urgency around sound bites like “missing money”, “keeping the lights on” and “generation adequacy”. The larger context of concerns about security of supply, fueled by cyber-threats and rising tensions in Europe’s relationship with its largest natural gas supplier, has colored the discussion about electricity market design in ways that are not always well grounded in reality.

These concerns become entangled in the discussion about the consequences an increasing share of variable renewables has had and should have for the design of wholesale power markets. The result is often a confused knot of competing policy prescriptions that can seem driven more by vested interests than by a sound grasp of theory or by the available evidence.

Some of the evidence available to inform this discussion comes from experience with these issues in markets outside of the EU, including markets in North America and Australia. This paper will review some of the more salient experience gained in those markets that may be relevant to the European discussion.

One country, many markets

There is no one U.S. wholesale power market. Only about two-thirds of North American electricity customers are in restructured markets, with the remaining third being served by traditional vertically integrated monopoly utilities. Within the restructured segment, there are seven different markets, each with its own market design and regulatory environment. Three of these operate within a single state (California, Texas and New York), while the other four operate across multiple states and Canadian provinces. (There are two additional single-jurisdiction restructured markets in the Canadian provinces of Alberta and Ontario.)

As a result, there is a breadth of experience with approaches to critical market design challenges. For instance, while U.S. markets are often cited as examples of the need for forward capacity markets, in fact only two of the seven markets employ anything of the kind, while two have no capacity market at all and the others employ very limited short-term reliability mechanisms.

Australia offers a smaller set of markets to study but a similarly differentiated body of experience. For instance, the National Energy Market covering the eastern third of the country has maintained an “energy only” market, while the smaller South West Interconnected System covering the western third of the country employs a capacity mechanism.

In other words, there is a greater variety of experiences in these markets than is often appreciated. This offers us ample possibility to learn from them, particularly with regard to the hot-button issue of supporting needed investment.


Much of the debate about wholesale power market design boils down to whether or not the market drives not only efficient allocation in the very short term but also the longer-term investment needed to meet the demand for security of supply at a reasonable cost. It is impossible to answer this question objectively without a sound approach to determining how much of what type of investment is needed or wanted. This is often referred to as “generation adequacy” or “resource adequacy”.

The simplest and clearest lesson has been that the amount of investment required to meet reliability expectations is significantly reduced by consolidating the balancing responsibility under a single system operator (SO) over the largest possible system footprint.1 The simple fact is that what constitutes an “adequate” level of investment on a regional basis can be quite a bit less than would be the case on a state-by-state basis. Much the same result has been achieved by virtually consolidating the operation of multiple interconnected balancing authorities, as has been demonstrated in the Nordic system and as is currently being implemented in the Western region of North America.

Industry practice concerning what is “adequate” has evolved in different ways in different places but in the great majority of cases is based on conventions with obscure origins and ambiguous or non-existent economic rationales. In the course of evaluating current power market designs, the U.S. Federal Energy Regulatory Commission (FERC) and several of the individual market operators have recently engaged outside experts in detailed reviews of the current state of practice.2 One conclusion reached is that the marginal cost of prevailing resource adequacy standards as currently applied exceeds, in some cases by orders of magnitude, the economic value customers place on avoiding interruptions of service. In Texas, for example, a reasonable interpretation of the standard is that a consumer should expect about 20 seconds of service interruptions in an average year due to resource adequacy issues, as compared to historical rates of 100-300 minutes in an average year from transmission and distribution system failures. Where adequacy standards have been adopted in Europe, they turn out to be very similar in the metrics chosen and the level of performance mandated.

Leaving aside the question of whether or not such stringent resource adequacy standards are useful or appropriate, one consequence of retaining them in a restructured market environment is that an efficient “simple” energy market – one that excludes the value implications of balancing services demanded by the system operator – would not produce the level of investment required to meet them. This is true even if the energy market were to accurately reflect the value consumers place on uninterrupted service for each of their energy needs, since the marginal cost of maintaining the level of resource investment required to meet these standards often far exceeds its value to consumers.3 I will come back shortly to a brief discussion of how U.S. SOs have experimented with a combination of different approaches to solving this problem.

Resource capabilities

North American markets that utilize capacity markets have learned from experience that adequacy cannot be established at lowest cost simply on the basis of the quantity of capacity investment, even if capacity is measured on a de-rated basis.4 ISO New England in particular has seen an increase in system stress events in recent years, despite operating the most robust of the U.S. capacity markets and carrying a reserve margin substantially above the reference level. All three of the Eastern independent system operators (ISOs) were tested by two severe winter storm events in the winter of 2013-2014, even though all of them had ample capacity relative to their reference reserve margins. How much capacity is “adequate” depends on the operational capabilities of capacity resources during the full range of expected system conditions.

Each of the markets has taken a somewhat different approach to driving more value to the more flexible and responsive resources in their portfolios. ISO New England has focused on their capacity market for now, introducing a bonus-penalty mechanism that applies to performance during designated system stress events regardless of the season. While this is expected to improve readiness to continue operating under peak demand events, it does not necessarily reward flexibility – it simply rewards availability. ISO New England has identified greater flexibility as an issue to be dealt with in the future.

PJM Interconnection, a regional transmission organization (RTO) in the eastern U.S., initially proposed separating bidders in its capacity market into sub-categories, with more flexible resources being given preferential consideration. After encountering resistance from stakeholders, PJM adopted a bonus-penalty structure similar to that adopted by ISO New England, intended to deal specifically with the widespread fossil plant failures experienced during the winter events. PJM, more so than ISO New England, has also adopted a number of measures in recent years to improve the way the energy and ancillary services markets reward more flexible resources, including improved participation by demand-side resources in setting energy market prices.

NYISO has taken a different tack, leaving their capacity mechanism intact for the moment and focusing on measures to improve the transparency of shortage pricing in the energy and services markets. NYISO has generally taken the approach of maximizing reliance on energy and services market pricing and minimizing the importance of the capacity mechanism in recognition of the need to better differentiate the value of the capacity resources in its portfolio.

Remunerating investment

One thing needs to be made clear before delving into experience with different approaches to the way power markets remunerate investment: “Energy only” power markets do not assume that long-term investments will be made and sustained solely on the basis of selling kilowatt-hours of energy into the short-term energy and balancing services markets, as is often asserted. Nor do they assume that the market price for electricity is or should always be equal to the production cost of the marginal generator on the system, as is also often asserted. As with any other commodity market, they assume that the market price will reflect the balance between supply and demand, dipping below the marginal cost of production during periods of surplus and rising above the marginal cost of production during periods of scarcity. Market participants (generators and suppliers) exposed to market price and volume risk will hedge that exposure by entering into a variety of short- and long-term undertakings, and it is these market undertakings that, in an energy-only market, provide the basis for long-term resource investments.

With that in mind, North American and Australian markets have accumulated experience with two basic alternatives for delivering investment, in many but not all cases combining aspects of both approaches in their market designs (so-called “hybrid markets”). The first approach is to drive energy market prices to better reflect surplus and scarcity in supply, thus sharpening the exposure of market participants to price and volume risk. The second approach is to supplement the energy market with some form of non-energy-based mechanism for being paid to maintain the capacity to produce energy (or reduce demand) as needed (often referred to as a “capacity mechanism”). Approaches differ on how far forward in time these commitments begin and for how long they last.

Within the restructured U.S. markets, the range of approaches is bounded by ISO New England at one end – with a centralized, mandatory forward capacity market that has until recently offered new resource commitment periods of up to five years5 – and the Electric Reliability Council of Texas (ERCOT) at the other end, which has no forward capacity market. ISO New England leans most strongly of all the U.S. markets on its forward capacity auctions to ensure sufficient investment, while ERCOT relies on the effectiveness of energy and services market pricing. The Southwest Power Pool also has no capacity market.

PJM, the largest and arguably most frequently cited of the U.S. markets, operates a capacity market that offers only one-year commitment periods three years in advance of the delivery year. PJM has also moved in recent years to strengthen the pricing signals in its energy and services markets and tends to rely in more or less equal measure on its forward capacity market and on price signals in its energy and ancillary service markets to sustain needed investment.

The New York ISO operates a type of short-term reliability obligations scheme requiring suppliers to have secured sufficient capacity for each of the winter and summer six-month “capability periods” for the next year through a combination of bilateral contracting, voluntary capability period auctions conducted by NYISO just prior to the start of each capability period, and subsequent monthly “spot” auctions. NYISO has moved strongly in recent years to improve the pricing signals in its energy and services markets, tending toward a greater reliance on energy and services markets pricing to sustain investment than either PJM or ISO New England.

The Midcontinent ISO conducts voluntary capacity auctions. California ISO operates three separate and somewhat conflicting capacity-related procurement processes, including a short-term reliability obligations scheme similar to NYISO and an ad hoc state-backed long-term contracting mechanism.

While there has been some convergence over time, the seven markets still diverge significantly in important design features, and there is not enough space here to delve into each of them in detail.6 It is possible, however, to gain insight into the relative performance of the different approaches by looking at the level of investment sustained in each of them, both in absolute terms and relative to their target reserve margins.

Surveying performance in four representative U.S. markets reveals the following:

  • ISO New England, with its centralized mandatory forward auctions three years in advance and one-to-five-year commitment periods for new resources, has seen new generation investment of approximately 5,100 MW and another 2,100 MW of new demand-side resources since its inception, in a system with peak demand of about 29,000 MW. Reserve margin in 2014 was about 28 percent against a target of about 15 percent.
  • PJM, with centralized mandatory forward auctions three years in advance and one-year commitment periods for all resources (existing and new), has seen new generation investment of approximately 16,000 MW and another 10,000 MW of new demand-side resources since its inception in 2007, in a system with peak demand of approximately 155,000 MW. Reserve margin in 2014 was approximately 27 percent against a target of 16 percent.
  • NYISO, with its short-term (six-month) reliability obligations scheme, has seen 10,411 MW of new generation investment and another 1,189 MW of new demand-side resources since 2000, with the first capability period auction occurring in 2005, in a system with peak demand of approximately 34,000 MW. Reserve margin in 2014 was about 25 percent against a target of about 17 percent.
  • ERCOT, with its energy-only market, has a reserve margin of 15.6 percent in a system with peak demand of about 67,000 MW, with approximately 4,000 MW of firm net capacity additions in the pipeline through 2017; ERCOT forecasts that with known capacity additions and expected load growth they will meet or exceed their target reserve margin of 13.75 percent through at least 2018.

Looking at the two Australian markets from the same perspective:

  • The NEM in Eastern Australia, with its energy-only market, has seen approximately 6,500 MW of new generation investment since 2007 in a system with peak demand of about 36,000 MW. Reserve margin in 2014 was approximately 38 percent.
  • The South West Interconnected System operates a reliability obligations scheme similar to the NYISO, with suppliers obligated to secure sufficient capacity annually to meet peak demand in the coming year. The SWIS has seen new generation investment of approximately 1,500 MW in a system with peak demand of 3,700 MW. Reserve margin in 2014 was approximately 62 percent.

It would be a mistake to read too much into these data given the multitude of contributing factors, but they suggest that markets relying more strongly on capacity mechanisms are not noticeably more successful at driving new investment than markets that rely more strongly, or entirely, on effective energy and ancillary services market pricing.

Some differentiation can be seen in the pattern of reserve margins, where markets with a strong reliance on mandatory forward capacity mechanisms appear to have a tendency to over-procure capacity relative to their established targets, whereas markets relying more strongly on energy and ancillary services pricing appear to attract new investment at a rate closer to that required to maintain their target reserve margins. Other observers have noted this pattern of over-procurement in markets with forward capacity mechanisms, for instance Harvey et al. in their recent analysis of the NYISO capacity mechanism.7

These data suggest the conclusion that driving the right level of investment in wholesale markets is not strongly tied to whether or not the market incorporates a capacity-based mechanism and may be driven much more by how well a given market design philosophy is implemented. Furthermore, the observed pattern of over-procurement in markets with mandatory forward capacity mechanisms suggests that whatever economic efficiency benefits there may be from capacity mechanisms in reducing the cost of capital are negated by the cost to consumers of over-procurement.

North American and Australian markets offer little experience of competitive markets with large fractions of very low marginal cost of production resources, but experience in the Nordic market belies the notion that energy price formation is necesssarily undermined in such markets. In 2013, for example, the Nordic system derived 59 percent if its energy from zero-marginal-cost resources and 82 percent from resources with marginal costs below €10/MWh, yet the average day-ahead wholesale price was €38,10/MWh, actually slightly higher than the average day-ahead wholesale price in Germany, where fossil plants set clearing prices in virtually every hour.

Adapting ancillary services markets and their relationship to energy markets

Ancillary (or balancing) services play an important role in the principle of effective energy market price formation. Several markets are gaining experience with approaches to improving the performance of services markets in driving effective market pricing.

The expression of the value of reliability in energy markets is, or should be, inextricably tied to the role the SO plays in factoring security of supply constraints into the provision of various categories of reserves and other critical services. As demand grows relative to supply (or as supply drops relative to demand), a larger share of the available generation is called upon to produce, eventually competing with the demand for capacity needed by the SO in reserve to satisfy security of supply constraints. To the extent this demand for reserves and other services is satisfied by the SO in ways that obscure their real-time value, energy market prices cannot accurately express shortage conditions.

Several U.S. markets, both those with capacity mechanisms (NYISO, PJM) and without (ERCOT), have in recent years adopted a practice known as co-optimization of ancillary services and energy markets. This ensures that the value of reserves required by the SO to satisfy security constraints is adequately reflected in day-ahead and intra-day energy prices. The system operator determines the level of reserves available relative to requirements and establishes a shadow price or price adder using an administrative price curve. This price curve gradually approaches the full value of lost load as the level of reserves approaches the point at which selective involuntary service interruptions would be required. This can be considered a capacity mechanism, but it is one that tops up rather than substitutes for the expression of full shortage value in the energy market.

As noted above, even as consumers gradually acquire the capability to respond directly or through intermediaries to real-time pricing signals (mitigating the oft-cited “demand-side” market failure), system operators will still play an important role, since most established reliability standards place a higher value on lost load than do consumers. Traditional fixed capacity mechanisms can help to address this extra layer of “generation adequacy”, but because they generally ignore operational capabilities, they are a poor substitute for fully functional energy markets in meeting the underlying consumer demand for reliability. Conversely, a sufficiently robust energy market, in which the SOs apply the higher reliability standard into their balancing services procurement, can address both the consumer demand for reliability and the extra layer of security embedded in many generation adequacy standards. Hybrid markets such as NYISO have been successful in addressing the gap between theory and practice. Based on the available evidence, however, practice should favor increasing reliance on effective energy and ancillary services market pricing, a conclusion that can be deduced from the actions taken by SOs in many of the existing markets.

Locational pricing

Experience in North America has led all of the markets there to evolve toward more granular pricing based on location relative to significant grid constraints, with all now operating nodal markets. In the Nordic market, a similar methodology referred to as “market splitting” has been employed for many years to create pricing zones. Both Australian markets also operate pricing zones, though dispatch is on a nodal basis. Experience with locational marginal pricing in these markets has demonstrated marked improvement, both in the investment incentives for generators and for increased investment in transmission, both of which are becoming significant constraints in the European grid. There is now general agreement that pricing at least based on grid-constrained zones, and ideally based on system nodes, is an important contributor to the accuracy of investment price signals. The same zonal/nodal methodology has also been adopted for all North American capacity mechanisms.

Demand-side resources

A full discussion of experience with demand-side resources in markets is beyond the scope of this paper. It is often noted, however, that a number of North American markets have been successful in promoting the role demand can play in balancing the wholesale market. In the markets with capacity mechanisms, the investment in demand-side resources has overwhelmingly been in traditional forms of demand response, with a limit to the number of calls, the length of operation and the season in which it is called.

As the quantity of demand-side resources has grown, the trend has been to amend markets to favor more flexible and non-seasonal demand resources, something that will become even more important as the fraction of variable renewables in systems grows. As the value of different types of demand response shifts, the importance of participation in energy and ancillary services markets will increase. Since the 2011 PJM auction, when it first began differentiating bids, traditional limited demand response has declined from 90 percent to 25 percent of bids cleared. ERCOT now gets over half of its ten-minute reserves from demand response, and in PJM’s daily auctions for ten-minute reserves, demand response resources constitute nearly a third of what clears.


Years of experience with multiple different approaches to a number of critical market design challenges, particularly in the liberalized markets of North America and Australia, offer a rich pool of information for those considering how to improve Europe’s internal energy market for electricity. This article has considered in particular the lessons learned regarding the assessment of how much investment is needed and how best to remunerate investment in different types of system resources. These lessons obviously must be interpreted in light of the European market’s own unique characteristics and context. It would be a mistake, however, to assume that they have nothing to offer European policy makers as they wrestle with the challenges before them. This article sought to offer an objective window into these markets and provide some insights into what lessons they have to offer.

  • 1 The U.S. ISO/RTOs are required to publish an annual “value proposition” comparing cost and performance on a regional basis with system operation on a state-by-state basis that can be found on their respective websites.
  • 2 See e.g. J.P. Pfeifenberger, K. Spees, K. Carden, N. Wintermantel: Resource Adequacy Requirements: Reliability and Economic Implications, prepared for FERC, September 2013; and S. Newell, K. Spees, J.P. Pfeifenberger, R. Mudge, M. DeLucia, R. Carlton: ERCOT Investment Incentives and Resource Adequacy, prepared for ERCOT, June 2012.
  • 3 See Resource Adequacy in Wholesale Electricity Markets: Principles and Lessons Learned, testimony of D. Patton, NYISO Independent Market Monitor, to FERC Technical Conference, 25 September 2013.
  • 4 The capacity of a system resource is de-rated by different system operators in different ways but generally involves discounting its contribution to resource adequacy by the likelihood, based on past performance of that resource or similar resources, that it will not be available when needed to meet peak demand on the system.
  • 5 ISO New England recently agreed to offer commitment periods of up to seven years to new resources.
  • 6 A good summary of the design features of the various markets is provided in Federal Energy Regulatory Commission: Centralized Capacity Market Design Elements, Commission Staff Report, 23 August 2013.
  • 7 S. Harvey, W. Hogan, S. Pope: Evaluation of the New York Capacity Market, March 2013, which noted at page xii a pattern of “forward planning process-driven inflation of capacity requirements and costs” at PJM and ISO New England since implementation of their forward capacity markets.

The 2030 Power System in Europe: Flexibility Needs, Integration Benefits and Market Design Implications

As part of its strategy to become a low-carbon region, the European Union aims to draw at least 27 per cent of its energy from renewables by 2030. This translates into a share of some 50 per cent in the power sector. Solar photovoltaics (PV) and wind power – driven by significant cost reductions – will contribute to more than half of this share. As wind and solar depend on the weather, future power systems will thus be characterised by fundamentally different generation patterns than those today, significantly increasing the need for flexibility and back-up capacity. In meeting this flexibility challenge, regional cooperation and power system integration offer important ways forward.

This article takes a deeper look at the future of regional market integration for power systems with high shares of wind and solar: What kinds of flexibility requirements arise from the projected growth of these two technologies? To what extent can further power market integration help meet the challenge? And what are the economic implications of power systems with high shares of wind and PV for the market design?

Cost development of renewable energies

The last two decades have seen dramatic cost reductions in onshore wind and solar photovoltaics. The example of solar PV is especially illustrative. Since 1980 the module price for PV dropped on average by 20.9 per cent for every doubling of cumulative produced capacity (see Figure 3). This has led to a continuous reduction in the specific generation costs of PV plants.

Figure 3
Historical price curve of PV modules, 1980-2014

Source: Fraunhofer ISE: Current and Future Cost of Photovoltaics. Long-term Scenarios for Market Development, System Prices and LCOE of Utility-Scale PV Systems, study on behalf of Agora Energiewende, 2015.

Robust evidence indicates that these cost reductions will continue in the future. Even conservative scenarios that assume business-as-usual technological progress, i.e. without any major technological breakthroughs, predict total electricity generation costs for utility-scale PV of four to six ct/kWh by 2025 and two to four ct/kWh by 2050 (see Figure 4), making PV the cheapest power generation source in many regions across the globe.

Figure 4
Cost of utility-scale solar power plants in southern and central Europe

Note: Real values in constant 2014 euros; bandwidths represent different scenarios of market, technology and cost developments, as well as power plant location between southern Germany (1190 kWh/kWp/y) and southern Spain (1680 kWh/kWp/y), assuming five per cent (real) weighted average cost of capital.

Source: Fraunhofer ISE: Current and Future Cost of Photovoltaics. Long-term Scenarios for Market Development, System Prices and LCOE of Utility-Scale PV Systems, study on behalf of Agora Energiewende, 2015.

Figure 5 illustrates that in 2013 large-scale PV and onshore wind were already the lowest-cost decarbonisation options, whereas nuclear power and fossil fuel power plants equipped with carbon capture and storage (CCS) technology were high-cost decarbonisation options.

Figure 5
Average remuneration for new nuclear, PV and wind and LCOE for coal and gas-fired CCS power plants
in €2013/MWh

Note: LCOE = levelised cost of electricity.

Source: Prognos AG: Comparing the Cost of Low-Carbon Technologies: What is the Cheapest Option?, analysis on behalf of Agora Energiewende, 2014.

Renewable energies in the European power system

The European Council agreed in October 2014 that Europe should reduce its greenhouse gas emissions by 2030 by 40 per cent below 1990 levels. As a means to achieve this, the share of renewables in overall energy consumption is to rise to at least 27 per cent by 2030. Due to the aforementioned cost reductions in wind power and PV, European decarbonisation and renewables targets imply that wind power and PV will be the main sources of renewable generation in Europe by 2030. This is also reflected in national energy strategies and official scenarios in line with the 2030 climate and energy targets. The renewables share of the European power system will be some 50 per cent,1 with wind power and PV amounting to some 30 per cent of total generation (see Figure 6).2

As wind power and PV depend on the weather, their power generation fluctuates. Thus, increasing shares of wind power and PV require a profound transformation of European power systems, as flexibility requirements will increase significantly. However, the integration of European power markets would mitigate flexibility needs considerably.3

Figure 6
Breakdown of the European power generation mix in 2030

Source: Fraunhofer IWES: The European Power System in 2030: Flexibility Challenges and Integration Benefits. An Analysis with a Focus on the Pentalateral Energy Forum Region, analysis on behalf of Agora Energiewende, 2015.

The benefits of power system integration

Reduced flexibility requirements

Depending on wind speeds and solar irradiation, the output of wind and solar power plants fluctuates considerably. Output can at times be almost zero and at other times close to the installed capacity. This brings a flexibility challenge to power systems, as the remaining dispatchable technologies have to adapt to the wind and PV-related output changes.

In general, power systems possess various means of addressing this flexibility challenge, e.g. demand-side response (DSR), flexible dispatchable power plants or storage. The most cost-effective flexibility option, however, is probably the geographical enlargement of the system scope, i.e. power system integration. This is because weather patterns are not perfectly correlated across Europe, which yields smoothing effects, especially for wind generation, but also for load.

As Table 1 shows, the correlation of wind power feed-in across neighbouring countries in the Pentalateral Energy Forum (PLEF) region (Germany, France, Belgium, the Netherlands, Luxembourg, Austria and Switzerland) is rarely higher than 50 per cent, which indicates significant potential for maximising the utilisation of wind power when coupling national power systems. Furthermore, load patterns are also not correlated perfectly across countries, though these correlations are typically higher than 50 per cent. In contrast, solar PV feed-ins, unsurprisingly, correlate highly, as PV generation is driven by the daily path of the sun, which obviously affects neighbouring countries in a similar manner.

Table 1
Correlation coefficients between PLEF countries for load, onshore wind and PV generation
AT 100 72 57 82 57 57 74
BE 72 100 63 73 66 57 70
CH 57 63 100 54 73 43 48
DE 82 73 54 100 52 61 77
FR 57 66 73 52 100 43 49
LU 57 57 43 61 43 100 54
NL 74 70 48 77 49 54 100
AT 100 24 45 35 27 29 22
BE 24 100 27 49 55 66 60
CH 45 27 100 28 39 32 22
DE 35 49 28 100 33 47 58
FR 27 55 39 33 100 52 34
LU 29 66 32 47 52 100 44
NL 22 60 22 58 34 44 100
AT 100 82 89 90 83 83 83
BE 82 100 86 88 87 92 94
CH 89 86 100 90 90 87 86
DE 90 88 90 100 86 89 88
FR 83 87 90 86 100 86 86
LU 83 92 87 89 86 100 90
NL 83 94 86 88 86 90 100

Source: Fraunhofer IWES: The European Power System in 2030: Flexibility Challenges and Integration Benefits. An Analysis with a Focus on the Pentalateral Energy Forum Region, analysis on behalf of Agora Energiewende, 2015.

Compared to individual wind turbine generation patterns, European-wide aggregation yields a more stable output with smaller and fewer ramps (i.e. changes in output from one hour to the next). For example, simulations based on the weather patterns observed in 2011 yield a maximum hourly ramp for onshore wind of minus ten per cent of installed capacity in Europe.4 However, if the simulation is based solely on France, for example, it yields a maximum hourly ramp of 21 per cent. Thus, through European power system integration, the residual power plant park in Europe will have to provide a maximum ramp of just ten per cent of installed wind capacity to compensate the largest wind fluctuations, while in a national autarchy case, the French power plant park would have to provide a ramp of 21 per cent of installed wind capacity. Interestingly, in this simulation, European-wide ramps of onshore wind are larger than plus or minus five per cent of the installed wind capacity in only 23 hours of the year.5

In addition to the smoothing effects arising from uncorrelated weather conditions, seasonal weather patterns also yield a more stable total monthly generation of wind power and PV. The reason is straightforward: wind generation is higher in winter months while PV generation is higher in summer months. Thus, the total generation of wind power and PV is rather stable throughout the year. Of course, to benefit from these smoothing effects, strong electricity grids are required – both within and across countries and regions.

Minimised renewables curtailment

Due to geographical smoothing effects, less electricity from fluctuating renewables has to be curtailed or stored when high feed-in situations occur,6 because it can instead be traded among countries. According to the aforementioned simulation, the curtailment occurring in the autarchy scenario would be about ten times higher than in an integrated European power system.7

To summarise, market integration reduces flexibility requirements arising from wind and PV deployment through geographical smoothing and minimises the storage needs (or the curtailment of renewable energy sources). Thus, European market integration increases the value of wind and PV. Clearly, market integration cannot eliminate all flexibility requirements. We therefore now address how conventional power plants will have to adapt to a power system with higher shares of wind and PV.

Consequences for the conventional power generation system

The deployment of wind and PV has consequences for the power plants that cover the remainder of the electricity demand, the so-called “residual power plant park”. Specifically, its structure and operation will have to change in order to react during both shorter and longer time periods.

To illustrate the dynamics in the system, Figure 7 depicts power generation in the PLEF power systems for a summer week with high PV generation in the year 2030. Here, PV production matches the higher power demand during the day. Conversely, when PV generation changes quickly in the morning and evening hours, stored hydropower (in Austria, France and Switzerland) provides the primary compensation for the difference in electricity load and PV output. Conventional thermal and flexible biomass plants in the PLEF region also increase their generation during these hours.

Figure 7
Power generation and demand for the week of June 9-15, 2030, for each PLEF region and in the aggregate

Source: Fraunhofer IWES: The European Power System in 2030: Flexibility Challenges and Integration Benefits. An Analysis with a Focus on the Pentalateral Energy Forum Region, analysis on behalf of Agora Energiewende, 2015.

To assess the consequences of wind and PV deployment for residual power plant parks, we look at the duration curves for load and residual generation.8 These two curves are shown in Figure 8. Duration curves yield the number of hours per year a specific load or generation level is exceeded. The residual generation curve is both steeper and lower than the load curve. Thus, the required number of power plants running “baseload” decreases. With 30 per cent wind and PV in the generation mix,9 the capacity requirement for power plants running more than 7000 hours per year is reduced by 50 per cent.

Figure 8
Load duration curve and duration curve of residual power plant park generation, PLEF region, 2030

Note: The difference between the load duration curve and the duration curve of residual power plant generation is the generation of variable renewables.

Source: Fraunhofer IWES: The European Power System in 2030: Flexibility Challenges and Integration Benefits. An Analysis with a Focus on the Pentalateral Energy Forum Region, analysis on behalf of Agora Energiewende, 2015.

However, for a few peak hours, the difference between the two curves is small. In these hours, conventional power plants and imports will have to cover almost the entire load, irrespective of the amount of installed wind and PV capacities. Thus, the future power plant mix will contain less baseload capacities and relatively more mid-merit and peak load capacities.

The required changes in the structure of the residual power plant park are most pronounced in Germany, as the envisioned share of wind and PV in 2030 is higher compared to the regional average. The impacts on the operational pattern of the residual power plant park that occur with high shares of wind and PV can thus be illustrated by a snapshot of the German power system.10 The German residual power plant park operated in 2013 at load levels between 25 and 75 GW. The maximum hourly output change of the residual power plant park was in the range of +/- 15 GW. For 2030, similar hourly output changes are expected, yet these ramps occur irrespective of the actual generation level of fossil power plants (the latter ranging from five to some 65 GW in 2030). Thus, fossil power plants will have to ramp more often, operate more often in partial load and have to be turned on and off more often.11

To summarise, a more flexible power system is required. The structure and operation of the conventional power plant park will need to change. This implies fewer baseload and relatively more mid-merit and peak-load plants. Besides an adjusted power plant park, additional flexibility options (e.g. DSR, storage) will be required. These options require economic incentives which have to arise from a refined power market design, the key pillars of which we now summarise.

Increasing system flexibility requires a refined power market design

To increase system flexibility, a refined market design is essential. Key elements that need to be considered include the long- and short-term energy markets, renewable energy support schemes, grid planning and operations, and potential additional instruments such as capacity mechanisms. For the design of all of these elements, it is key that the technical flexibility requirements of future power systems are taken into account. The future European market design therefore needs to reflect several key aspects.12

First, an advanced energy-only market should be at the heart of any future power market design. Strong energy price signals, reflecting the real-time value of electricity, are required to manage the flexibility challenge efficiently. Thus, the spot price should serve as a central and undistorted dispatch signal for all market parties. To achieve this, it is crucial to make the short-term energy markets faster (e.g. shorter trading products and reducing gate closure times) and larger (integrating across balancing areas). Further integrating short-term markets across borders as well as vertically linking the different segments (day-ahead, intraday and balancing markets as well as imbalance settlement methodologies) can help to reduce flexibility requirements. This will also allow markets to better reflect the real-time value of energy and balancing resources. Adjusting the design of balancing markets (e.g. shorter contracting periods, modified technical prequalification criteria) will allow new market actors (DSR, storage, renewables) to offer balancing services. It is indeed increasingly appreciated that increasing the flexibility of short-term markets is a no-regret option for all involved countries.13

Second, resource adequacy should be assessed at a regional, cross-border level.14 Regional adequacy assessments lower the costs of achieving a reliable power system and, as has been shown above, mitigate the need for flexibility by considering the benefits from cross-border power flows and smoothing effects. For a given adequacy standard, the quantity of required resources decreases and the options for balancing the system expand as the market size increases. The Pentalateral Energy Forum serves as an important role model. Its publication of the first regional adequacy forecast applies a common methodology from a regional assessment perspective.15

Third, if resource adequacy is addressed through a capacity mechanism, resource capability rather than capacity needs to be the primary focus. Security of supply will increasingly become a dynamic issue. Future capacity mechanisms will need to consider this by focussing not just on capacity in a quantitative sense but also on operational capabilities. As such, they should rather be conceived as capability mechanisms and not capacity mechanisms.16 Then price spillover effects of capacity mechanisms to energy-only markets can be minimised while also fostering greater reliability at lower costs.

Summary, outlook and remaining challenges

This paper points to the increasing flexibility requirements prevailing in future power systems arising from further deployment of wind and PV. By 2030, renewables will be the main source of electricity generation in Europe. Official scenarios in line with the European 2030 targets indicate that wind and PV should make up some 30 per cent of the share in meeting European power demand.

Owing to the natural variability of wind and PV generation, flexibility requirements in the power systems will increase. Importantly, power system and market integration, facilitated by strong domestic and cross-border grids, can mitigate flexibility requirements through smoothing effects and maximise the utilisation of wind and PV.

Yet market integration alone will not completely resolve the flexibility challenge. Thus, a more flexible power system is required. There is a large portfolio of flexibility options:

  • Demand-side management/response: This comprises the modification of consumption patterns (e.g. load reduction in critical hours or load-shifting from peak to off-peak hours).
  • Flexible conventional power generation: Flexible thermal power plants are characterised by short start-up times, high ramping rates and low minimum load levels.
  • Flexible renewable generation: Renewables (including wind and PV) can contribute to ancillary services such as providing balancing energy.
  • Grids, interconnection and market integration: Linking neighbouring power systems enables flexibility through access to a larger resource portfolio and geographical smoothing effects.
  • Storage: Allows surplus power generation (e.g. in times of high output from renewables and low demand) to be stored and subsequently released in times of scarcity.
  • Power-to-heat, power-to-X, renewables curtailment: Once higher shares of renewables have been achieved, moderate curtailment can reduce the flexibility challenge (e.g. compared to expanding the grid to take up renewables generation at all times). Also, surplus generation can be utilised effectively if the electricity sector is more closely linked with the heating sector (power-to-heat) or other sectors (mobility, chemicals, etc.).

An important aspect of the new flexible power system is an adjusted power plant park with less baseload capacity and relatively more mid-merit and peak load capacities and active demand-side participation. The power market design has to provide the incentives for flexibility options to be in position to offer flexibility. Thus, an enabling market design and regulatory framework needs to be in place to facilitate the evolution towards a power system with flexibility at its core.

We have pointed out that a strong, undistorted spot price reflecting the real-time value of electricity is required to manage the complexity efficiently. Such a price signal can serve as an undistorted dispatch signal for all market participants. It requires faster and larger energy and balancing markets; linking day-ahead, intraday and balancing energy markets; and minimising fossil must-run capacities by adjusting balancing energy products and their procurement.

A reformed energy-only market, regionally embedded and integrated, is certainly a no-regret way forward for all involved countries. Yet, it might not be the solution for all challenges. For example, this paper has not touched upon the issue of how to best structure grid tariffs and other surcharges to incentivise market actors to behave in a “system-friendly” manner. The complexity will certainly increase as we move to a more decentralised energy system, with potentially millions of actors. Furthermore, the organisation of cross-border, regional system operation, the cooperation of grid operators and the creation of a regional governance structure are all important and unresolved questions.

When moving into a world of power systems that rely on renewables for half of their power, another key issue arises – how to handle the financing issue in a power system based on capital expenditures (CAPEX). We are certainly moving into a world where investment costs and capital expenditures will dominate the economics of the power system. This concerns CAPEX-intensive renewables, where wind power and PV have essentially zero short-term generation costs, implying low wholesale prices when wind and PV produce, which creates a refinancing challenge. But it also concerns back-up power plants, which will become more CAPEX-intensive as well. As their running hours decrease alongside wind and PV deployment, CAPEX will become more important and, in an energy-only market setting, will have to be earned back in fewer hours as well.

Prospective issues in a CAPEX-intensive power system thus concern questions regarding remuneration and (re-)financing, but also the facilitation of consistency among different technologies in the market design. Long-term contracts, capacity remuneration mechanisms and a new kind of investment market are just some examples of how this could be facilitated, all of which bring about fascinating and important questions for how to best structure the economics of new, renewables-based power systems.

  • 1 European Commission: Impact assessment accompanying the communication: A policy framework for climate and energy in the period from 2020 up to 2030, 2014.
  • 2 Note also that this trend is already taking place: in 2014 almost 74 per cent of all investments in generation capacities in Europe were for wind power and PV. See EWEA: Wind in power, 2014 European statistics, February 2015.
  • 3 The results presented here are taken from Fraunhofer IWES: The European Power System in 2030: Flexibility Challenges and Integration Benefits. An Analysis with a Focus on the Pentalateral Energy Forum Region, analysis on behalf of Agora Energiewende, 2015.
  • 4 The largest ramp based on the weather data for the year 2011 yields a ten per cent output reduction.
  • 5 Weather data from 2011.
  • 6 Curtailment of wind and PV occurs when their feed-in is higher than the prevailing domestic load and cross-border interconnectors are fully utilised. See Fraunhofer IWES, op. cit.
  • 7 This is because of a lack of exchange options with other regions. In the integration scenario, European-wide curtailment (or additional storage needs) amounts to some five TWh, whereas the cumulative national curtailment in the autarchy scenario is some 45 TWh.
  • 8 A duration curve is derived by sorting hourly values for one year from the highest to the lowest value. The generation of the residual power plant park is derived by subtracting variable renewables generation from the load and adding net exports. We subtract variable renewables generation because their short-run generation costs are essentially zero. Thus, in power markets they are dispatched before (residual) thermal generation.
  • 9 The simulations yield a European-wide share of wind and PV of 30 per cent and a share of 34 per cent within the PLEF region.
  • 10 The residual power plant park in the other simulated countries needs to provide somewhat smaller and fewer output changes to the market. For further details, see Fraunhofer IWES, op. cit.
  • 11 When assessing production changes from one day to the next, even larger parts of the residual power plant park will need to be turned on and off more frequently.
  • 12 This section is based on Regulatory Assistance Project: Power Market Operations and System Reliability: A contribution to the market design debate in the Pentalateral Energy Forum, study on behalf of Agora Energiewende, 2014.
  • 13 See Pentalateral Energy Forum: Second Political Declaration of the Pentalateral Energy Forum of 8 June 2015; and 12 Electrical Neighbours: Joint Declaration for Regional Cooperation on Security of Electricity Supply in the Framework of the Internal Energy Market, 2015.
  • 14 Resource adequacy is defined as the investment dimension of power system reliability, the short-term dimension being system security. See Regulatory Assistance Project, op. cit.
  • 15 See Pentalateral Energy Forum Support Group 2: Generation Adequacy Assessment, 2015.
  • 16 For further details, see Regulatory Assistance Project, op. cit.; and Regulatory Assistance Project: What Lies “Beyond Capacity Markets”? Delivering Least-Cost Reliability Under the New Resource Paradigm, 2012.

Future Changes and Challenges for the European Power Market

This paper addresses the changes taking place in the European power sector and the resulting challenges for both regulators and operators. In particular, it aims to draw attention to several important issues which have not been widely discussed, such as (i) allowing power to play its economically correct role in the wider energy transition, and (ii) appropriately allocating risk among different stakeholders and generally keeping risks low to avoid the cost of capital from rising excessively.

The changes taking place

In the over 15 years since the European power markets were liberalised, they have been subject to both the greatest and fastest changes in their history. One political requirement has been layered upon another, with the result that the market scarcely had a chance to catch its breath after one major policy decision before the next one arrived.

The changes expected to take place at the European level have been widely discussed, but it is worthwhile to briefly summarise them here and outline their main implications.1 Figure 9 shows the change in fuel shares of electricity generation between 2012 and 2030 under two alternative scenarios – the International Energy Agency’s “New Policies Scenario” and the more ambitious “450 Scenario”.2

Figure 9
Future shares of power production in Europe
in %

Source: IEA World Outlook, 2014.

Three main points emerge from the chart:

  • There will be an increase in the share of power with almost zero variable costs for all generation types except fossil fuels. Even in 2012 this share was already at the 50 per cent level. In addition, the generating fleet is becoming even more capital-intensive, partly because by their nature renewable energy power stations only produce with relatively low load factors.
  • There will be a rise in the share of renewable energy production, with three important consequences: the system will be even more driven by meteorological conditions than previously; a rising proportion of power will be generated at the distribution system operator level for all renewable energy except large hydro and offshore wind; and new players (including households) rather than the incumbents will be generating a progressively larger share.
  • There will be a phase-down of nuclear power, whether due to ideological reasons or simply through the aging of existing stations and their consequent declining profitability. Conversely, there are attempts being made to build new capacity, although these are struggling at present.

An additional factor with which the industry may have to reckon is declining demand due to strong efficiency initiatives and industrial restructuring, even though these may be partly offset by new applications, e.g. e-mobility and heat pumps.

Challenges for regulators

This section examines several challenges not so frequently discussed and briefly reviews issues of supply security and the integration of intermittent renewable energy, which have been much more widely covered elsewhere.3

Enabling power to play its correct role in the energy transition

One of the first challenges for the regulators looking at the entire canvas of the energy transition is to ensure that power is allowed to make its economically correct contribution to decarbonisation, not only within but also outside its current mainstream applications, for example in transportation and space heating. In this respect, it is important that electricity is not unduly burdened by special taxes and excessive environmental costs. This form of energy is being called upon to carry the lion’s share of renewable targets, and there is an argument that the associated costs should be shared more uniformly across all forms of energy and/or be partly covered by the taxpayer, as is the case in the US.

Achieve the transition at the lowest cost

The energy transition will be very expensive, even in the best circumstances, and therefore it is of paramount importance to plan it in such a way that the costs are kept fully under control. This is probably where the greatest conflicts with other objectives could arise and where market mechanisms may be challenged. There are, broadly speaking, two primary components to this challenge:

  • Gain the maximum mileage from existing stations. Given that the costs for such stations are already sunk, the first consideration should be to give such stations the maximum opportunity to contribute in accordance with headline environmental criteria. They should not have to bear inappropriate environmental burdens that will not contribute to the overall European CO2 emissions reduction target or – if the stations’ utilisation will in any case be very low – will not make a material contribution to other forms of emissions.
  • Strike a careful balance between competition generally and the overall level of risks. This will keep the cost of capital (the new “fuel”) as low as possible. Experience over large periods of time and across many industry sectors speaks favourably of the societal benefits of competition. Therefore, it is correct to expose as much of the supply chain as possible to competition, but it is also important to keep in mind the associated risks, since they will determine the cost of capital, which will be such an important component in the new highly capital-intensive power world.

Appropriate risk allocation

It is remarkable how little attention has been given to the overall issue of appropriate risk allocation among different stakeholders. The risk allocation prevailing in European power systems is nothing if not arbitrary and can be summarised as follows:

  • Following market liberalisation starting in 1998, all of the risks of operating thermal power stations have been passed on to the owners.
  • The risk of renewable energy power stations (other than construction) is borne to a large extent by customers or in some countries also by the state.
  • The risks of the distribution and transportation networks (natural monopolies) are borne principally by customers but are reduced by incentive regulation.

Now, with the combination of prospectively declining demand, increasing self-production and the rapidly evolving technology, the power industry is moving into unchartered territory with respect to risk. Therefore, risk allocation should be addressed much more consciously.

Redesigning tariff structures

The current consumer tariffs, with their high emphasis on the variable component and very little emphasis on the fixed element, are a legacy from the past, where the variable cost of supply (i.e. coal) represented a higher proportion of the total costs than it does today. Now that we are moving to a fixed-cost world and need to keep as much market as possible in the power system, it is important that there should be some move towards a more cost-reflective price structure to avoid market distortions, such as the artificial incentives for household photovoltaic installation.

Moving towards a more appropriate price structure is no easy task, mainly because of the social consequences which would result – the fixed costs would weigh more heavily on those who consume less, so their bills would increase disproportionately. However, countries’ tax systems could be utilised to mitigate this concern.

Integrate intermittent renewable energy and ensure supply security

These challenges have been widely discussed elsewhere,4 and they rely upon a wide number of available instruments, such as flexibility services, demand-side response, and capacity markets being made available and incentivised to provide flexibility. The thermal power stations are an important source of such flexibility, and certain investments are being made to improve their contribution.

As has been argued elsewhere,5 it is most unlikely politically that the wholesale price for electricity would be allowed to rise to the levels required for new zero-carbon station investment. Moreover, there is the question as to whether it would manage to support investment in peaking stations or even to keep required existing stations on stream. For this reason, some instrument other than the wholesale market will very probably be required in the course of time to meet the required supply quality.

Challenges for operators

Conventional generators

Very little attention has been given to the consequences of market liberalisation since 1998 and, in particular, how it shifted the risk allocation. In a nutshell, whereas previously almost all the risks of different kinds were carried by the customers (or even by the state), the three EU Energy Packages have shifted the conventional energy generation risks completely onto the power companies.

Looking to the future, apart from selected investments in peaking stations, few, if any, new conventional stations will be built, so the challenges relate to making the correct future decisions about the existing fleet. In this respect, there are two categories of challenges:

  • to improve the profitability of existing stations through further cost reductions (e.g. manpower) and revenue optimisation between the wholesale and ancillary services markets;
  • to make decisions about upgrading, mothballing, closures and investment that are focussed on improving efficiency or the station flexibility (ramping rates and minimum load).

Renewable generators

A stable regulatory framework is the most important requirement and has four aspects:

  • the remuneration system must be clear and not adjusted retroactively;
  • as direct marketing is now the rule, rather than relying purely on feed-in tariffs, it is important that operators have a good basis for forecasting the wholesale price;
  • there must be certainty that the stations can be connected to the grid at the agreed point in time;
  • when technology-specific targets are set (e.g. for offshore capacity), these should not be subsequently modified, since this then poses problems for the entire supply chain in providing the necessary components and services.

A new challenge with which the industry is now confronted following the EU State Aid Guidelines of April 2014 relates to the competitive tendering procedure, which is clearly an important means of introducing (ex ante) competition into the market, since with almost zero variable production costs, the ex post scope is very limited.

The design of the auctions themselves is a challenge both for the regulators and the participants, since different designs can determine who is willing to take part and therefore ultimately the market premium awarded. One of the key problems of auctions is the so-called winner’s curse: a market participant’s winning bid is below their actual costs, and consequently they take a loss on the project. This risk needs to be minimised by introducing an intelligent combination of prequalification rules and financial securities and penalties.

The further growth of renewable energy also needs to address problems with local resistance to large projects in their communities. In order to overcome resistance on renewable energy projects, information must be given, and locals should ideally be allowed to participate in the economic benefits of renewable energy, e.g. by including a measure of local content.

With the growing market share of renewable energy, the market integration and minimisation of system costs becomes more important: renewable energy needs to be integrated into the spot and intraday markets and might also participate in the ancillary services markets. In case of zero hourly spot prices, renewable energy should be incentivised through a suitable compensation system to be shut down to reduce the amount of negative prices, which otherwise increase risks for future renewable investments. The more renewable energy is exposed to market risk, the more it becomes necessary to limit such risks so that risk premia for investment do not rise.

Grid constraints could also endanger the optimal use of renewable energy; hence, grid access needs to be provided as soon as possible. Grid constraints at all voltage levels need to be reduced to an optimal level to achieve the right balance between their costs and renewable energy curtailment. Also, stronger interconnection between regions or countries can reduce the wind integration cost, as there is less correlation among wind power generation sites when these sites are located further away from each other.

With increasing pressure from auctioning and market price risks, renewable energy operators have to seek competitive advantages from economies of scope and scale in operating sufficiently large portfolios, negotiating optimal supply contracts with original equipment manufacturers, optimising their operation and maintenance strategies, and their asset management. Furthermore, in the auctioning process, gaining access to the best sites is crucial, which is helped by land lease agreements and local partnerships.

Financing is potentially a challenge, but the experience so far has been that funds are always forthcoming for good projects within stable regulatory environments.

Grid operators

A perennial and growing challenge is that of obtaining acceptability for new overhead cables, as is seen in Germany, where new highways have to be built to carry wind power from the north to replace nuclear power in the south. Local resistance reached such heights that the only solution will be to locate some of the capacity underground at much higher costs.

Next, as a direct result of the need for decarbonisation and the general preference for renewable rather than nuclear power generation, the trend is that energy production is moving away from the transmission system operator to the distribution system operator domain. The result is that networks which were originally designed to deliver power at lower voltage levels from the transmission network to consumers are increasingly being required to host power generation and to allow for flows in a reverse direction.

With incentive regulation being increasingly applied, grid operators start with the challenge of reducing their costs (and in some cases improving the overall service provided) in order to maximise their earnings. But on top of this may come the challenge of potentially declining demand as a result of both efficiency measures and self-consumption.


After generation and transportation, retailers are the third area of the power business to be facing major challenges, though of different sorts. These include the fact that the business is already extremely competitive; there is the prospect of non-power retailers (e.g. telecommunications, security companies or social media) moving into the retailing space; customers want to generate some of their own power; and retailers must determine how to deal with their scope to offer new flexibility services, partly driven off of consumers’ appliances.

While the entry of non-utilities into the energy retailing area has been slow (although certain supermarkets have now managed to achieve established positions), the prospect of a big game-changer remains, and this is being made easier every day through the development of communications and the accumulation of data. Energy purchasing is unfortunately (for the retailers) a low-engagement activity, and the risk is high that retailing could be subsumed by another platform (e.g. Facebook) with which customers have a much stronger engagement.

The trend to self-generation need not be totally negative for retailers if, instead of fighting it, they support it and look to earn a margin on the equipment needed and also to supply new services, such as those to help customers optimise their own energy systems. The retailers have the advantage of a very large customer base, and thus they ought to be more in tune with their wishes than other prospective market entrants, but they need to be nimble.

Offering new flexibility services (e.g. demand-side response, and potentially more services, when some customers will have batteries installed in a few years) also represents an additional revenue opportunity. But again, there could be challenges from non-utility operators. In the US, for example, a telecommunications company offers households a $100 Walmart voucher if they allow their refrigerator to be cycled within a certain temperature band according to the power supply-demand balance.


The above set of challenges shows how the operating environment is continuing to become more difficult for the power sector. Society has placed so many different and changing requirements on it that it is difficult to provide the necessary consistency and stability for the industry to function efficiently and be financially positioned to make the investments required. The imperative of decarbonisation is clear, and now that the main guidelines have been set, along with the ambitious intermediate target to achieve 40 per cent decarbonisation in 2030 vs. 1990 levels (i.e. which entails a 20 per cent reduction in a decade, as compared with the same reduction over three decades through 2020 – and that was with the benefit of the closures of inefficient factories in Eastern Europe), it is now important to provide the maximum stability to the industry and to avoid the temptation for further political interventions. Only then does it stand a chance of providing this vital “oxygen” to society under conditions acceptable to all the stakeholders.

The views expressed here are the author’s alone and do not necessarily represent those of RWE AG.

  • 1 European Commission: 2011 Energy Roadmap, COM/2011/885; Eurelectric: Energy Roadmap 2050: Empowering Europe, February 2012.
  • 2 The International Energy Agency uses several different potential scenarios to run its projections. The “New Policies Scenario” takes account of broad policy commitments that have been announced by countries, including pledges to reduce greenhouse gas emissions and phase out fossil fuel subsidies, even if the implementing measures have yet to be identified. The “450 Scenario” limits the concentration of greenhouse gases in the atmosphere to around 450 parts per million of CO2, which is consistent with the goal of limiting the global temperature increase to 2°C. For more information, see
  • 3 Eurelectric: Integrating intermittent renewables sources into the EU electricity system by 2020: challenges and solutions, May 2010; German Energy Agency (DENA): Efficient integration of renewable energy into future energy systems: Development of European energy infrastructures in the period 2030 to 2050, October 2011.
  • 4 J. Pfeifenberger, K. Spees: Characteristics of Successful Capacity Markets, APEx Conference, Brattle Group, October 2013; S. Hesmondhalgh, J. Pfeifenberger, D. Robinson: Resource Adequacy and Renewable Energy in Competitive Wholesale Electricity Markets, British Institute of Energy Economics, September 2010; Ecofys: Necessity of Capacity Mechanisms, September 2012.
  • 5 C. Waddams Price, K. Pham: The Impact of Electricity Market Reform on Consumers, ESRC Centre for Competition Policy, University of East Anglia, August 2007; Consumer Utilities Advocacy Centre Ltd: Cost Reflective Pricing Engaging with Network Tariff Reform in Victoria, Melbourne, June 2015.

Download as PDF

DOI: 10.1007/s10272-015-0541-3