Reservations on the Use of Event Studies to Evaluate Economic Policy

Event studies represent an increasingly popular method to evaluate the welfare effects of economic policy decisions. The basic idea is that stock market reactions to the announcement of policy decisions contain superior information about the welfare effects of these decisions. This paper investigates the degree of reliability of event studies as a policy evaluation method by critically reflecting upon two underlying assumptions. Since both the information superiority of financial markets and the determination of economic welfare effects based on abnormal returns consist of considerable interpretation problems, we issue a note of caution: scientists and policy makers should be very reluctant to rely on stock market reactions as a referee on economic policy decisions. Event studies cannot replace thorough theoretical economic analysis.

A growing body of literature attempts to evaluate economic policy decisions or regulatory reforms by analysing the stock market reactions to the announcement of these decisions. So-called “event studies” use sophisticated econometric techniques to isolate what they call abnormal returns, i.e. movements in stock prices that are affected only by the “event” (announcement) and not by other influences. This is a well-established method for the empirical analysis of drivers of stock market prices. However, event studies are increasingly being used to evaluate the welfare effects of regulatory reforms. The event-specific reaction of the stock prices is believed to reveal superior information about the welfare effects of announced economic policy decisions (which represent the events). Thus, they are believed to be able to serve as a referee for the economic “quality” of the announced reforms. The underlying conceptual idea is “hiring the stock market as an advisor”.1 Along these lines, inter alia, regional trade agreements,2 merger control decisions,3 sector regulation4 or economic recovery and anti-crisis programmes5 are exposed and subjected to the judgement of the stock market with the intention of deriving economic policy recommendations. While this is usually done by scientists, a few competition authorities have already employed event studies themselves in order to evaluate some of their merger control decisions, namely Greece, Japan and Switzerland.6

This paper reflects upon the question of whether we should “hire” the stock market to tell us if our economic policy decisions are correct (in terms of welfare effects). We call for a note of caution: scepticism about the competence of stock markets to reveal superior knowledge about economic policy programmes is justified when looking at the empirical validity of the assumptions on which the event study approach to evaluating policy programmes rests. In this paper, we briefly describe the event study methodology before analysing event studies as an evaluation approach by introducing three selected examples from recent literature. Making further use of these three examples, we then critically review the general approach of using stock market reactions as policy evaluators and demonstrate the principal ambiguities of interpreting stock market signals in terms of the welfare effects of policy decisions.

Fundamentals of event studies

Event studies are a statistical methodology which was initially employed in economics in areas like accounting and financial market analysis. In a nutshell, event studies quantitatively estimate the influence of specified “events” (often announcements distributed via media regarding, for instance, business and company news, merger announcements, economic policy decisions, regulatory changes, strategic business decisions, enactment of major legislation, etc.) on the prices of securities, stocks and bonds listed and traded on stock exchanges.7 The generic model in the econometric analysis of most event studies is the following market model:

R_t,j = a_j + b_j * RM_t+ e_t,j

R_t,j denotes the return of security j on day t, with RM_t denoting the overall market return and the term e_t,j defining the firm-specific return. a_j and b_j are the linear coefficients for the regression equation above. The target in event studies is to estimate the abnormal returns (A_t ), i.e. the unexpected return obtained on the day of the announcement given the expected overall market return.

A_t = R_t - E ( R_t | RM_t ) = R_t - a - b * RM_t

The significance of abnormal returns is determined through statistical tests. A large variety of tests have been proposed for exploring and validating the existence of abnormal returns in response to an event. The inherent technical challenge of event studies lies in the proper distinction of event-specific effects from other, more general influences on the observed stock price movement.8

However, this paper does not aim to discuss the sophisticated econometrics of event studies.9 Instead, our analysis focuses on the interpretation of event study results and the derivation of economic policy conclusions. In this specific context, two different applications of the event study method must be distinguished:

1. Studies that try to find out how announcements (“events”) in the media or by companies, policy makers, authorities, governments or agencies influence stock market prices (event studies to identify drivers of stock market prices). Here the aim is to explore whether or not significant abnormal returns result as a consequence of the announcements of decisions. Thus, the research looks to explain the determinants of stock price dynamics and to identify which factors influence stock prices and how they do so. This is the original way event studies were applied, and it represents an important and hardly controversial contribution to economic research.
2. Studies that try to evaluate the welfare effects of economic policy decisions by measuring stock market price changes (abnormal returns) caused by the announcement of these decisions (event studies to evaluate policy programmes). It is this second way of applying event studies that we focus on in this paper.

Even though both applications rest on the same method, they are distinguishable with regard to their research targets: while the first type explains stock market reactions, the second type uses these reactions to evaluate policy decisions. The inherent logic of this second type of event studies crucially and sensitively rests on two (interrelated) assumptions.

The first assumption is that traders on the stock market exchange correctly anticipate the effects of the announced economic policy decisions on the profitability – including the future profitability – of the companies whose stocks they are trading. The anticipated profitability changes are immediately reflected in the current prices. Ultimately, this assumption rests on the so-called efficient financial market hypothesis (EFMH). According to the EFMH, stock markets process the information codified in the event in a perfectly rational and efficient way.

The second assumption is that the increasing or decreasing profitability of stock market companies allows one to draw conclusions regarding the welfare effects of the announced economic policy decisions. This assumption rests on the specific economic theory being utilised with regard to the effects of the announced policy decision. Thus, the exact expression of this second assumption differs among studies in different application fields.

Evaluating policy programmes or decisions with the help of event studies: three examples

Three recent examples from the literature serve to illustrate how economic theory is employed to specify the link between the profitability expectations of stock market traders and the expected economic effects of policy programmes or regulatory decisions. This link is crucial for the role of stock markets as referees on the quality of political decisions.

Regional Free Trade Agreements

The first example refers to the announcement of Regional Free Trade Agreements (RFTA).10 If the announcement causes an abnormal increase in stock prices (i.e. traders expect companies’ profits to increase), this is interpreted as an indication that the agreement will lead to more trade and therefore, according to standard trade theory, to an increase in welfare. However, if the abnormal returns are negative, the interpretation is that the country in question will not benefit from the RFTA, possibly because the negotiated trade conditions for the country are disadvantageous. Thus, abnormal stock market returns are employed as a referee in order to judge whether an announced RFTA is welfare-increasing (good policy decision) or welfare-decreasing (bad policy decision). Since stock markets react not only to final RFTA announcements but also to leaked information about the conditions of the RFTA and the probability of an agreement, the referee service provided by stock market reactions could actually be used during the negotiation process already to inform politicians of the effects of their potential decisions.

Merger control decisions

A second example refers to merger control decisions.11 It is assumed that the welfare effects of horizontal mergers can be evaluated by looking at the reactions of the stock prices of the merging companies and especially those of their rivals. Positive abnormal returns for the merging companies signal expectations of increasing profitability for the merger partners. However, this signal is ambiguous since increasing profitability can be caused by efficiency gains (a procompetitive effect) as well as by market power (an anticompetitive effect detrimental to welfare). Therefore, the focus is usually on the remaining rivals to the merging companies and follows a specific economic logic: a merger that reduces competition in the relevant market will increase the price level in that market. As a consequence, the rivals to merging companies will benefit from the merger to the degree that the lower post-merger competition intensity allows them to increase their prices, too, thus boosting their profitability. A procompetitive merger, however, will increase post-merger competition in the relevant market due to the efficiency effects and thus harm rival companies’ profitability. Consequently, positive abnormal returns for rivals to the announced merger signal an anticompetitive merger, whereas negative abnormal returns for rivals to the announced merger signal a procompetitive merger. This example shows that sometimes rather sophisticated economic theory is required to make the stock market signals unambiguously interpretable.

IMF-supported recovery programmes

The third example addresses economic recovery programmes supported by the International Monetary Fund (IMF).12 In an economic or financial crisis, the IMF offers support via programmes that help the affected countries to improve their economic conditions and overcome the crisis. The welfare effects of such programmes are evaluated by looking at the reactions of the stock market prices of companies from various sectors to the announcement of IMF-supported programmes. The general assumption in this case is that positive abnormal returns for some specific sectors imply increasing profitability of the companies within this sector and thus positive average welfare effects of the programmes. Conversely, negative abnormal returns for the companies within other sectors signal decreasing profitability and thus negative welfare effects. Thus, the stock market reactions are employed to evaluate the effects of IMF-supported reform programmes on different sectors of the economy. The underlying economic mechanism rests on the assumption that increasing profitability stems from a better sector-specific economic climate, in particular through increased demand (i.e. the programmes induce an economic upswing) or through decreased costs (i.e. the programmes induce structural reforms which decrease factor and/or input costs).

A critical reflection of event studies as referees

So, shall we hire the stock market as a referee on economic policy decisions? The answer depends on the reliability of the basic underlying assumption – the EFMH – as well as to the soundness and unambiguity of the employed economic theories interpreting the stock market signals.13

Shall we rely on the rational efficiencies of financial markets?

The central question regarding the empirical reliability of the EFMH is whether stock markets are truly efficient and rational. If one looks at the high frequency and speed of stock price changes for a given enterprise, for instance, is it really reflecting actual changes in the profitability prospects of that company? Do conditions of production and the market environment change the fundamental prospects of firms so often as the ever-changing ups and downs of stock prices would imply? Event-triggered stock price changes would reflect the true value of stocks only if stock markets functioned perfectly, i.e. only if traders acted (hyper-)rationally, if all relevant information were available and if this information were perfectly and efficiently processed.

However, within financial economics the reliability of the EFMH is viewed rather critically. In particular, the increasingly popular branch of behavioural finance (and behavioural economics in general) casts doubt on the EFMH, particularly on assumptions such as the availability of all relevant information and the efficient and undistorted processing of information.14 Furthermore, based upon extensive empirical and experimental evidence and with reference to cognitive and psychological economics, (subjectively) rational behaviour is viewed as quite distinct from “correct” behaviour (in the sense of being always right in assessments and decisions). Behavioural stock market phenomena like herd behaviour, bubbles, and over- and under-shooting effects – as well as all sorts of biases in individual (selective) information perception, processing and interpretation – are not particularly compatible with the idea that stock markets reveal superior knowledge about the true economic effects of economic policy programmes and decisions.

Even if only the assumption of the availability of all relevant information is relaxed, it remains doubtful whether abnormal stock market returns reliably signal future profitability effects. Already then, the effects anticipated by investors (ex ante expected impact) start to deviate from the real effects (ex post actual impact).15 The greater the extent to which the assumptions of complete information, perfect information processing and perfect (hyper-)rationality are relaxed, the greater the discrepancy between investor expectations and effects. As a consequence, the use of event studies as referees on the welfare effects of economic policy decisions must be viewed very sceptically if subjective rationality, incomplete information, and imperfect, selective and biased information processing represent an adequate description of real-world stock markets.

Furthermore, it can be questioned whether investors reacting to an announcement make their trading decisions with the same time horizon in mind that the welfare effects of the announced policy decision require to unfold. For instance, many economic effects of RFTAs and IMF-supported recovery programmes require substantial time before manifesting. Additionally, the effects may follow a j-curve, i.e. larger positive welfare (profitability) effects may be preceded by an interim period of smaller negative effects. If investors do not plan to hold the stocks for such a long time horizon, they will rationally try to anticipate the stock prices at a point in time where the eventual effects may not have occurred. For instance, there are some indications that merger announcements temporarily increase the stock prices of merging companies and rivals (in times of economic upswings) before they eventually decrease after the merger is consummated. Investors who are not planning to hold their shares until the merger process is completed may well speculate on this temporary increase and buy as soon as the initial information becomes available (thus contributing to the rise in stock prices) – only to sell for a profit before the merger is completed. Consequently, rather than anticipating future profitability, a successful stock market strategy may be more accurately described as guessing what others will likely guess and strategically including this in speculative and short-run stock trading. This type of trade is likely to be very event-sensitive (i.e. announcements as well as rumours) and represent event-specific effects (abnormal returns).

Another obvious problem with using event studies as referees for welfare effects is the question of whether stock market companies are sufficiently representative of the whole economy. Companies whose stocks are traded in stock markets with sufficient frequency (to allow for meaningful inclusion) represent only a subset of the whole economy. Furthermore, their structure in terms of size, affected industries, regional distribution, etc. is usually not representative.

Are the economic theories used to interpret the signals sufficiently unambiguous?

What about the second step of the referee role of event studies: can the welfare effects of policies be fully concluded from changes in the (expected) profitability of stock market companies following the announcement of such policies? This leads us back to the three examples introduced previously.

Regional Free Trade Agreements

With regard to the first example, is it really sensible to declare positive welfare effects based on expected increasing profits (positive abnormal returns) triggered by the announcement of an RFTA? Positive returns could indeed be caused by welfare-increasing liberalisation gains and the trade advantages of the agreement. However, positive returns could also result if a trade agreement included less liberalisation than hitherto expected and preserved anticompetitive rents. If an RFTA protects the anticompetitive rents of big stock market companies (or creates even more protectionism) in the shadow of an ostensible (politically labelled) “liberalisation” agreement, then abnormal returns merely signal the maintenance (or creation) of supra-competitive profits. The announcement-specific reaction may be due to (i) prior expectations that these anticompetitive rents would be eroded by the free trade agreement or (ii) the unexpected inclusion of new protectionism measures. This effect would be especially strong if the gains from freer trade predominantly benefit smaller and/or non-stock market companies and thus are neglected or under-proportionally reflected in stock market reactions. Consequently, a decrease in stock prices as a reaction to the announcement of a trade agreement can go hand in hand with welfare gains if the losses are concentrated on a few big stock market companies (hitherto enjoying anticompetitive protection rents), whereas the gains are decentralised and dispersed among a large number of companies, including a large share of non-stock market companies, and also leads to the emergence of new entrepreneurship.

Merger control decisions

In the second example regarding merger control decisions, a central link is represented by the theory that anticompetitive mergers increase rivals’ profits through the provision of a price umbrella (i.e. higher prices benefitting all companies in the market) whereas procompetitive mergers decrease rivals’ profits due to the price-reducing effects of efficiencies. First of all, it is widely accepted that this holds only for horizontal mergers and not for vertical or conglomerate mergers. Since many mergers involve complex multiproduct companies, there is often a mixture of horizontal, vertical and conglomerate effects that might be difficult to disentangle. Furthermore, among the many affected markets (both product and geographical) of such a merger, the (often few) markets that trigger the anticompetitive concerns of competition authorities must be important enough for the overall multiproduct and multinational companies to dominate the abnormal returns.

Secondly, even in the case of pure horizontal mergers, the price umbrella effect is strongest in markets that resemble quantity competition in rather homogenous Cournot oligopolies. In cases of price competition in differentiated product markets, the price umbrella differs in strength for different rivals and may become rather marginal for some. If horizontal mergers take place in markets that do not display sufficient similarities to the standard oligopoly models, then the interpretation of rivals’ profit changes becomes ambiguous and unclear.

Thirdly, economic theory also refers to cases in which anticompetitive horizontal mergers harms rivals’ profits, inter alia, through (vertical) foreclosure effects regarding procurement and distribution or predatory and deterrence strategies as well as raising rivals’ costs strategies that enrich the arsenal of the (horizontally) merged companies as a consequence of their more powerful post-merger position. Fourthly, merger announcements often create “fantasies” (in stock exchange parlance) of follow-up mergers among rivals to the merging companies. As a consequence, abnormal returns may be driven by those speculations about rivals rather than by a rational assessment of the profitability effects on rivals.

Ultimately, we are not aware of any study that empirically analyses whether investors actually believe in the “anticompetitive mergers are good for rivals” theory from industrial economics. If the mental models of investors instead represent the belief that anticompetitive mergers usually harm rivals’ profitability (even if this is based on unscientific common knowledge rather than game-theoretic economics), then the interpretation of stock market signals is systematically flawed. In summary, it would be rather impudent to claim to be able to determine the competitive effects of a merger based on expectations about rivals’ profits. Relying on ambiguous stock market signals when evaluating a merger involves a considerable risk of getting it wrong.

IMF-supported recovery programmes

The third example involves IMF-supported recovery programmes. Policy decisions on short-run macroeconomic strategies, for instance directed at the stabilisation of exchange rates, or long-run fundamental policy strategies targeting structural reforms are particularly prone to the general problems of differing time horizons between investors and effects, j-curve effects, the lack of representativeness of stock market companies, etc. This is even more true since the actual ex-post impacts will be based on how the society reacts to the announcement of the IMF programme, and more importantly, on how the government reacts to the IMF advice.

Experience shows that there may be a considerable difference between proposed programmes and the ultimately implemented policies, for instance due to governments evading or diluting necessary (but unpopular) reforms or contradicting their effects with other policy strategies. This further stretches the anticipation capabilities of investors. Furthermore, the negative abnormal returns of a sector (triggered by the IMF-supported programme) need not necessarily correspond to overall negative welfare effects. If the major companies of a sector (maybe very few if the sector is highly concentrated or even dominated by a government-related quasi-monopolist) lose privileges, protection from international competition or other anticompetitive rents in the course of the reforms of the IMF-supported programme, then investors will rationally expect their profits to decline.

However, economic theory predicts that the corresponding increase in consumers’ rent will outweigh the loss of the anticompetitive rents by some margin and the welfare effect will be positive. Thus, negative abnormal returns may well correspond to welfare gains. In addition, the same reservation made in the RFTA example regarding concentrated and stock market-relevant losses being overcompensated by decentralised and stock market-irrelevant gains applies here.

Conclusion

It is tempting to hire the stock market as a “neutral” referee for assessing economic policy decisions. One reason is certainly the desire for an external, unbiased and neutral evaluator who is not subject to party interests or career concerns. Another reason is related to feasibility bias: the data availability for event studies is often much better than for other evaluation methods, thus facilitating the production of this type of academic study.16

However, it is necessary to issue a note of caution. Firstly, it is doubtful whether stock market reactions (abnormal returns) really reveal superior knowledge about economic effects.17 Secondly, the obtained signals usually offer more than one economic interpretation and thus are quite ambiguous. Consequently, it is rather doubtful that event studies can be successfully employed to evaluate the “quality” of economic policy decisions, programmes or reforms with sufficient reliability. Even if the studies were conducted to the highest econometric standards, it would represent a risky gamble to adjust policy decisions according to the reactions of the stock market. Consequently, any tendency towards the increased use of event studies as an instrument to evaluate economic policy decisions must be viewed with serious concern. Event studies represent a highly useful methodology in the context of their original objective, i.e. research aiming to identify what drives stock market prices. However, they are not suited to replace or complement thorough economic analysis of the effects of policy decisions, programmes and regulatory reforms.

• 1 C. Moser, A.K. Rose: To Agree or not to Agree on Regional Trade: Hiring the Stock Market as an Advisor, 2011, http://www.voxeu.org/article/what-stock-markets-say-about-regional-trade-agreements.
• 2 C. Moser, A.K. Rose: Who Benefits from Regional Trade Agreements? The View from the Stock Market, CEPR Discussion Paper, No. 8566, 2011.
• 3 T. Duso, K. Gugler, B. Yurtoglu: How Effective is European Merger Control?, in: European Economic Review, Vol. 55, No. 7, 2011, pp. 980-1006.
• 4 A.W. Dnes, D. Kodwani, J.S. Seaton, D. Wood: The Regulation of the United Kingdom Electricity Industry: An Event Study of Price-Capping Measures, in: Journal of Regulatory Economics, Vol. 13, 1998, pp. 207-225.
• 5 T. Miyakoshi, Y. Tsukuda, J. Shimada: The Effects of IMF Supported-Program on the Asian Crisis, Discussion Paper, No. 07-24, 2007, Osaka School of International Public Policy (OSIPP).
• 6 According to these countries’ written reports to the 2011 OECD roundtable on “Impact Evaluation of Merger Decisions”.
• 7 C.J. Corrado: Event Studies: A Methodology Review, in: Accounting and Finance, Vol. 51, No. 1, 2011, pp. 207-234; A. McWilliams, D. Siegel: Event Studies in Management Research: Theoretical and Empirical Issues, in: Academy of Management Journal, Vol. 40, No. 3, 1997, pp. 626-657.
• 8 C.J. Corrado, op. cit.
• 9 For this, see C.J. Corrado, op. cit., and the literature cited therein.
• 10 C. Moser, A.K. Rose: To Agree or not…, op. cit.; C. Moser, A.K. Rose: Who Benefits from…, op. cit.
• 11 The study by T. Duso et al., op. cit., represents one of the best developed examples. See O. Budzinski: Impact Evaluation of Merger Dicisions, in: OECD (ed.): Roundtable Impact Evaluation of Merger Decisions, Paris 2011, OECD, pp. 149-160; O. Budzinski: Empirische Ex-Post Evaluation wettbewerbspolitischer Entscheidungen: Methodische Anmerkungen, in: T. Theurl (ed.): Methodische Grundlagen der empirischen Institutionenökonomik, Berlin 2012, Duncker & Humblot, pp. 45-71; O. Budzinski: Impact Evaluation of Merger Control Decisions, in: European Competition Journal, Vol. 9, 2013 (forthcoming).
• 12 T. Miyakoshi, op. cit; A.M. Kutan, G. Muradoglu, B.G. Sudjana: IMF Programs, Financial and Real Sector Performance, and the Asian Crisis, in: Journal of Banking and Finance, Vol. 36, No. 1, 2012, pp. 164-182.
• 13 See, inter alia, Y.S. Hopkins, J.M. Connor: A Re-examination of Event Studies Applied to Challenged Horizontal Mergers, Working Paper NE-165, 1992; K.M. Reynolds: Anticipated versus Realized Benefits: Can Event Studies Be Used to Predict the Impact of New Regulations, in: Eastern Economic Journal, Vol. 34, No. 3, 2008, pp. 310-324; and O. Budzinski: Impact Evaluation of Merger Decisions…, op. cit.; O. Budzinski: Empirische Ex-Post Evaluation…, op. cit.; O. Budzinski: Impact Evaluation of Merger Control Decisions…, op. cit., and the literature cited therein.
• 14 See inter alia A. Shleifer, L. Summers: The Noise-Trader Approach to Finance, in: Journal of Economic Perspectives, Vol. 4, No. 2, 1990, pp. 19-33; A. Shleifer: Inefficient Markets, Oxford 2000, Oxford University Press; R.J. Shiller: From Efficient Markets Theory to Behavioral Finance, in: The Journal of Economic Perspectives, Vol. 17, No. 1, 2003, pp. 83-104.
• 15 M. Cichello, D.J. Lamdin: Event Studies and the Analysis of Antitrust, in: International Journal of the Economics of Business, Vol. 13, No. 2, 2006, pp. 229-245; K.M. Reynolds, op. cit.
• 16 O. Budzinski: Impact Evaluation of Merger Control Decisions…, op. cit.
• 17 See D. Neven, H. Zenger: Ex Post Evaluation of Enforcement: A Principal-Agent Perspective, in: De Economist, Vol. 156, No. 4, 2008, pp. 477-490, for an evaluation that concludes experts may enjoy superior knowledge to stock markets.