Asset-pricing models as theories of good synthetic controls

In 1988, California passed a major piece of tobacco-control legislation called Proposition 99. This bill increased the tax on cigarettes by $0.25 a pack and triggered a wave of bans on smoking indoors throughout the state. After the bill was passed in California, it became more expensive to smoke in California and there were fewer places to do so.

It makes sense that Prop 99 could have caused lots of people in California to stop smoking. And, consistent with this hypothesis, AbadieDiamondHainmueller2010 found that per capita cigarette consumption in California fell by around 40 packs per year from 1985 to 1995. In 1985, the typical Californian bought 100 packs per year. By 1995, the average Californian bought fewer than 60 packs per year.

But was the effect causal? Did the passage of Prop 99 cause per capita cigarette consumption in California really drop by 40 packs per year? To answer this question, you need to know how many packs of cigarettes each Californian would have bought in 1995 had Prop 99 not been passed.

It’s not obvious how you should compute this counterfactual. You can’t just assume that, in the absence of Prop 99, cigarette consumption in California would have been the same in 1995 as it was in 1985. The popularity of smoking has been falling over time throughout the country. You also can’t naively compare cigarette consumption in California in 1995 to that of a neighboring state, like Nevada, in the same year. People in Nevada are more likely to partake in all sorts of vices (smoking, drinking, gambling, etc). Comparing per capita cigarette sales in California to that of Nevada in 1995 will tend to overstate the effect of Prop 99.

But, what if rather than using just Nevada in 1995 as your stand-in for California sans Prop 99, you instead used a composite Frankenstate that has the same observable characteristics as California. e.g., people in Nevada may be much more likely to smoke, drink, and gamble relative to people in California, but people in Utah are much less likely to do all of those things than Californians. So a weighted average of per capita cigarette consumption in Nevada and Utah in 1995 might represent a good synthetic control for California.

This post first outlines the idea behind using a synthetic control. Then, I make a connection between literatures: when an asset-pricing researcher computes a stock’s abnormal return by subtracting off the return of a replicating portfolio with the same risk exposures, he’s using a synthetic control. In fact, this is exactly what the OG synthetic control paper does! AbadieGardeazabal2003 computes abnormal returns for Basque companies relative to the CAPM and the FamaFrench1993 three-factor model. I wrap up by pointing out some interesting takeaways from this connection for both asset-pricing researchers and metrics folks.

Problem setup

Here’s the canonical synthetic-control problem. Imagine that you’ve got data on how much people spend on smoking and drinking in three different states, $n \in \{ \texttt{CA}, \, \texttt{NV}, \, \texttt{UT} \}$ , in two particular years, $t \in \{ \texttt{1985}, \, \texttt{1995} \}$ . For simplicity, I’m going to talk about Prop 99 as a policy that banned indoor smoking outright:

$\begin{equation*} \mathit{IndoorSmokingPolicy}_{n,t} = \left( \begin{array}{r|cc} & \texttt{1985} & \texttt{1995} \\ \hline \texttt{CA} & \mathtt{\emptyset} & \texttt{Ban} \\ \texttt{NV} & \mathtt{\emptyset} & \mathtt{\emptyset} \\ \texttt{UT} & \mathtt{\emptyset} & \mathtt{\emptyset} \end{array} \right) \end{equation*}$

Thus, you have one state-year observation with a smoking ban in place and five without one.

Let $\mathit{CigSales}_{n,t}(p)$ denote the number of packs bought by the average person in state $n$ during year $t$ given the prevailing indoor smoking policy $p \in \{ \mathtt{\emptyset}, \, \texttt{ban} \}$ . You want to know how Prop 99 affected cigarette sales:

$\begin{equation*} \underbrace{\mathit{CigSales}_{\texttt{CA},\texttt{95}}(\texttt{Ban})}_{\substack{\text{\phantom{j}observed\phantom{j}} \\ \text{outcome}}} - \underbrace{\mathit{CigSales}_{\texttt{CA},\texttt{95}}(\mathtt{\emptyset})}_{\substack{\text{hypothetical} \\ \text{counterfactual}}} = \text{causal effect of Prop 99 on cigarette sales} \end{equation*}$

The first term, $\mathit{CigSales}_{\texttt{CA},\texttt{95}}(\texttt{Ban})$ , is the per capita cigarette sales observed in California during 1995 after Prop 99 had been implemented. The second term, $\mathit{CigSales}_{\texttt{CA},\texttt{95}}(\mathtt{\emptyset})$ , reflects packs per person during 1995 in an alternative world where everything is the same except that Prop 99 was never passed.

The key empirical challenge rests on the fact that, while I can observe cigarette sales for the year 1995 in California where Prop 99 has already been passed, $\mathit{CigSales}_{\texttt{CA},\texttt{95}}(\texttt{Ban})$ , I cannot directly observe cigarette sales in a version of 1995 California where Prop 99 didn’t go into law, $\mathit{CigSales}_{\texttt{CA},\texttt{95}}(\mathtt{\emptyset})$ . This counterfactual world is a hypothetical scenario. It never happened. The challenge is to come up with some stand-in value for $\mathit{CigSales}_{\texttt{CA},\texttt{95}}(\mathtt{\emptyset})$ based on the data that I can observe in other states.

So, you want to think about a data-generating process where there’s a potential effect coming from a one-time policy change AND a bunch of things that affect statewide cigarette sales during normal times:

$\begin{equation*} \mathit{CigSales}_{n,t}(p) = \!\!\! \underbrace{\alpha \cdot 1_{\{p= \texttt{Ban}\}}}_{\substack{\text{causal\phantom{j}effect of one-} \\ \text{time policy change}}} \!\!\! + \underbrace{\mu_t + \lambda \cdot X_n + \varepsilon_{n,t}}_{\substack{\text{determinants of cigarette} \\ \text{sales during normal times}}} \qquad \qquad \varepsilon_{n,t} \overset{\scriptscriptstyle \text{IID}}{\sim} \mathrm{Normal}(0, \, \sigma^2) \end{equation*}$

You want to know whether the introduction of Prop 99, which is captured by the $1_{\{p= \texttt{Ban}\}}$ term, had an effect on cigarette sales in California. i.e., you want to know whether $\alpha < 0$ . If Prop 99 had no effect, then $\alpha = 0$ .

The remaining determinants of statewide cigarette sales during normal times are important because they dictate which observables might be a good stand-in for the counterfactual version of California in 1995 where Prop 99 was never passed as illustrated in the interactive figure below. The left panel depicts the number of cigarette packs purchased by an average resident in each of your three states during 1985 (y axis) as a function of liquor consumption in that state (x axis). The right panel shows the same thing but for 1995. The solid circles represent observed values of per capita annual cigarette sales. The dotted circle represents the counterfactual value for California in a world where Prop 99 was not passed ( $p = \mathtt{\emptyset}$ ).

$X_n$ represents liquor consumption in state $n$ . People in Nevada are more likely to spend money on all sorts of vices (smoking included) than people in California. $X_n$ is a proxy for this statewide predisposition. So, if you ignore this background variable and naively compare cigarette purchases in California to that in Nevada during 1995, then it’ll look like Prop 99 had an outsized effect. People in Nevada purchase an extra $15$ packs per year relative to California in the figure. Thus, using Nevada during 1995 as your counterfactual observation would cause you to overstate the true causal effect of Prop 99 by $15$ packs per person annually.

$\mu_t$ is the average per capita cigarette sales during year $t$ across all states. Cigarette sales have been falling over time, so $\mu_{\texttt{95}} < \mu_{\texttt{85}}$ . However, in its initial configuration, cigarette sales are constant over time in the figure above, $\Delta \mu = (\mu_{\texttt{95}} - \mu_{\texttt{85}}) = 0$ . If that were the case, then you could use per capita cigarette sales in California during 1985 as your counterfactual observation. But, by moving the $\Delta \mu$ slider, you can see how a downward time-series trend in cigarette sales would cause you to again overstate the effect of Prop 99:

$\begin{equation*} \Exp\big[ \, \underbrace{\mathit{CigSales}_{\texttt{CA}, \texttt{95}}(\texttt{Ban})}_{\text{observed}} - \underbrace{\mathit{CigSales}_{\texttt{CA}, \texttt{85}}(\mathtt{\emptyset})}_{\text{observed}} \, \big] = \Delta \mu + \alpha \end{equation*}$

Synthetic control

Because smoking has been growing less and less popular over time, you want to create the counterfactual for cigarette sales in California during 1995 using contemporaneous data from other states. But you also recognize that no other state is a perfect doppelganger for California. People in Nevada are more likely to smoke, drink, and gamble relative to people in California. Utah residents are less likely to do all of those things than Californians. So why no do the obvious thing and average these two values?

For conreteness, suppose that the drinking rate in California are exactly halfway in between the rates in Utah and Nevada:

$\begin{equation*} X_{\texttt{CA}} = (1/2) \cdot X_{\texttt{UT}} + (1/2) \cdot X_{\texttt{NV}} \end{equation*}$

In practice, the weights wouldn’t be exactly $(1/2)$ . But you could estimate these values using your 1985 data. And you could use these weights to construct a Voltron-esque counterfactual for cigarette sales in California during 1995 out of the contemporaneous values for Utah and Nevada:

$\begin{equation*} \begin{split} \widehat{\mathit{CigSales}}_{\texttt{CA}, \texttt{95}}(\mathtt{\emptyset}) &= (1/2) \cdot \overbrace{\mathit{CigSales}_{\texttt{UT},\texttt{95}}(\mathtt{\emptyset})}^{\text{observed}} + (1/2) \cdot \overbrace{\mathit{CigSales}_{\texttt{NV},\texttt{95}}(\mathtt{\emptyset})}^{\text{observed}} \\ &= \mu_{\texttt{95}} + \lambda \cdot X_{\texttt{CA}} + (1/2) \cdot (\varepsilon_{\texttt{UT},\texttt{95}} + \varepsilon_{\texttt{NV},\texttt{95}}) \end{split} \end{equation*}$

Since it’s made out of observations from 1995, this synthetic control is not confounded by the nationwide drop in cigarette sales from 1985 through 1995. By matching California’s alcohol sales, $X_{\texttt{CA}}$ , this synthetic control also accounts for persistent differences in cigarette sales across states due to differing propensities to partake in all vices. And, if we’ve done everything correctly, then we can compute

$\begin{equation*} \Exp\big[ \, \underbrace{\mathit{CigSales}_{\texttt{CA}, \texttt{95}}(\texttt{ban})}_{\text{observed}} - \underbrace{\widehat{\mathit{CigSales}}_{\texttt{CA}, \texttt{95}}(\mathtt{\emptyset})}_{\text{calculated}} \, \big] = \alpha \end{equation*}$

where $\alpha$ denotes the true causal effect of Prop 99 on annual per capita cigarette sales in California.

Risk adjustment

The synthetic control for cigarette sales in California during 1995 was a weighted average of cigarette sales in Nevada and Utah where the weights were chosen to replicate California’s value of $X_{\mathtt{CA}}$ . While this approach has some advantages, it’s also somewhat unsatisfying in that there’s no real physical analog to the counterfactual it produces. There’s no process by which you can take a weighted average of Utah and Nevada. This is purely a statistical construct.

The key insight in this post is that, when an asset-pricing researcher computes a risk-adjusted return for some asset relative to particular model, he’s using this same synthetic control methodology. And in an asset-pricing context, there’s a clear physical analog to the resulting counterfactual. The synthetic control represents a portfolio of the underlying assets with appropriately chosen portfolio weights. e.g., in the case of the CAPM, a synthetic control observation for a particular asset is a replicating portfolio with weights chosen so that it has the exact same market beta.

For example, suppose you think expected returns are governed by the CAPM. Then $\mu_t = \mathit{RiskfreeRate}_t$ is the prevailing risk-free rate at time $t$ , $X_n = \Cov[\mathit{Return}_{n,t}, \, \mathit{Market}_t] \, / \, \Var[\mathit{Return}_{n,t}]$ is the market beta on the $n$ th asset, and $\lambda$ is price of an increase in exposure to this market risk factor:

$\begin{equation*} \mathit{Return}_{n,t}(p) = \underbrace{\alpha \cdot 1_{\{ p = \texttt{hi} \}}}_{\substack{\text{effect\phantom{j}of\phantom{j}anom-} \\ \text{lous predictor}}} + \underbrace{\mu_t + \lambda \cdot X_n + \varepsilon_{n,t}}_{\substack{\text{what\phantom{j}determines\phantom{j}returns} \\ \text{in asset-pricing model}}} \qquad \qquad \varepsilon_{n,t} \overset{\scriptscriptstyle \text{IID}}{\sim} \mathrm{Normal}(0, \, \sigma^2) \end{equation*}$

The asset’s return should be higher if the risk-free rate is higher ( $\mu_t$ is high), if it has lots of exposure to market risk ( $X_n$ is high), and/or if the price of this exposure to market risk is high ( $\lambda$ is high).

The core claim in any asset-pricing model (CAPM included) is that, after controlling for the $X$ variables specified in the model, it shouldn’t be possible to find another predictor, $p \in \{\texttt{lo}, \, \texttt{hi} \}$ , that forecasts returns:

$\begin{equation*} \text{claim: $\alpha = 0$ for every predictor $p$ that you can think of} \end{equation*}$

And how would an asset-pricing researcher test to see if $\alpha = 0$ ? He’d compare the $n$ th asset’s returns to the returns of a replicating portfolio whose weights were chosen so that it had the exact same value of $X_n$ . e.g., suppose we’re in a CAPM world, and the $n$ th asset has a market beta of $X_n = 0.50$ . If there are two other assets with betas equal to $0.20$ and $0.80$ respectively, you should compare the $n$ th asset’s return to the return of a equally weighted portfolio of those two assets, $X_n = (1/2) \cdot 0.20 + (1/2) \cdot 0.80 = 0.50$ . Exact same situation! And the original synthetic-control paper (AbadieGardeazabal2003) pointed out as much!

Some takeaways

This connection between the synthetic-control literature and the asset-pricing literature delivers some interesting takeaways on both sides. First, let’s think about it from the perspective of an asset-pricing researcher. There have been several recent econometric advances in the study of synthetic controls. e.g., Chen2023 frames the synthetic-control procedure as an online learning problem. The paper then uses this parallel to give policymakers some guidance on when and where synthetic control is most likely to be successful. By framing risk adjustment as a specific instance of a more general approach to producing synthetic controls, asset-pricing researchers might be able to port over some of these recent advances.

I think the message is a bit less positive when traveling from asset pricing back to the econometrics of synthetic controls. It’s been 50 years since Merton1973 introduced the ICAPM, and asset-pricing researchers have yet to agree on which $X$ s to use when doing our risk adjustments. This fact should give econometricians pause when considering the limits of the synthetic-control approach. In a recent review article, Abadie2021 argues that the synthetic-control methodology offers a “safeguard against specification searches. (p406)” Judging by the current state of the asset-pricing literature, I’m not sure this is true.

Abadie2021 also argues that, while a researcher using the synthetic-control procedure might make an error in choosing control variables, at least the procedure is transparent about how a counterfactual is being constructed. The procedure itself is certainly transparent. But I’m not sure how many people really think through the logic now that synthetic control has gone mainstream. How many people think of the buildup of pus in a pimple when they use the phrase “coming to a head”? The conceptual metaphor is perfectly transparent. But most people never look. In a similar vein, asset-pricing researchers often use the FamaFrench1993 three-factor model to “control for risk” in spite of the fact that real-world investors aren’t trying to use their stock portfolios to buy insurance against these risk factors. An empirical procedure which initially encourages introspection can eventually turn into a stale thoughtless idiom. The asset-pricing literature suggests that econometricians should be more worried about this trend.