Market data, investor surveys, and lab experiments

An asset-pricing model is a claim about which optimization problem people are solving when they choose their investment portfolios. One way to make such a claim testable is to derive a condition that should hold if people were actually solving this optimization problem. And the standard approach to testing whether an asset-pricing model is correct involves using market data to estimate the key parameters in this condition, plugging in the results, and checking if it does hold.

For example, the consumption capital asset-pricing model (CCAPM) argues that people view the stock market as a way to insure consumption shocks. If this is correct, then the expected return on the market should be equal to the riskfree rate plus a multiple of the market’s consumption beta, $\beta = \frac{\Cov[R_{t+1}, \, \Delta \log C_{t+1}]}{\Var[\Delta \log C_{t+1}]}$ :

(1) $\begin{equation*} \Exp[R_{t+1}] = R_f + \lambda \cdot \beta \end{equation*}$

The standard test of the CCAPM involves computing average stock returns and the market’s consumption beta using historical data, plugging in these values to Equation (1), and checking whether it holds.

Of course, this isn’t the only way to test an asset-pricing model. Researchers can use surveys to learn about which optimization problem people are solving (see here, here, and here). They can also run lab experiments where participants trade with one another in a simulated market environment (see here, here, and here). These two alternative approaches have gained popularity in recent years (here, here, and here).

Unfortunately, many researchers view investor surveys and lab experiments as just another way of generating the data needed to test the moment conditions implied by an asset-pricing model. The prevailing view is that the data produced by these two empirical approaches is just a poor substitute for market data. Not so! These empirical approaches are not substitutes. They are each good for doing different things. Investor surveys and lab experiments can answer questions that market data can’t address.

What sorts of empirical questions are best suited to being answered via sample statistics computed using market data? Where do survey responses have a comparative advantage? When can we learn the most from a well-designed lab experiment? This post outlines how I think about the answers to these questions.

Canonical asset-pricing model

To make things concrete, I’m going to talk about testing a particular asset-pricing model—namely, the CCAPM. This model says that a representative investor solves the following optimization problem:

(2) $\begin{equation*} \begin{split} \text{maximize} &\quad \Exp_0\left[ \, {\textstyle \sum_{t=0}^{\infty}} \, e^{-\rho \cdot t} \cdot \mathrm{U}(C_t) \, \right] \\ \text{subject to} &\quad \parbox[b]{1.00cm}{\raggedleft $\mathrm{U}(C)$} = {\textstyle \frac{C^{1-\gamma} - 1}{1-\gamma}} \\ &\quad \parbox[b]{1.00cm}{\raggedleft $W_t$} \geq C_t + P_t \cdot S_t + B_t \\ &\quad \parbox[b]{1.00cm}{\raggedleft $W_{t+1}$} = (P_{t+1} + D_{t+1}) \cdot S_t + (1+R_f) \cdot B_t \\ &\quad \parbox[b]{1.00cm}{\raggedleft $D_{t+1}$} = D_t \cdot e^{\mu + \sigma \cdot \varepsilon_{t+1}} \quad \varepsilon_{t+1} \overset{\scriptscriptstyle \textnormal{IID}}{\sim} \mathrm{Normal}(0, \, 1) \\ \text{on inputs} &\quad \parbox[b]{1.00cm}{\raggedleft $W_0$}, \, D_0 \! > \! 0, \, \mu \! > \! \rho \! > \! 0, \, \sigma \! > \! 0, \, \gamma \! > \! 1, \, R_f \! > \! 0 \end{split} \end{equation*}$

Here’s a line-by-line breakdown of what all this means. (Line 1; Objective) The investor tries to maximize the present discounted value of his future utility from consumption given time preference parameter $\rho > 0$ . (Line 2; Preferences) The investor has power utility with risk aversion parameter, $\gamma > 1$ . (Line 3; Budget Constraint) At time $t$ , the investor must have enough wealth $W_t$ to cover his consumption $C_t$ AND to pay for the $S_t$ shares of the market that he buys at a price of $P_t$ dollars per share AND to pay for the $B_t$ dollars he invests in riskless bonds. (Line 4; Wealth Evolution) The investor’s will receive a $D_{t+1}$ dollar dividend for each share of the market he bought at time $t$ AND he will be able to sell the share for $P_{t+1}$ dollars AND he will earn the riskfree rate of $R_f > 0$ on each dollar he invested in bonds at time $t$ . (Line 5; Dividend Process) The total market dividend grows at an average rate of $\mu > 0$ per period, and it has a per period volatility of $\sigma > 0$ .

The solution to this model reveals that:

(3) $\begin{equation*} P_t = \Exp \left[ \, e^{-(\rho + \gamma \cdot \Delta \log C_{t+1})} \times (D_{t+1} + P_{t+1}) \, \right] \end{equation*}$

The price an investor is willing to pay at time $t$ for a share of the stock market (think: level of the S&P 500) is equal to his expectation of the total future payout to owning this share at time $t+1$ (total payout = the dividend $D_{t+1}$ plus the sale price $P_{t+1}$ ) discounted back at a rate that accounts for both his preference for getting paid earlier rather than later, $\rho$ , and his aversion to fluctuations in consumption, $\gamma \cdot \Delta \log C_{t+1}$ .

According to the CCAPM, the investor should be willing to pay more for the aggregate stock market if it provides insurance against drops in consumption, $\Delta \log C_{t+1} < 0$ . And, if we define the return to holding a share of the market portfolio as $1 + R_{t+1} = (D_{t+1} + P_{t+1}) \, / \, P_t$ , then standard manipulations of Equation (3) result in the moment condition in Equation (1). Thus, if investors are solving the optimization problem in Equation (2), then the expected return on the market, $\Exp[R_{t+1}]$ , should be equal to the riskfree rate plus a term proportional to the market’s consumption beta, $\beta = \frac{\Cov[R_{t+1}, \, \Delta \log C_{t+1}]}{\Var[\Delta \log C_{t+1}]}$ .

Why we care if investors are using this model

Physicists use the principle of least action to model the path that an object will take through a field. Under this approach, it is as if the object chooses the route that minimizes the total difference between its kinetic and potential energies through time. What will the arc of a baseball look like as it travels from the center fielder’s hand to the catcher’s mitt? We know that a baseball cannot actually CHOOSE anything. Yet the predictions made by acting as if it can describe the ball’s trajectory so well that physicists find it productive to pretend otherwise.

You can imagine a world where the same reasoning applies to the CCAPM. It could be that the CCAPM makes predictions that fit the data so well that it’s useful to pretend that investors solve the optimization problem in Equation (2) regardless of whether or not they actually do. We do not live in such a world. The central empirical prediction of the CCAPM (Equation 1) is not true on average. And, even if it were, most of the variation in expected returns over time and across assets cannot be explained by differences in consumption betas. Consumption growth is nowhere near volatile enough.

In general, even when an asset-pricing model’s predictions are correct on average, the fit is poor. Financial markets are complex. Asset-pricing models are simple caricatures of the forces at work. The goal is to write down a model that captures some small kernel of truth about these forces. An asset-pricing model is only useful to the extent that we can count on its noisy predictions holding up in novel as-yet-unseen time periods, countries, asset classes, etc. What would asset prices look like if $X$ changed? Researchers write down asset-pricing models so that we can analyze such counterfactual settings. This is the point of a model.

Researchers care about whether investors are actually solving the optimization problem associated with a given asset-pricing model because this is what gives us confidence that we can count on the model’s noisy predictions in those settings. If investors are usually solving the optimization problem in Equation (2), then we can trust the model’s predictions to be roughly correct in a new market setting where the input parameters had changed. Why? Investors are usually solving that problem. A researcher who knows which optimization problem investors are solving can make robust out-of-sample predictions.

With this background in place, I can now describe the sort of question that’s best suited to be answered by each kind of empirical approach: market data, investor surveys, and lab experiments.

Are the model’s empirical predictions correct?

An asset-pricing model is a claim about which optimization problem investors are solving. The resulting first-order conditions imply that certain moment conditions should hold in the data. If your goal is to test whether these predictions are true in a given market setting, then start by looking at market data. For instance, if you want to test whether the expected return on the market is equal to the riskfree rate plus a multiple of the market’s consumption beta as predicted by the CCAPM in Equation (1), then it makes sense to estimate $\hat{\Exp}[R_{t+1}]$ and $\hat{\beta}$ using historical data on market returns and aggregate consumption.

Of course, the $\Exp[R_{t+1}]$ in Equation (1) represents INVESTORS’ beliefs about what future returns will look like on average. Likewise, the $\beta = \frac{\Cov[R_{t+1}, \, \Delta \log C_{t+1}]}{\Var[\Delta \log C_{t+1}]}$ represents INVESTORS’ beliefs about what the covariance between market returns and consumption growth will be. So you should also be able to ask investors questions about these two statistical objects. And the results should also satisfy Equation (1).

This is the usual way that researchers use investor surveys to test asset-pricing models (see here, here, and here). They ask investors about the key parameters in some moment condition. Then, they plug the results into this condition and check whether it holds. This is a perfectly fine thing to do. But it’s not an application where surveys have a comparative advantage over other empirical approaches.

Are they correct for the right reasons?

An asset-pricing model’s predictions can be correct on average for reasons that have nothing to do with the ones in the model. We know that differences in consumption betas do not explain differences in expected returns as predicted by the CCAPM. But, even if they did, it would not imply investors were actually solving the optimization problem in Equation (2) when choosing investment portfolios. Any observed correlation between expected returns and consumption betas could be a spurious correlation.

As I point out in my recent JF paper with Sam Hartzmark and Abby Sussman, investor surveys give researchers a way to investigate why a model’s predictions do or don’t hold in the data. Surveys give investors an opportunity to show their work—to describe how they arrived at the outcomes we observe in the data. This is something that you can’t examine using market data alone. This is where investor surveys have a comparative advantage over other empirical approaches (see this post for more details).

At this point, you might be thinking: “But prices can move for reasons that no individual investor can understand.” I totally agree. But this is beside the point when testing an asset-pricing model. Asset prices can move for lots of reasons. An asset-pricing model makes a claim about what one of those reasons is. The fact that there are also other reasons is neither here nor there.

We expect the set of moment conditions implied by an asset-pricing model to hold in the data because investors are maximizing the objective function in the model. So there should be some evidence that investors are actually trying to do this. And surveys are the best way to gather such evidence. Every investor doesn’t need to be thinking exactly like the agents in a model. But it’s problem if no investor thinks that way.

For what parameters does the model apply?

This brings us to the last empirical approach: lab experiments. The main concern with this sort of approach is that the experimental setup in a lab might be missing something important about how real-world markets operate, and it’s hard to know what that missing something is. The only evidence might be that your results from the lab don’t line up with what you observe in the real world.

This is a feature not a bug.

Suppose you think that some optimization problem like the one in Equation (2) captures the essence of some important market phenomenon. The resulting moment conditions are satisfied when you plug in parameter estimates using market data. And, when you ask traders about how they are trading, you get responses that are consistent with the logic of your model. But, when you set up a trading game where your model should govern participant behavior, you find no evidence of the phenomenon you’re interested in. You’ve just learned that you’re missing something. At the very least, you need tweak the input parameters.

When you run a trading game, do you get results that are consistent with the model for sensible input parameters? Do you tend to get similar results when you run lab experiments with the same input parameters? When you adjust these input parameters, at what point do the predictions of the asset-pricing model start to diverge from the results of the experiments? These are the right sorts of questions to be asking via lab experiments. A lab experiment SHOULD NOT be the first test of an asset-pricing model. In epidemiology, “Check whether a healthy person gets sick if you inject him with the cultured microorganism.” is the third of Koch’s postulates not the first.

If you show that some outcome can be generated by a trading game, you have no idea whether the conditions responsible for the outcome occur in the wild. Lab experiments are useful when working in the other direction. Suppose you think you fully understand the conditions that lead to a certain outcome in real-world financial markets. Ok, then you should be able to reproduce this outcome by simulating those conditions in the lab. Once you do that, you can also use the experiment to see how far can you adjust the input parameters and still get valid predictions. In addition to verifying your understanding, lab experiments also offer a way to assess the effective range of an asset-pricing model.