Rust (1987) Part 2: Counterfactuals

Part 1: Interpreting Rust’s Results

Last class, using the nested fixed point algorithm, we estimated that replacement costs were 9.802820 and monthly maintenance costs were 2.712503. Actually, we used v_keep <- -theta[1] * 0.001 * (1:S) + beta * EV_keep, so maintenance costs were .001 * 2.712503 = 0.002712503. That is, we rescaled by .001 to make maintenance costs on the same scale as the replacement cost for the accuracy of the estimation. So we estimated that the height of the “ledge” is 9.802820 and the height of each step in the staircase is 0.002712503.

What do these numbers actually mean?

Replacement costs were actually easily observed by Harold Zurcher: according to him, the cost of a new bus engine and the labor to replace it was $9499 in the mid-1980s.
Maintenance costs (the average amount costs increase for an engine as it moves up mileage buckets), on the other hand, was not known by Zurcher. That’s because it’s a complicated function of costs of the routine repairs required and the cost of the lower reliability of an old engine, including the costs of downtime from breakdowns and lost goodwill from bus riders.

But if we scale Rust’s results into dollars, we know the average cost of maintenance, inferred from Zurcher’s replacement decisions!

Question 1: If the ratio between replacement costs and monthly maintenance costs are 9.802820:0.002712503, and if replacement costs are $9499, what are monthly maintenance costs for a new engine? And how much would monthly maintenance costs be (on average) for a bus with 300,000 miles on it?

Part 2: Counterfactual Analysis: Deriving the Demand Curve for Bus Engines

Econometrics has two broad traditions: reduced-form and structural approaches.

In the Fulton Fish Market project at the end of Unit 1, we used instrumental variables to estimate a demand function. That method is reduced-form: it estimates how quantity responds to price directly from observed data.

Here, we will take a different route. Instead of estimating demand directly, we will use a structural model of Zurcher’s replacement decisions to simulate how demand changes when the price of a new engine changes.

Recall that a demand function tells you how consumer behavior changes given a change in price. That is, if the cost of replacing a bus engine decreased by $1000, how many more bus engines would Harold Zurcher purchase? We’ll solve this in steps.

Step 1: What is Zurcher’s current quantity demanded?

Let “quantity demanded” be how many buses Zurcher replaces per month. We need to find out the long-run state distribution of the endogenous markov chain, given theta = (replacement costs, maintenance costs).

Recall that in the pizza place/Mexican restaurant example, we had a transition matrix like this:

P <- matrix(c(.7, .3, .4, .6), nrow = 2, byrow = T)

# And if we took P to a high enough power, we'd see
# both rows become equivalent, indicating we've
# reached the steady state distribution of how many
# people are visiting each restaurant:
library(expm)
P %^% 50

# We want to do the same kind of thing for the Rust model.

First, we’ll build the transition matrix under Zurcher’s policy function.

In state $s$, with probability p_keep(s), the next state follows row $s$ of t_keep.
And with probability 1 - p_keep(s), the next state follows row $s$ of t_replace.

So the endogenous transition matrix is:

\[P_{\theta}(s) = p_{keep}(s) T_{keep}(s) + (1 - p_{keep}(s)) T_{replace}(s)\]

Note: you’ll need to bring some functions in from the previous classwork for this to work.

policy_transition <- function(theta) {
  p_keep <- choice_prob(theta)
  P <- matrix(rep(0, 90*90), nrow = 90)
  
  for (s in 1:90) {
    P[s, ] <- p_keep[s] * t_keep[s, ] + (___) * ___
  }
  
  return(P)
}

# Look at the first 5 rows and columns:
policy_transition(c(2.628434, 9.802820))[1:5, 1:5]

# This is the markov transition matrix, taking into
# account Zurcher's choice probabilities, which depend
# on theta!

# Take `policy_transition(c(2.628434, 9.802820))` to a
# sufficiently large power so that all rows are equal.
p_lr <- policy_transition(c(2.628434, 9.802820)) %^% ___

view(p_lr)

# Check that you've reached the long-run (all rows
# are equal).
library(testthat)
test_that("rows of P_lr close enough to equal", {
  expect_equal(nrow(unique(round(p_lr, 10))), 1)
})

Next, we’ll use the stationary distribution p_lr and turn that into “replacements per month”, which is our baseline quantity demanded.

The expected replacements per month is just $E[X] = \sum_i x_i p_i$: the sum of the number of buses at each mileage bucket times the probability a bus is replaced at each mileage bucket:

Take the long-run distribution of mileage (any row of p_lr)
Multiply it by the probability the engine is replaced if it’s in such a mileage (1 - p_keep)
Take the sum of the result!

p_keep <- choice_prob(c(2.628434, 9.802820))
sum(p_lr[1, ] * (1 - p_keep))

# Multiply this number by 104 (Zurcher has 104 buses in his fleet):
104 * sum(p_lr[1, ] * (1 - p_keep))

# So on average, Zurcher replaces 1.279 buses 
# every month: that is his quantity demanded 
# when the replacement cost is $9499.

Step 2: Repeat the analysis: calculate Zurcher’s quantity demanded if replacement costs were to increase by $1000 up to $10499?

Find theta[2]. Maintenance costs stay the same, so theta[1] stays at 2.628434.
Find p_lr.
Find p_keep.
Finally, find the quantity of new engines Zurcher will demand every month.

theta2 <- ___

p_lr <- ___

test_that("rows of P_lr close enough to equal", {
  expect_equal(nrow(unique(round(p_lr, 10))), 1)
})

p_keep <- ___

___
# You should find that Zurcher replaces 1.174871 buses 
# every month when replacement costs increase by $1000.

Step 3: Write a function `demand()` that takes an engine replacement cost and returns Zurcher’s quantity demanded.

Then draw the demand curve with a ggplot.

demand <- function(RC) {
  theta2 <- ___
  p_lr <- ___
  p_keep <- ___
  ___
}

# For the visualization: precompute on a grid
RC_grid <- seq(2000, 20000, by = 1000)
demand_data <- tibble(
  RC = RC_grid,
  Q = map_dbl(RC_grid, demand)
)

ggplot(demand_data, aes(x = RC, y = Q)) +
  geom_line()

Rust (1987) Reflection Questions

What is the inverse reinforcement learning problem? How is Rust (1987) an example?
Rust (1987) is a seminal paper in the dynamic discrete choice literature in Econometrics. For an economist, why are dynamic discrete choice models useful? Compare this to how an inverse reinforcement learning model is useful to a machine learning practitioner.
In Rust (1987), what is t_keep and t_replace? What’s the difference between t_keep, t_replace, and p_lr from this assignment?
In Rust (1987), what is the inner loop (value iteration), what does it take, what does it return, and why is it important?
In Rust (1987), what is choice_prob()? What does it take and why does it do what it does to those inputs? What does it return, and why is it important?
In Rust (1987), what is log_lik? What does it take and why does it do what it does to those inputs?
In a couple of sentences, summarize the nested fixed point algorithm.
What data is required to estimate a dynamic discrete choice model, and how does that data enter into the nested fixed point algorithm?

Download this assignment

Here’s a link to download this assignment.