---
title: "5.1 Growing a Regression Tree"
format:
  html:
    self-contained: TRUE
---

A **decision tree** is a method that can be used for regression problems (the dependent variable is continuous), or classification problems (the dependent variable is discrete). In this assignment, you'll practice growing a regression decision tree to estimate 1987 salaries of major league baseball players based on their hits, runs, and home runs the previous season (1986), along with the number of years they had spent in the major leagues.

Like K Nearest Neighbors, decision trees do well at estimating highly nonlinear, complex relationships between variables. But compared to KNN, decision trees are much easier to interpret: in fact, they are even easier to interpret than linear regression! Here's an example of a simple decision tree on baseball player salaries in logs:

![](images/tree.png){width=60%}

This tree has two internal nodes and three terminal nodes. The left-hand branches correspond to "TRUE" and the right-hand branches correspond to "FALSE". So players with less than 4.5 years of experience have a predicted log salary of 5.11. For more experienced players, the number of hits they had in the previous season is found to be highly predictive of salary.

Decision trees segment the predictor space into rectangular regions, and the prediction is the average salary in the training set for that region. At each branch, you'll consider all predictors and all cutpoints $s$, selecting the predictor and cutpoint that minimizes the RSS (residual sum of squares). Continue until all regions have a sufficiently small number of training observations.

In this assignment, we'll work on growing a decision tree by hand. Then we'll compare our decision tree to the one created with the R package `tree`.

```{r, message = F}
library(tidyverse)
hitters <- read_csv("https://raw.githubusercontent.com/cobriant/teaching-datasets/refs/heads/main/hitters.csv") %>%
  mutate(Salary = log(Salary))

train <- hitters %>% 
  filter(training == 1) %>%
  select(-training)
```

## `dense_rank`

The dplyr function `dense_rank` gives a rank to each value in a vector, and ties get the smaller value. For example:

```{r}
df <- tibble(x = c(5, 3, 3, 3, 8, 4))

df %>%
  mutate(r = dense_rank(x))
```

This will be really helpful in this assignment. In particular, we'll use it to create regions with smaller values versus larger values, preserving ties in the same region. If we want to create a region with the first rank versus another region with all other ranks:

```{r}
n <- 2

df %>%
  mutate(region = if_else(dense_rank(x) < n, "small", "large"))
```

<div style="background-color:#c7f041; border-width:3px; border-style:solid; border-color:#c7f041; padding: 1em;">
1. Consider `HmRun` and the training data set `train`. This variable takes on 35 distinct values in this data set (between 0 and 40 home runs in the previous season). If you create two regions using a cutpoint of the 5th value for HmRun ($H \geq H_5$ or $H \lt H_5$), and you predict the average salary for each region, what is the RSS and predicted log salaries for players above and below that cutpoint? 

```{r}
hitters %>% count(HmRun)

# train %>% 
#   select(Salary, HmRun) %>%
#   mutate(
#     rank = ___,
#     region = if_else(rank >= 5, "high", "low")
#   ) %>%
#   group_by(___) %>%
#   summarize(
#     prediction = mean(___),
#     RSS = sum((___)^2),
#     variable_min = min(HmRun)
#     ) %>%
#   reframe(RSS = sum(RSS), region, prediction, cutpoint = max(variable_min)) %>%
#   pivot_wider(names_from = region, values_from = prediction)
```

Interpretation: players with less than ___ home runs last season receive a log salary of ___ on average; players with ___ or more home runs last season receive a log salary of ___ on average. Cutting home runs at this point gives an RSS of ___.
</div>

## RSS Helper Function

<div style="background-color:#c7f041; border-width:3px; border-style:solid; border-color:#c7f041; padding: 1em;">
2. Repeat the process from question 1, this time writing a helper function `RSS` that takes a variable and a cutpoint rank number, and returns the RSS and the cutpoint.

```{r}
# RSS <- function(variable, cut_rank, dataset) {
#   dataset %>% 
#     select(Salary, {{ variable }}) %>%
#     ___
# }

# Test: you should get the same answer as in question 1
# RSS(HmRun, 10, train)
```
</div>

## Cutpoint Helper Function

<div style="background-color:#c7f041; border-width:3px; border-style:solid; border-color:#c7f041; padding: 1em;">
3. Write another helper function `cutpoint` that takes a variable and searches over all possible cut rank numbers to find the cutpoint that results in the lowest possible RSS. Your function should return the RSS, the best cutpoint, and the predicted salary for players below and above that cutpoint. Make sure to use `RSS` along with `map`.

```{r}
# cutpoint <- function(variable, dataset) {
#   max_rank <- dataset %>%
#     distinct({{ variable }}) %>%
#     nrow()
#   
#   map(
#     .x = 1:max_rank,
#     .f = function(x) {
#       ___
#     }
#   ) %>%
#     list_rbind() %>%
#     slice_min(___)
# }

# cutpoint(HmRun, train)
```

Interpretation: the best cutpoint for home runs are ___ home runs. Players with less than ___ home runs last season receive a salary of ___ on average; players with ___ or more home runs last season receive a salary of ___ home runs on average. Cutting home runs at this point gives an RSS of ___.
</div>

## Variable Search Helper Function

<div style="background-color:#c7f041; border-width:3px; border-style:solid; border-color:#c7f041; padding: 1em;">
4. Write another helper function `variable_search` that searches over all explanatory variables *and* cutpoints to find the explanatory variable and cutpoint that minimizes the RSS.

```{r}
# variable_search <- function(dataset) {
#   vars <- dataset %>% select(-Salary) %>% names()
#   map(
#     ___,
#     function(varname) {
#       var_sym <- sym(varname)
#       cutpoint(!!var_sym, dataset) %>%
#         mutate(var = varname)
#     }
#   ) %>%
#     list_rbind() %>%
#     slice_min(___, n = 1)
# }

# variable_search(train)
```

Interpretation: the first node in the tree will be ___ < ___. For players with ___, the average log salary is ___, and for players with ___, the average log salary is ___. This cutpoint yields an RSS of ___.
</div>

## Grow the Tree

<div style="background-color:#c7f041; border-width:3px; border-style:solid; border-color:#c7f041; padding: 1em;">
5. Drill into each region from `variable_search(train)`, growing a subtree. Stop growing a branch after it reaches a depth of 4 nodes. It will probably be useful to draw the tree on a sheet of paper as you're working.
</div>

## Compare your tree: `tree` package

<div style="background-color:#c7f041; border-width:3px; border-style:solid; border-color:#c7f041; padding: 1em;">
```{r, eval = F}
install.packages("tree")
```

```{r}
library(tree)

bb_tree <- tree(Salary ~ Hits + Years + Runs + HmRun, data = train)
bb_tree
bb_tree %>% summary()

plot(bb_tree)
text(bb_tree)
```
</div>
