3.2 The Method of Maximum Likelihood

Understanding dnorm()

https://youtu.be/HJPBNSTdgwg

  1. Let \(X\) be a random variable with a normal distribution, mean 0, standard deviation 1 (\(X \sim N(0, 1)\)). Calculate the PDF of \(X\) evaluated at 0. To do this, use two methods to check your work: one method is to use dnorm() in R, and the other method is to use the equation for the pdf of the normal distribution: \(f(X) = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp \left( \frac{(X - \mu)^2}{-2\sigma^2} \right)\), where the mean \(\mu = 0\) and the variance \(\sigma^2 = 1\).

Answer:


  1. Use dnorm() to show that .0965 is the joint probability density for two independent random variables \(X_1\) and \(X_2\), both distributed \(N(0, 1)\), where \(X_1 = 0\) and \(X_2 = 1\).

Answer:

Maximum Likelihood

https://youtu.be/ZLnf6YFtmP0

  1. Again let \(X \sim N(\mu, 1)\). Suppose X was observed to take on the values 0 and 2. Find, using all the math steps for the method of maximum likelihood, the most likely value for \(\mu\).

Answer: \[\begin{align} \end{align}\]

Logit Estimation

https://youtu.be/OuYUcYmL6nM

  1. What is the log likelihood of seeing these six data points given \(\beta_0 = -1\) and \(\beta_1 = 0.05\)? Is it more or less likely than \(\beta_0 = 1\) and \(\beta_1 = 2\)? You can use my LL function:
LL <- function(params) {
  
  b0 <- params[1]
  b1 <- params[2]
  
  log(1 / (1 + exp(b0))) +
    log(1 / (1 + exp(b0 + b1 * 20))) +
    log(exp(b0 + b1 * 40) / (1 + exp(b0 + b1 * 40))) +
    log(exp(b0 + b1 * 60) / (1 + exp(b0 + b1 * 60))) +
    log(1 / (1 + exp(b0 + b1 * 80))) +
    log(exp(b0 + b1*100) / (1 + exp(b0 + b1 * 100)))
}

Answer:

To finish the logit estimation, we’ll let the function maxLik() handle searching for the values of \(\hat{\beta_0}\) and \(\hat{\beta_1}\) that give the largest likelihood of seeing the data. You’ll need to install maxLik before using it: install.packages("maxLik"), but make sure to not include code to install a package in your document: it will prevent the compilation of the document.

  1. maxLik takes as arguments a log likelihood function and a starting point for its search. Fill in the blank and run the code. Does our procedure match the logit that the function glm() finds?

Answer:

library(maxLik)
library(tidyverse)
tibble(
    discount = seq(0, 100, by = 20),
    purchase = c(0, 0, 1, 1, 0, 1)
) %>%
  glm(purchase ~ discount, data = ., family = "binomial") %>%
  broom::tidy(conf.int = TRUE)
# A tibble: 2 × 7
  term        estimate std.error statistic p.value conf.low conf.high
  <chr>          <dbl>     <dbl>     <dbl>   <dbl>    <dbl>     <dbl>
1 (Intercept)  -1.68      1.80      -0.937   0.349  -6.57       1.39 
2 discount      0.0337    0.0307     1.10    0.273  -0.0177     0.118


  1. Imperative programming practice: Complete Project Euler problem 2: even fibonacci numbers.

Answer: