12 Extra Credit Project

This extra credit project can mean up to +10 added to your final grade.

Ground Rules

  • I want to read your own thoughts and writing (even if the writing is not perfect), so if Canvas detects AI-generated text, you’ll get a zero.

  • This is not a project to work on in groups: if your report is extremely similar to another students’ report, you’ll both get zeros.

  • The main point is to do the replications: if yours is missing, you’ll get a zero.

Instructions

In a quarto document:

  • Step 1: Read these three articles:
      1. P-Hacking: How to (Not) Manipulate the P-Value: https://www.datacamp.com/tutorial/p-hacking
      1. Replicating Anomalies: https://global-q.org/uploads/1/2/2/6/122679606/houxuezhang2020rfs.pdf
      1. AI-Powered (Finance) Scholarship: https://www.nber.org/system/files/working_papers/w33363/w33363.pdf
  • Step 2: Answer these questions in 3-5 paragraphs in total:
      1. What is p-hacking, why is it tempting for researchers to use, and why is it unethical? Why is it that conducting replications is the best tool for detecting p-hacking?
      1. What did Robert Novy-Marx and Mihail Z. Veilkov do in their paper AI-Powered (Finance) Scholarship? Why does it matter and what does it say about p-hacking and the replication crisis in finance?
      1. In your opinion, what would the single best safeguard be to protect against the replication crisis in finance, especially when AI can instantly generate hundreds of polished papers with new statistically significant findings?
  • Step 3: Choose one of these 3 papers we replicated in class:
    • Jegadeesh and Titman (1993)
    • Da, Engelberg, and Gao (2015)
    • Hirshleifer and Shumway (2003)
    • Estimate the main empirical findings using data from as many different time periods that match the length of the original time period as possible. Then estimate the main empirical findings using data from all these time periods together.
    • Discuss whether the findings hold up when you replicate the analysis with each different time period. Do you think the relationship is real, or does it seem like the finding was an example of p-hacking?
  • Step 4: Upload your .qmd file rendered to html to the Canvas Discussion board for the extra credit project before the final exam (Friday 3/20 at 3:00 pm).
    • After you upload your file, you will be able to see other students’ submissions. Before Monday 3/23 at 5:00 pm, write at least two responses to classmates’ posts. In your responses, comment on things like:
    • differences between their approach and yours,
    • anything they may have overlooked,
    • something you find especially interesting,
    • and something they did especially well.

Grading Rubric (100 points in total: up to +10 added to your final grade)

1. Understanding of the Readings (30 points)

3–5 paragraphs that clearly answer the questions about p-hacking, the AI-Powered Scholarship paper, and safeguards against the replication crisis.

  • 30-25 points: Clear explanations and thoughtful connections.
  • 24-15 points: Mostly correct but somewhat incomplete or superficial.
  • 14-0: Major misunderstandings or missing answers.

2. Replication Analysis (40 points)

Replicate the main result from one assigned paper across multiple comparable time periods and using all periods combined.

  • 40-30 points: Correct replication with clear comparison across time periods.
  • 29-15 points: Partial replication or limited analysis.
  • 14-0 points: Parts of the replication is missing or unclear.

3. Interpretation (10 points)

Discuss whether the result appears real or possibly due to p-hacking.

  • 10-8 points: Thoughtful interpretation connected to the readings.
  • 7-4 points: Basic discussion.
  • 3-0: Little or no interpretation.

4. Quarto Document Quality (10 points)

  • 10-8 points: clear, organized, and reproducible.
  • 7-4 points: mostly clear.
  • 3-0 points: hard to follow or incomplete.

5. Canvas Discussion Responses (10 points)

  • 10-8 points: Two thoughtful responses to classmates’ work that engages with what they’ve done, citing specifics.
  • 7-4 points: Two responses that are brief or superficial.
  • 3–1 points: Only one response.
  • 0 points: No responses.