Power Analysis & Simulation

Monte Carlo simulation for event study power analysis — estimate detection rates, calibrate study design, and assess test statistic performance.

Before running an event study, a natural question arises: will my test detect the effect if it exists? Power analysis via Monte Carlo simulation answers this by generating synthetic data with a known effect, running the event study many times, and counting how often the test correctly rejects the null.

Why Power Analysis?

Question Power Analysis Answers
Is my sample size sufficient? Rejection rate for given \(N\), effect size, and test
Which test is most powerful? Compare rejection rates across test statistics
How large must the effect be? Minimum detectable effect at 80% power
Does window length matter? Power as a function of event window width

Rule of thumb. A study should have at least 80% power (detect the effect 80% of the time when it exists). If power is below 50%, the study is more likely to miss the effect than find it.

Monte Carlo Simulation

The Idea

  1. Specify a data-generating process (DGP): Define how stock returns are generated, including the abnormal return you want to detect
  2. Simulate many datasets: Generate \(S\) synthetic datasets from the DGP
  3. Run the event study on each: Apply your chosen model and test statistic
  4. Count rejections: The fraction of simulations where the test rejects \(H_0\) is the estimated power

Running in R

# Power analysis: can we detect a 1% abnormal return with 20 firms?
sim_result <- simulate_event_study(
  n_events = 20,
  n_simulations = 500,
  abnormal_return = 0.01,
  estimation_window_length = 120,
  event_window = c(-2, 2),
  alpha = 0.05
)
sim_result
Event Study Simulation
  N events:        20 
  Event window:   [ -2 , 2 ]
  Abnormal return: 0.01 
  Test statistic:  CSectT 
  Alpha:           0.05 
  N simulations:   500 
  Power (day 0):   0.99

Parameters

Parameter Description Default
n_events Number of events (firms) in each simulation 20
n_simulations Number of Monte Carlo replications 1000
abnormal_return True abnormal return injected at event date 0
estimation_window_length Length of estimation window (trading days) 120
event_window Event window as c(start, end) c(-5, 5)
alpha Significance level 0.05
return_model Model to use (e.g., MarketModel$new()) Market Model
test_statistic Test to evaluate (e.g., "CSectT") "CSectT"
dgp_params Data-generating process parameters (list) list()

Power Curves

Vary one parameter at a time to see how power changes:

# Power as a function of sample size
sizes <- c(10, 20, 30, 50, 100)
power_by_n <- sapply(sizes, function(n) {
  sim <- simulate_event_study(
    n_events = n,
    n_simulations = 500,
    abnormal_return = 0.01
  )
  sim$power
})

data.frame(n_events = sizes, power = round(power_by_n, 3))
# Power as a function of effect size
effects <- c(0.005, 0.01, 0.015, 0.02, 0.03)
power_by_effect <- sapply(effects, function(ar) {
  sim <- simulate_event_study(
    n_events = 30,
    n_simulations = 500,
    abnormal_return = ar
  )
  sim$power
})

data.frame(abnormal_return = effects, power = round(power_by_effect, 3))

Size Analysis

Under the null (\(AR = 0\)), the rejection rate should equal \(\alpha\). If it exceeds \(\alpha\), the test over-rejects (inflated Type I error).

# Size check: rejection rate under H0
size_result <- simulate_event_study(
  n_events = 30,
  n_simulations = 1000,
  abnormal_return = 0,  # No effect
  alpha = 0.05
)
# Should be close to 0.05
size_result$rejection_rate
Outcome Interpretation Action
Rejection rate \(\approx\) 0.05 Test is correctly sized Safe to use
Rejection rate >> 0.05 Test over-rejects Use a robust alternative (e.g., BMP, Rank)
Rejection rate << 0.05 Test is conservative Lower power, but valid

Design Recommendations

Design Choice Impact on Power Recommendation
More firms Higher power Aim for \(N \geq 30\)
Longer estimation window Better variance estimate Use 120+ days
Narrower event window Less noise Use \([-1, +1]\) unless theory suggests wider
Lower \(\sigma\) (less volatile stocks) Higher power No control, but be aware of differences
BMP or Rank test Higher power under non-normality Use when residuals are non-normal

Literature

Next Steps