Notes from the Video
Comparison Group Designs
- Naïve
- Basic Value-Added Design: Regression adjusted for a preprogram measure
- Regression adjusted covariate design
- Value-added design adjusted for additional covariates
- Interrupted time-series designs
Naive design - It could have been cause by anything else.
“Selection bias is the most pernicious form of bias that plagues evaluators’ attempts to accurately estimate program effects” (p. 144, Fraenkel et al., 2023).
Interrupted time-series
Regression Discontinuity
- [Youtube Video](<div class="video-card__frame">
</div>)
- A quasi-experimental approach that attempts to determine causal effects of an intervention by evaluating the differences of individuals just above and below the cutoff for the intervention.
- [Difference in Difference](<div class="video-card__frame">
</div>) - Prepost comparison between two groups.
- (Yta - Y+b) - (Yca - Yc) = DiD
-
diff-in-diff formula.png
- You need data on
- Treatment and control groups, covering pre and post treatment.
- Structure Data Options: Aggregate, pooled cross section or longitudinal data
Matching Design
- Be cautious when making causal claims.
- Counterfactual - What if there wasn’t an intervention?
- Matching Coursera
- What level are you matching at? - district, school, classroom, etc?
- Steps in Implementing Matching Methods
- Defining Closeness
- Distance Measures
- One Challenge
- Lose a lot of data with a small dataset.
- Nearest “Greedy” Matching Steps
- Matching up with someone not close because you run out of matches.
- Weight to a certain score.
- Propensity Score
- Summary of Matching
- “VIRTUAL TWINS”
- National Charter School Study 2013
- Reading - 2009
- 2013 - Reading (Continuing Schools)
- Math - 2009
- Is this a good design?
- What type of study design should you do?