4.2 — Difference-in-Difference Models

ECON 480 • Econometrics • Fall 2021

Ryan Safner
Assistant Professor of Economics
safner@hood.edu
ryansafner/metricsF21
metricsF21.classes.ryansafner.com

Outline

Difference-in-Difference Models

Example II: "The" Card-Kreuger Minimum Wage Study

Clever Research Designs Identify Causality

Again, this toolkit of research designs to identify causal effects is the economist’s comparative advantage that firms and governments want!

Difference-in-Difference Models

Natural Experiments

Difference-in-Difference Models IOften, we want to examine the consequences of a change, such as a law or policy intervention

Difference-in-Difference Models I

Often, we want to examine the consequences of a change, such as a law or policy intervention
Example: how do States that implement policy see changes in
- Treatment: States that implement
- Control: States that did not implement

Difference-in-Difference Models I

Often, we want to examine the consequences of a change, such as a law or policy intervention
Example: how do States that implement policy see changes in
- Treatment: States that implement
- Control: States that did not implement
If we have panel data with observations for all states before and after the change...
Find the difference between treatment & control groups in their differences before and after the treatment period

Difference-in-Difference Models I

Often, we want to examine the consequences of a change, such as a law or policy intervention
Example: how do States that implement policy see changes in
- Treatment: States that implement
- Control: States that did not implement
If we have panel data with observations for all states before and after the change...
Find the difference between treatment & control groups in their differences before and after the treatment period

Difference-in-Difference Models I

Often, we want to examine the consequences of a change, such as a law or policy intervention
Example: how do States that implement policy see changes in
- Treatment: States that implement
- Control: States that did not implement
If we have panel data with observations for all states before and after the change...
Find the difference between treatment & control groups in their differences before and after the treatment period

Difference-in-Difference Models IIThe difference-in-difference model (aka “diff-in-diff” or “DND”) identifies treatment effect by differencing the difference pre- and post-treatment values of Y between treatment and control groups

Difference-in-Difference Models II

The difference-in-difference model (aka “diff-in-diff” or “DND”) identifies treatment effect by differencing the difference pre- and post-treatment values of between treatment and control groups

Difference-in-Difference Models II

The difference-in-difference model (aka “diff-in-diff” or “DND”) identifies treatment effect by differencing the difference pre- and post-treatment values of between treatment and control groups

Difference-in-Difference Models II

The difference-in-difference model (aka “diff-in-diff” or “DND”) identifies treatment effect by differencing the difference pre- and post-treatment values of between treatment and control groups

	Control	Treatment	Group Diff
Before
After
Time Diff			Diff-in-diff

Silly Example: Hot Dogs

Is there a discount when you get cheese and chili?

ABCDEFGHIJ0123456789

price <dbl>	cheese <dbl>	chili <dbl>
2.00	0	0
2.35	1	0
2.35	0	1
2.70	1	1

Silly Example: Hot Dogs

Is there a discount when you get cheese and chili?

ABCDEFGHIJ0123456789

price <dbl>	cheese <dbl>	chili <dbl>
2.00	0	0
2.35	1	0
2.35	0	1
2.70	1	1

lm(price ~ cheese + chili + cheese*chili,
   data = hotdogs) %>%
  tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>
(Intercept)	2.00
cheese	0.35
chili	0.35
cheese:chili	0.00

Silly Example: Hot Dogs

Is there a discount when you get cheese and chili?

	No Cheese	Cheese	Cheese Diff
No Chili	$2.00	$2.35	$0.35
Chili	$2.35	$2.70	$0.35
Chili Diff	$0.35	$0.35	$0.00 (Diff-in-diff)

lm(price ~ cheese + chili + cheese*chili,
   data = hotdogs) %>%
  tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>
(Intercept)	2.00
cheese	0.35
chili	0.35
cheese:chili	0.00

Silly Example: Hot Dogs

Is there a discount when you get cheese and chili?

	No Cheese	Cheese	Cheese Diff
No Chili	$2.00	$2.35	$0.35
Chili	$2.35	$2.70	$0.35
Chili Diff	$0.35	$0.35	$0.00 (Diff-in-diff)

lm(price ~ cheese + chili + cheese*chili,
   data = hotdogs) %>%
  tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>
(Intercept)	2.00
cheese	0.35
chili	0.35
cheese:chili	0.00

Diff-n-diff is just a model with an interaction term between two dummies!

Visualizing Diff-in-Diff

Control group
: value of for control group before treatment
: time difference (for control group)

Visualizing Diff-in-Diff

Control group

: value of for control group before treatment
: time difference (for control group)
Treatment group

Visualizing Diff-in-Diff

Control group

: value of for control group before treatment
: time difference (for control group)
Treatment group
: difference between groups before treatment

Visualizing Diff-in-Diff

Control group

: value of for control group before treatment
: time difference (for control group)
Treatment group
: difference between groups before treatment
: difference-in-difference (treatment effect)

Visualizing Diff-in-Diff II

for Control group before:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:
for Treatment group after:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:
for Treatment group after:
Group Difference (before):

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:
for Treatment group after:
Group Difference (before):
Time Difference:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:
for Treatment group after:
Group Difference (before):
Time Difference:
Difference-in-difference: (treatment effect)

Comparing Group Means (Again)

	Control	Treatment	Group Diff
Before
After
Time Diff			Diff-in-diff

Key Assumption: Counterfactual

Key assumption for DND: time trends (for treatment and control) are parallel
Treatment and control groups assumed to be identical over time on average, except for treatment
Counterfactual: if the treatment group had not recieved treatment, it would have changed identically over time as the control group

Key Assumption: Counterfactual

If the time-trends would have been different, a biased measure of the treatment effect !

Example I: HOPE in Georgia

Diff-in-Diff Example I

Example: In 1993 Georgia initiated a HOPE scholarship program to let state residents with at least a B average in high school attend public college in Georgia for free. Did it increase college enrollment?

Micro-level data on 4,291 young individuals

Diff-in-Diff Example I

Micro-level data on 4,291 young individuals

Diff-in-Diff Example I

Micro-level data on 4,291 young individuals

Diff-in-Diff Example I

Micro-level data on 4,291 young individuals

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Note: With a dummy dependent (Y) variable, coefficients estimate the probability , i.e. the probability a person is enrolled in college

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time
Treatment period: after 1992

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time
Treatment period: after 1992
Treatment: Georgia

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time
Treatment period: after 1992
Treatment: Georgia
Differences-in-differences:

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time
Treatment period: after 1992
Treatment: Georgia
Differences-in-differences:
Regression equation:

Example: Data

ABCDEFGHIJ0123456789

StateCode <fct>	Age <dbl>	Year <fct>	Weight <dbl>	Age18 <dbl>	LowIncome <dbl>	InCollege <dbl>
56	19	89	1396	0	1	1
56	19	89	1080	0	NA	1
56	18	89	924	1	1	1
56	19	89	891	0	0	1
56	19	89	1395	0	NA	0
56	18	89	1106	1	1	1
56	19	89	965	0	NA	0
56	18	89	958	1	NA	0
56	19	89	1006	0	NA	0
56	19	89	1183	0	1	1

Example: Data

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Example: RegressionDND_reg <- lm(InCollege ~ Georgia + After + Georgia*After, data = hope)
DND_reg %>% tidy()
ABCDEFGHIJ0123456789
term
<chr>
estimate
<dbl>
std.error
<dbl>
statistic
<dbl>
p.value
<dbl>
(Intercept)0.4057826520.0109239037.14631824.221545e-262
Georgia-0.1052362040.03778114-2.78541655.369384e-03
After-0.0044596090.01585224-0.28132357.784758e-01
Georgia:After0.0893298280.048893291.82703646.776378e-02
4 rows

  

Example: Regression

DND_reg <- lm(InCollege ~ Georgia + After + Georgia*After, data = hope)
DND_reg %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
(Intercept)	0.405782652	0.01092390	37.1463182	4.221545e-262
Georgia	-0.105236204	0.03778114	-2.7854165	5.369384e-03
After	-0.004459609	0.01585224	-0.2813235	7.784758e-01
Georgia:After	0.089329828	0.04889329	1.8270364	6.776378e-02

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
:

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
:

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
: After 1992, non-Georgians are 0.4% less likely to be college students

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
: After 1992, non-Georgians are 0.4% less likely to be college students
:

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
: After 1992, non-Georgians are 0.4% less likely to be college students
: After 1992, Georgians are 8.9% more likely to enroll in colleges than neighboring states

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
: After 1992, non-Georgians are 0.4% less likely to be college students
: After 1992, Georgians are 8.9% more likely to enroll in colleges than neighboring states
Treatment effect: HOPE increased enrollment likelihood by 8.9%

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:
Non-Georgian enrollment probability post-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:
Non-Georgian enrollment probability post-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:
Non-Georgian enrollment probability post-1992:
Georgian enrollment probability post-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:
Non-Georgian enrollment probability post-1992:
Georgian enrollment probability post-1992:

Example: Comparing Group Means in R# group mean for non-Georgian before 1992
hope %>%
  filter(Georgia == 0,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.4057827
1 row

  

prob <dbl>
0.4057827

Example: Comparing Group Means in R# group mean for non-Georgian before 1992
hope %>%
  filter(Georgia == 0,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.4057827
1 row
# group mean for non-Georgian AFTER 1992
hope %>%
  filter(Georgia == 0,
         After == 1) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.401323
1 row

  

prob <dbl>
0.4057827

prob <dbl>
0.401323

Example: Comparing Group Means in R II# group mean for Georgian before 1992
hope %>%
  filter(Georgia == 1,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.3005464
1 row

  

prob <dbl>
0.3005464

Example: Comparing Group Means in R II# group mean for Georgian before 1992
hope %>%
  filter(Georgia == 1,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.3005464
1 row
# group mean for Georgian AFTER 1992
hope %>%
  filter(Georgia == 1,
         After == 1) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.3854167
1 row

  

prob <dbl>
0.3005464

prob <dbl>
0.3854167

Example: Diff-in-Diff Summary

	Neighbors	Georgia	Group Diff
Before
After
Time Diff			Diff-in-diff:

Example: Diff-in-Diff Summary

	Neighbors	Georgia	Group Diff
Before
After
Time Diff			Diff-in-diff:

Diff-in-Diff Summary & Data

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Example: Diff-in-Diff Graph

Example: Diff-in-Diff Graph (& Counterfactual)

Generalizing DND Models

Generalizing DND ModelsDND can be generalized with a two-way fixed effects model:
^Yit=β1(Treatedi×Aftert)+αi+θt++νit
αi: group fixed effects (treatments/control groups)
θt: time fixed effects (pre/post treatment)
β1: diff-in-diff (interaction effect, β3 from before)

  

Generalizing DND ModelsDND can be generalized with a two-way fixed effects model:
^Yit=β1(Treatedi×Aftert)+αi+θt++νit
αi: group fixed effects (treatments/control groups)
θt: time fixed effects (pre/post treatment)
β1: diff-in-diff (interaction effect, β3 from before)
Flexibility: many periods (not just before/after), many different treatment(s)/groups, and treatment(s) can occur at different times to different units (so long as some do not get treated)

  

Generalizing DND Models

DND can be generalized with a two-way fixed effects model:
^Yit=β1(Treatedi×Aftert)+αi+θt++νit
- : group fixed effects (treatments/control groups)
- : time fixed effects (pre/post treatment)
- : diff-in-diff (interaction effect, from before)

Flexibility: many periods (not just before/after), many different treatment(s)/groups, and treatment(s) can occur at different times to different units (so long as some do not get treated)
Can also add control variables that vary within units and over time

Our Example, Generalized I

StateCode is a variable for the State create State fixed effect
Year is a variable for the year create year fixed effect

Our Example, Generalized II

Using LSDV method...

DND_fe <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year),
           data = hope)
DND_fe %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>
(Intercept)	0.418057478	0.02261133
Georgia	-0.141501255	0.03936119
After	0.075340594	0.03128021
factor(StateCode)57	-0.014181112	0.02739708
factor(StateCode)58	NA	NA
factor(StateCode)59	-0.062378540	0.01954266
factor(StateCode)62	-0.132650271	0.02806143
factor(StateCode)63	-0.005103868	0.02627780
factor(Year)90	0.046608845	0.02833625
factor(Year)91	0.032275789	0.02856877

Our Example, Generalized II

Using LSDV method...

DND_fe <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year),
           data = hope)
DND_fe %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>
(Intercept)	0.418057478	0.02261133
Georgia	-0.141501255	0.03936119
After	0.075340594	0.03128021
factor(StateCode)57	-0.014181112	0.02739708
factor(StateCode)58	NA	NA
factor(StateCode)59	-0.062378540	0.01954266
factor(StateCode)62	-0.132650271	0.02806143
factor(StateCode)63	-0.005103868	0.02627780
factor(Year)90	0.046608845	0.02833625
factor(Year)91	0.032275789	0.02856877

By de-meaning data, using fixest

library(fixest)
DND_fe_2 <- feols(InCollege ~ Georgia*After | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_2 %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>
Georgia:After	0.0914202	0.005643298	16.19978

Our Example, Generalized II

Using LSDV method...

DND_fe <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year),
           data = hope)
DND_fe %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>
(Intercept)	0.418057478	0.02261133
Georgia	-0.141501255	0.03936119
After	0.075340594	0.03128021
factor(StateCode)57	-0.014181112	0.02739708
factor(StateCode)58	NA	NA
factor(StateCode)59	-0.062378540	0.01954266
factor(StateCode)62	-0.132650271	0.02806143
factor(StateCode)63	-0.005103868	0.02627780
factor(Year)90	0.046608845	0.02833625
factor(Year)91	0.032275789	0.02856877

By de-meaning data, using fixest

library(fixest)
DND_fe_2 <- feols(InCollege ~ Georgia*After | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_2 %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>
Georgia:After	0.0914202	0.005643298	16.19978

Our Example, Generalized, with Controls II

Using LSDV method...

DND_fe_controls <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year) + Black + LowIncome,
           data = hope)
DND_fe_controls %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>
(Intercept)	0.735574222	0.02990710
Georgia	-0.108940276	0.04765262
After	-0.005753553	0.03737027
factor(StateCode)57	-0.043406073	0.03047696
factor(StateCode)58	NA	NA
factor(StateCode)59	-0.053175645	0.02306160
factor(StateCode)62	-0.116104615	0.03283310
factor(StateCode)63	0.007389866	0.03056444
factor(Year)90	0.039364315	0.03326291
factor(Year)91	0.029227969	0.03347850

Our Example, Generalized, with Controls II

Using LSDV method...

DND_fe_controls <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year) + Black + LowIncome,
           data = hope)
DND_fe_controls %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>
(Intercept)	0.735574222	0.02990710
Georgia	-0.108940276	0.04765262
After	-0.005753553	0.03737027
factor(StateCode)57	-0.043406073	0.03047696
factor(StateCode)58	NA	NA
factor(StateCode)59	-0.053175645	0.02306160
factor(StateCode)62	-0.116104615	0.03283310
factor(StateCode)63	0.007389866	0.03056444
factor(Year)90	0.039364315	0.03326291
factor(Year)91	0.029227969	0.03347850

By de-meaning data, using fixest

library(fixest)
DND_fe_controls_2 <- feols(InCollege ~ Georgia*After + Black + LowIncome | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_controls_2 %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>
Black	-0.09398715	0.01273233	-7.381769
LowIncome	-0.30172426	0.03066188	-9.840369
Georgia:After	0.02343679	0.01281838	1.828374

Our Example, Generalized, with Controls II

Using LSDV method...

DND_fe_controls <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year) + Black + LowIncome,
           data = hope)
DND_fe_controls %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>
(Intercept)	0.735574222	0.02990710
Georgia	-0.108940276	0.04765262
After	-0.005753553	0.03737027
factor(StateCode)57	-0.043406073	0.03047696
factor(StateCode)58	NA	NA
factor(StateCode)59	-0.053175645	0.02306160
factor(StateCode)62	-0.116104615	0.03283310
factor(StateCode)63	0.007389866	0.03056444
factor(Year)90	0.039364315	0.03326291
factor(Year)91	0.029227969	0.03347850

By de-meaning data, using fixest

library(fixest)
DND_fe_controls_2 <- feols(InCollege ~ Georgia*After + Black + LowIncome | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_controls_2 %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>
Black	-0.09398715	0.01273233	-7.381769
LowIncome	-0.30172426	0.03066188	-9.840369
Georgia:After	0.02343679	0.01281838	1.828374

Our Example, Generalized, with Controls II

(1)(2)(3)

Intercept0.4058 ***                  

(0.0109)                     

Georgia-0.1052 **                   

(0.0378)                     

After-0.0045                      

(0.0159)                     

Georgia x After0.0893    0.0914 ***0.0234    

(0.0489)   (0.0056)   (0.0128)   

Low Income                  -0.3017 ***

                  (0.0307)   

Black                  -0.0940 ***

                  (0.0127)   

Fixed EffectsNone         State & Year         State & Year         

N4291         4291         2967         

R-Squared0.0019    0.0115    0.1036    

SER0.4893    0.4875    0.4731    

 *** p < 0.001;  ** p < 0.01;  * p < 0.05.

Our Example, Generalized, with Controls II

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Intuition behind DND

Diff-in-diff models are the quintessential example of exploiting natural experiments
A major change at a point in time (change in law, a natural disaster, political crisis) separates groups where one is affected and another is not---identifies the effect of the change (treatment)
One of the cleanest and clearest causal identification strategies

Example II: “The” Card-Kreuger Minimum Wage Study

Example: ”The” Card-Kreuger Minimum Wage Study I

Example: The controversial minimum wage study, Card & Kreuger (1994) is a quintessential (and clever) diff-in-diff.

Card, David, Krueger, Alan B, (1994), "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review 84 (4): 772–793

Card & Kreuger (1994): Background I

Card & Kreuger (1994) compare employment in fast food restaurants on New Jersey and Pennsylvania sides of border between February and November 1992.
Pennsylvania & New Jersey both had a minimum wage of $4.25 before February 1992
In February 1992, New Jersey raised minimum wage from $4.25 to $5.05

Card & Kreuger (1994): Background II

If we look only at New Jersey before & after change:
- Omitted variable bias: macroeconomic variables (there's a recession!), weather, etc.
- Including PA as a control will control for these time-varying effects if they are national trends
Surveyed 400 fast food restaurants on each side of the border, before & after min wage increase
- Key assumption: Pennsylvania and New Jersey follow parallel trends,
- Counterfactual: if not for the minimum wage increase, NJ employment would have changed similar to PA employment

Card & Kreuger (1994): Comparisons

Card & Kreuger (1994): Summary I

Card & Kreuger (1994): Summary II

Card & Kreuger (1994): Model

PA Before:
PA After:
NJ Before:
NJ After:
Diff-in-diff:

Card & Kreuger (1994): Model

PA Before:
PA After:
NJ Before:
NJ After:
Diff-in-diff:

	PA	NJ	Group Diff
Before
After
Time Diff			Diff-in-diff

Card & Kreuger (1994): Results

Help

Keyboard shortcuts

↑, ←, Pg Up, k

Go to previous slide

↓, →, Pg Dn, Space, j

Go to next slide

Home

Go to first slide

End

Go to last slide

Number + Return

Go to specific slide

b / m / f

Toggle blackout / mirrored / fullscreen mode

Clone slideshow

Toggle presenter mode

Restart the presentation timer

?, h

Toggle this help

Tile View: Overview of Slides

4.2 — Difference-in-Difference Models

ECON 480 • Econometrics • Fall 2021

Ryan Safner
Assistant Professor of Economics
safner@hood.edu
ryansafner/metricsF21
metricsF21.classes.ryansafner.com

Outline

Difference-in-Difference Models

Example I: HOPE in Georgia

Generalizing DND Models

Example II: "The" Card-Kreuger Minimum Wage Study

Clever Research Designs Identify Causality

Again, this toolkit of research designs to identify causal effects is the economist’s comparative advantage that firms and governments want!

Difference-in-Difference Models

Natural Experiments

Difference-in-Difference Models IOften, we want to examine the consequences of a change, such as a law or policy intervention

Difference-in-Difference Models I

Often, we want to examine the consequences of a change, such as a law or policy intervention
Example: how do States that implement policy see changes in
- Treatment: States that implement
- Control: States that did not implement

Difference-in-Difference Models I

Often, we want to examine the consequences of a change, such as a law or policy intervention
Example: how do States that implement policy see changes in
- Treatment: States that implement
- Control: States that did not implement
If we have panel data with observations for all states before and after the change...
Find the difference between treatment & control groups in their differences before and after the treatment period

Difference-in-Difference Models I

Often, we want to examine the consequences of a change, such as a law or policy intervention
Example: how do States that implement policy see changes in
- Treatment: States that implement
- Control: States that did not implement
If we have panel data with observations for all states before and after the change...
Find the difference between treatment & control groups in their differences before and after the treatment period

Difference-in-Difference Models I

Often, we want to examine the consequences of a change, such as a law or policy intervention
Example: how do States that implement policy see changes in
- Treatment: States that implement
- Control: States that did not implement
If we have panel data with observations for all states before and after the change...
Find the difference between treatment & control groups in their differences before and after the treatment period

Difference-in-Difference Models IIThe difference-in-difference model (aka “diff-in-diff” or “DND”) identifies treatment effect by differencing the difference pre- and post-treatment values of Y between treatment and control groups

Difference-in-Difference Models II

The difference-in-difference model (aka “diff-in-diff” or “DND”) identifies treatment effect by differencing the difference pre- and post-treatment values of between treatment and control groups

Difference-in-Difference Models II

The difference-in-difference model (aka “diff-in-diff” or “DND”) identifies treatment effect by differencing the difference pre- and post-treatment values of between treatment and control groups

Difference-in-Difference Models II

The difference-in-difference model (aka “diff-in-diff” or “DND”) identifies treatment effect by differencing the difference pre- and post-treatment values of between treatment and control groups

	Control	Treatment	Group Diff
Before
After
Time Diff			Diff-in-diff

Silly Example: Hot Dogs

Is there a discount when you get cheese and chili?

ABCDEFGHIJ0123456789

price <dbl>	cheese <dbl>	chili <dbl>
2.00	0	0
2.35	1	0
2.35	0	1
2.70	1	1

Silly Example: Hot Dogs

Is there a discount when you get cheese and chili?

ABCDEFGHIJ0123456789

price <dbl>	cheese <dbl>	chili <dbl>
2.00	0	0
2.35	1	0
2.35	0	1
2.70	1	1

lm(price ~ cheese + chili + cheese*chili,
   data = hotdogs) %>%
  tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>
(Intercept)	2.00
cheese	0.35
chili	0.35
cheese:chili	0.00

Silly Example: Hot Dogs

Is there a discount when you get cheese and chili?

	No Cheese	Cheese	Cheese Diff
No Chili	$2.00	$2.35	$0.35
Chili	$2.35	$2.70	$0.35
Chili Diff	$0.35	$0.35	$0.00 (Diff-in-diff)

lm(price ~ cheese + chili + cheese*chili,
   data = hotdogs) %>%
  tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>
(Intercept)	2.00
cheese	0.35
chili	0.35
cheese:chili	0.00

Silly Example: Hot Dogs

Is there a discount when you get cheese and chili?

	No Cheese	Cheese	Cheese Diff
No Chili	$2.00	$2.35	$0.35
Chili	$2.35	$2.70	$0.35
Chili Diff	$0.35	$0.35	$0.00 (Diff-in-diff)

lm(price ~ cheese + chili + cheese*chili,
   data = hotdogs) %>%
  tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>
(Intercept)	2.00
cheese	0.35
chili	0.35
cheese:chili	0.00

Diff-n-diff is just a model with an interaction term between two dummies!

Visualizing Diff-in-Diff

Control group
: value of for control group before treatment
: time difference (for control group)

Visualizing Diff-in-Diff

Control group

: value of for control group before treatment
: time difference (for control group)
Treatment group

Visualizing Diff-in-Diff

Control group

: value of for control group before treatment
: time difference (for control group)
Treatment group
: difference between groups before treatment

Visualizing Diff-in-Diff

Control group

: value of for control group before treatment
: time difference (for control group)
Treatment group
: difference between groups before treatment
: difference-in-difference (treatment effect)

Visualizing Diff-in-Diff II

for Control group before:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:
for Treatment group after:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:
for Treatment group after:
Group Difference (before):

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:
for Treatment group after:
Group Difference (before):
Time Difference:

Visualizing Diff-in-Diff II

for Control group before:
for Control group after:
for Treatment group before:
for Treatment group after:
Group Difference (before):
Time Difference:
Difference-in-difference: (treatment effect)

Comparing Group Means (Again)

	Control	Treatment	Group Diff
Before
After
Time Diff			Diff-in-diff

Key Assumption: Counterfactual

Key assumption for DND: time trends (for treatment and control) are parallel
Treatment and control groups assumed to be identical over time on average, except for treatment
Counterfactual: if the treatment group had not recieved treatment, it would have changed identically over time as the control group

Key Assumption: Counterfactual

If the time-trends would have been different, a biased measure of the treatment effect !

Example I: HOPE in Georgia

Diff-in-Diff Example I

Micro-level data on 4,291 young individuals

Diff-in-Diff Example I

Micro-level data on 4,291 young individuals

Diff-in-Diff Example I

Micro-level data on 4,291 young individuals

Diff-in-Diff Example I

Micro-level data on 4,291 young individuals

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Note: With a dummy dependent (Y) variable, coefficients estimate the probability , i.e. the probability a person is enrolled in college

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time
Treatment period: after 1992

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time
Treatment period: after 1992
Treatment: Georgia

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time
Treatment period: after 1992
Treatment: Georgia
Differences-in-differences:

Diff-in-Diff Example II

We can use a DND model to measure the effect of HOPE scholarship on enrollments
Georgia and nearby States, if not for HOPE, changes should be the same over time
Treatment period: after 1992
Treatment: Georgia
Differences-in-differences:
Regression equation:

Example: Data

ABCDEFGHIJ0123456789

StateCode <fct>	Age <dbl>	Year <fct>	Weight <dbl>	Age18 <dbl>
56	19	89	1396	0
56	19	89	1080	0
56	18	89	924	1
56	19	89	891	0
56	19	89	1395	0
56	18	89	1106	1
56	19	89	965	0
56	18	89	958	1
56	19	89	1006	0
56	19	89	1183	0

Example: Data

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Example: RegressionDND_reg <- lm(InCollege ~ Georgia + After + Georgia*After, data = hope)
DND_reg %>% tidy()
ABCDEFGHIJ0123456789
term
<chr>
estimate
<dbl>
std.error
<dbl>
statistic
<dbl>
p.value
<dbl>
(Intercept)0.4057826520.0109239037.14631824.221545e-262
Georgia-0.1052362040.03778114-2.78541655.369384e-03
After-0.0044596090.01585224-0.28132357.784758e-01
Georgia:After0.0893298280.048893291.82703646.776378e-02
4 rows

  

Example: Regression

DND_reg <- lm(InCollege ~ Georgia + After + Georgia*After, data = hope)
DND_reg %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
(Intercept)	0.405782652	0.01092390	37.1463182	4.221545e-262
Georgia	-0.105236204	0.03778114	-2.7854165	5.369384e-03
After	-0.004459609	0.01585224	-0.2813235	7.784758e-01
Georgia:After	0.089329828	0.04889329	1.8270364	6.776378e-02

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
:

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
:

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
: After 1992, non-Georgians are 0.4% less likely to be college students

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
: After 1992, non-Georgians are 0.4% less likely to be college students
:

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
: After 1992, non-Georgians are 0.4% less likely to be college students
: After 1992, Georgians are 8.9% more likely to enroll in colleges than neighboring states

Example: Interpretting the Regression

: A non-Georgian before 1992 was 40.6% likely to be a college student
: Georgians before 1992 were 10.5% less likely to be college students than neighboring states
: After 1992, non-Georgians are 0.4% less likely to be college students
: After 1992, Georgians are 8.9% more likely to enroll in colleges than neighboring states
Treatment effect: HOPE increased enrollment likelihood by 8.9%

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:
Non-Georgian enrollment probability post-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:
Non-Georgian enrollment probability post-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:
Non-Georgian enrollment probability post-1992:
Georgian enrollment probability post-1992:

Example: Comparing Group Means

A group mean for a dummy is , i.e. the probability a student is enrolled:
Non-Georgian enrollment probability pre-1992:
Georgian enrollment probability pre-1992:
Non-Georgian enrollment probability post-1992:
Georgian enrollment probability post-1992:

Example: Comparing Group Means in R# group mean for non-Georgian before 1992
hope %>%
  filter(Georgia == 0,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.4057827
1 row

  

Example: Comparing Group Means in R# group mean for non-Georgian before 1992
hope %>%
  filter(Georgia == 0,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.4057827
1 row
# group mean for non-Georgian AFTER 1992
hope %>%
  filter(Georgia == 0,
         After == 1) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.401323
1 row

  

Example: Comparing Group Means in R II# group mean for Georgian before 1992
hope %>%
  filter(Georgia == 1,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.3005464
1 row

  

Example: Comparing Group Means in R II# group mean for Georgian before 1992
hope %>%
  filter(Georgia == 1,
         After == 0) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.3005464
1 row
# group mean for Georgian AFTER 1992
hope %>%
  filter(Georgia == 1,
         After == 1) %>%
  summarize(prob = mean(InCollege))
ABCDEFGHIJ0123456789
prob
<dbl>
0.3854167
1 row

  

Example: Diff-in-Diff Summary

	Neighbors	Georgia	Group Diff
Before
After
Time Diff			Diff-in-diff:

Example: Diff-in-Diff Summary

	Neighbors	Georgia	Group Diff
Before
After
Time Diff			Diff-in-diff:

Diff-in-Diff Summary & Data

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Example: Diff-in-Diff Graph

Example: Diff-in-Diff Graph (& Counterfactual)

Generalizing DND Models

Generalizing DND ModelsDND can be generalized with a two-way fixed effects model:
^Yit=β1(Treatedi×Aftert)+αi+θt++νitαi: group fixed effects (treatments/control groups)
θt: time fixed effects (pre/post treatment)
β1: diff-in-diff (interaction effect, β3 from before)

Generalizing DND ModelsDND can be generalized with a two-way fixed effects model:
^Yit=β1(Treatedi×Aftert)+αi+θt++νitαi: group fixed effects (treatments/control groups)
θt: time fixed effects (pre/post treatment)
β1: diff-in-diff (interaction effect, β3 from before)

Flexibility: many periods (not just before/after), many different treatment(s)/groups, and treatment(s) can occur at different times to different units (so long as some do not get treated)

  

Generalizing DND Models

DND can be generalized with a two-way fixed effects model:
^Yit=β1(Treatedi×Aftert)+αi+θt++νit
- : group fixed effects (treatments/control groups)
- : time fixed effects (pre/post treatment)
- : diff-in-diff (interaction effect, from before)

Flexibility: many periods (not just before/after), many different treatment(s)/groups, and treatment(s) can occur at different times to different units (so long as some do not get treated)
Can also add control variables that vary within units and over time

Our Example, Generalized I

StateCode is a variable for the State create State fixed effect
Year is a variable for the year create year fixed effect

Our Example, Generalized II

Using LSDV method...

DND_fe <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year),
           data = hope)
DND_fe %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
(Intercept)	0.418057478	0.02261133	18.4888517	1.734550e-73
Georgia	-0.141501255	0.03936119	-3.5949436	3.281224e-04
After	0.075340594	0.03128021	2.4085706	1.605717e-02
factor(StateCode)57	-0.014181112	0.02739708	-0.5176140	6.047544e-01
factor(StateCode)58	NA	NA	NA	NA
factor(StateCode)59	-0.062378540	0.01954266	-3.1919172	1.423556e-03
factor(StateCode)62	-0.132650271	0.02806143	-4.7271383	2.350298e-06
factor(StateCode)63	-0.005103868	0.02627780	-0.1942274	8.460071e-01
factor(Year)90	0.046608845	0.02833625	1.6448486	1.000745e-01
factor(Year)91	0.032275789	0.02856877	1.1297577	2.586417e-01

Our Example, Generalized II

Using LSDV method...

DND_fe <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year),
           data = hope)
DND_fe %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
(Intercept)	0.418057478	0.02261133	18.4888517	1.734550e-73
Georgia	-0.141501255	0.03936119	-3.5949436	3.281224e-04
After	0.075340594	0.03128021	2.4085706	1.605717e-02
factor(StateCode)57	-0.014181112	0.02739708	-0.5176140	6.047544e-01
factor(StateCode)58	NA	NA	NA	NA
factor(StateCode)59	-0.062378540	0.01954266	-3.1919172	1.423556e-03
factor(StateCode)62	-0.132650271	0.02806143	-4.7271383	2.350298e-06
factor(StateCode)63	-0.005103868	0.02627780	-0.1942274	8.460071e-01
factor(Year)90	0.046608845	0.02833625	1.6448486	1.000745e-01
factor(Year)91	0.032275789	0.02856877	1.1297577	2.586417e-01

By de-meaning data, using fixest

library(fixest)
DND_fe_2 <- feols(InCollege ~ Georgia*After | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_2 %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
Georgia:After	0.0914202	0.005643298	16.19978	1.633762e-05

Our Example, Generalized II

Using LSDV method...

DND_fe <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year),
           data = hope)
DND_fe %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
(Intercept)	0.418057478	0.02261133	18.4888517	1.734550e-73
Georgia	-0.141501255	0.03936119	-3.5949436	3.281224e-04
After	0.075340594	0.03128021	2.4085706	1.605717e-02
factor(StateCode)57	-0.014181112	0.02739708	-0.5176140	6.047544e-01
factor(StateCode)58	NA	NA	NA	NA
factor(StateCode)59	-0.062378540	0.01954266	-3.1919172	1.423556e-03
factor(StateCode)62	-0.132650271	0.02806143	-4.7271383	2.350298e-06
factor(StateCode)63	-0.005103868	0.02627780	-0.1942274	8.460071e-01
factor(Year)90	0.046608845	0.02833625	1.6448486	1.000745e-01
factor(Year)91	0.032275789	0.02856877	1.1297577	2.586417e-01

By de-meaning data, using fixest

library(fixest)
DND_fe_2 <- feols(InCollege ~ Georgia*After | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_2 %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
Georgia:After	0.0914202	0.005643298	16.19978	1.633762e-05

Our Example, Generalized, with Controls II

Using LSDV method...

DND_fe_controls <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year) + Black + LowIncome,
           data = hope)
DND_fe_controls %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
(Intercept)	0.735574222	0.02990710	24.5953037	1.155308e-121
Georgia	-0.108940276	0.04765262	-2.2861342	2.231699e-02
After	-0.005753553	0.03737027	-0.1539607	8.776512e-01
factor(StateCode)57	-0.043406073	0.03047696	-1.4242257	1.544869e-01
factor(StateCode)58	NA	NA	NA	NA
factor(StateCode)59	-0.053175645	0.02306160	-2.3058092	2.119033e-02
factor(StateCode)62	-0.116104615	0.03283310	-3.5362060	4.121675e-04
factor(StateCode)63	0.007389866	0.03056444	0.2417799	8.089675e-01
factor(Year)90	0.039364315	0.03326291	1.1834297	2.367342e-01
factor(Year)91	0.029227969	0.03347850	0.8730370	3.827140e-01

Our Example, Generalized, with Controls II

Using LSDV method...

DND_fe_controls <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year) + Black + LowIncome,
           data = hope)
DND_fe_controls %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
(Intercept)	0.735574222	0.02990710	24.5953037	1.155308e-121
Georgia	-0.108940276	0.04765262	-2.2861342	2.231699e-02
After	-0.005753553	0.03737027	-0.1539607	8.776512e-01
factor(StateCode)57	-0.043406073	0.03047696	-1.4242257	1.544869e-01
factor(StateCode)58	NA	NA	NA	NA
factor(StateCode)59	-0.053175645	0.02306160	-2.3058092	2.119033e-02
factor(StateCode)62	-0.116104615	0.03283310	-3.5362060	4.121675e-04
factor(StateCode)63	0.007389866	0.03056444	0.2417799	8.089675e-01
factor(Year)90	0.039364315	0.03326291	1.1834297	2.367342e-01
factor(Year)91	0.029227969	0.03347850	0.8730370	3.827140e-01

By de-meaning data, using fixest

library(fixest)
DND_fe_controls_2 <- feols(InCollege ~ Georgia*After + Black + LowIncome | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_controls_2 %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
Black	-0.09398715	0.01273233	-7.381769	0.0007172767
LowIncome	-0.30172426	0.03066188	-9.840369	0.0001846526
Georgia:After	0.02343679	0.01281838	1.828374	0.1270331132

Our Example, Generalized, with Controls II

Using LSDV method...

DND_fe_controls <- lm(InCollege ~ Georgia*After + factor(StateCode) + factor(Year) + Black + LowIncome,
           data = hope)
DND_fe_controls %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
(Intercept)	0.735574222	0.02990710	24.5953037	1.155308e-121
Georgia	-0.108940276	0.04765262	-2.2861342	2.231699e-02
After	-0.005753553	0.03737027	-0.1539607	8.776512e-01
factor(StateCode)57	-0.043406073	0.03047696	-1.4242257	1.544869e-01
factor(StateCode)58	NA	NA	NA	NA
factor(StateCode)59	-0.053175645	0.02306160	-2.3058092	2.119033e-02
factor(StateCode)62	-0.116104615	0.03283310	-3.5362060	4.121675e-04
factor(StateCode)63	0.007389866	0.03056444	0.2417799	8.089675e-01
factor(Year)90	0.039364315	0.03326291	1.1834297	2.367342e-01
factor(Year)91	0.029227969	0.03347850	0.8730370	3.827140e-01

By de-meaning data, using fixest

library(fixest)
DND_fe_controls_2 <- feols(InCollege ~ Georgia*After + Black + LowIncome | factor(StateCode) + factor(Year),
           data = hope)
DND_fe_controls_2 %>% tidy()

ABCDEFGHIJ0123456789

term <chr>	estimate <dbl>	std.error <dbl>	statistic <dbl>	p.value <dbl>
Black	-0.09398715	0.01273233	-7.381769	0.0007172767
LowIncome	-0.30172426	0.03066188	-9.840369	0.0001846526
Georgia:After	0.02343679	0.01281838	1.828374	0.1270331132

Our Example, Generalized, with Controls II

(1)(2)(3)

Intercept0.4058 ***                  

(0.0109)                     

Georgia-0.1052 **                   

(0.0378)                     

After-0.0045                      

(0.0159)                     

Georgia x After0.0893    0.0914 ***0.0234    

(0.0489)   (0.0056)   (0.0128)   

Low Income                  -0.3017 ***

                  (0.0307)   

Black                  -0.0940 ***

                  (0.0127)   

Fixed EffectsNone         State & Year         State & Year         

N4291         4291         2967         

R-Squared0.0019    0.0115    0.1036    

SER0.4893    0.4875    0.4731    

 *** p < 0.001;  ** p < 0.01;  * p < 0.05.

Our Example, Generalized, with Controls II

Dynarski, Susan, 1999, “Hope for Whom? Financial Aid for the Middle Class and its Impact on College Attendance,” National Tax Journal 53(3): 629-661

Intuition behind DND

Diff-in-diff models are the quintessential example of exploiting natural experiments
A major change at a point in time (change in law, a natural disaster, political crisis) separates groups where one is affected and another is not---identifies the effect of the change (treatment)
One of the cleanest and clearest causal identification strategies

Example II: “The” Card-Kreuger Minimum Wage Study

Example: ”The” Card-Kreuger Minimum Wage Study I

Example: The controversial minimum wage study, Card & Kreuger (1994) is a quintessential (and clever) diff-in-diff.

Card, David, Krueger, Alan B, (1994), "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review 84 (4): 772–793

Card & Kreuger (1994): Background I

Card & Kreuger (1994) compare employment in fast food restaurants on New Jersey and Pennsylvania sides of border between February and November 1992.
Pennsylvania & New Jersey both had a minimum wage of $4.25 before February 1992
In February 1992, New Jersey raised minimum wage from $4.25 to $5.05

Card & Kreuger (1994): Background II

If we look only at New Jersey before & after change:
- Omitted variable bias: macroeconomic variables (there's a recession!), weather, etc.
- Including PA as a control will control for these time-varying effects if they are national trends
Surveyed 400 fast food restaurants on each side of the border, before & after min wage increase
- Key assumption: Pennsylvania and New Jersey follow parallel trends,
- Counterfactual: if not for the minimum wage increase, NJ employment would have changed similar to PA employment

Card & Kreuger (1994): Comparisons

Card & Kreuger (1994): Summary I

Card & Kreuger (1994): Summary II

Card & Kreuger (1994): Model

PA Before:
PA After:
NJ Before:
NJ After:
Diff-in-diff:

Card & Kreuger (1994): Model

PA Before:
PA After:
NJ Before:
NJ After:
Diff-in-diff:

	PA	NJ	Group Diff
Before
After
Time Diff			Diff-in-diff

Card & Kreuger (1994): Results

	(1)	(2)	(3)
Intercept	0.4058 ***
	(0.0109)
Georgia	-0.1052 **
	(0.0378)
After	-0.0045
	(0.0159)
Georgia x After	0.0893	0.0914 ***	0.0234
	(0.0489)	(0.0056)	(0.0128)
Low Income			-0.3017 ***
			(0.0307)
Black			-0.0940 ***
			(0.0127)
Fixed Effects	None	State & Year	State & Year
N	4291	4291	2967
R-Squared	0.0019	0.0115	0.1036
SER	0.4893	0.4875	0.4731
* p < 0.001; p < 0.01; * p < 0.05.

4.2 — Difference-in-Difference Models

ECON 480 • Econometrics • Fall 2021

Ryan Safner Assistant Professor of Economics safner@hood.edu ryansafner/metricsF21 metricsF21.classes.ryansafner.com

Outline

Clever Research Designs Identify Causality

Difference-in-Difference Models

Natural Experiments

Difference-in-Difference Models I

Difference-in-Difference Models I

Difference-in-Difference Models I

Difference-in-Difference Models I

Difference-in-Difference Models I

Difference-in-Difference Models II

Difference-in-Difference Models II

Difference-in-Difference Models II

Difference-in-Difference Models II

Silly Example: Hot Dogs

Silly Example: Hot Dogs

Silly Example: Hot Dogs

Silly Example: Hot Dogs

Visualizing Diff-in-Diff

Visualizing Diff-in-Diff

Visualizing Diff-in-Diff

Visualizing Diff-in-Diff

Visualizing Diff-in-Diff II

Visualizing Diff-in-Diff II

Visualizing Diff-in-Diff II

Visualizing Diff-in-Diff II

Visualizing Diff-in-Diff II

Visualizing Diff-in-Diff II

Visualizing Diff-in-Diff II

Comparing Group Means (Again)

Key Assumption: Counterfactual

Key Assumption: Counterfactual

Example I: HOPE in Georgia

Diff-in-Diff Example I

Diff-in-Diff Example I

Diff-in-Diff Example I

Diff-in-Diff Example I

Diff-in-Diff Example II

Diff-in-Diff Example II

Diff-in-Diff Example II

Diff-in-Diff Example II

Diff-in-Diff Example II

Example: Data

Example: Data

Example: Regression

Example: Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Interpretting the Regression

Example: Comparing Group Means

Example: Comparing Group Means

Example: Comparing Group Means

Example: Comparing Group Means

Example: Comparing Group Means

Example: Comparing Group Means

Example: Comparing Group Means

Example: Comparing Group Means

Example: Comparing Group Means in R

Example: Comparing Group Means in R

Example: Comparing Group Means in R II

Example: Comparing Group Means in R II

Example: Diff-in-Diff Summary

Example: Diff-in-Diff Summary

Diff-in-Diff Summary & Data

Example: Diff-in-Diff Graph

Example: Diff-in-Diff Graph (& Counterfactual)

Generalizing DND Models

Generalizing DND Models

Generalizing DND Models

Generalizing DND Models

Our Example, Generalized I

Ryan Safner
Assistant Professor of Economics
safner@hood.edu
ryansafner/metricsF21
metricsF21.classes.ryansafner.com

Ryan Safner
Assistant Professor of Economics
safner@hood.edu
ryansafner/metricsF21
metricsF21.classes.ryansafner.com