Nonlinearities

Nonlinearities

Nonlinearities

Nonlinearities

Logarithmic Models

The Natural Logarithm

The Natural Logarithm: Review I

The Natural Logarithm: Review II

• Logarithms can have any base, but common to use the natural logarithm \((\ln)\) with base \(\mathbf{e=2.71828...}\) $$\text{If } e^n=y\text{, then } \ln(y)=n$$

The Natural Logarithm: Properties

• Natural logs have a lot of useful properties:

1. \(\ln(\frac{1}{x})=-\ln(x)\)

2. \(\ln(ab)=\ln(a)+\ln(b)\)

3. \(\ln(\frac{x}{a})=\ln(x)-\ln(a)\)

4. \(\ln(x^a)=a \, \ln(x)\)

5. \(\frac{d \, \ln \, x}{d \, x} = \frac{1}{x}\)

The Natural Logarithm: Example

Elasticity

$$\epsilon_{Y,X}=\frac{\% \Delta Y}{\% \Delta X} =\cfrac{\left(\frac{\Delta Y}{Y}\right)}{\left( \frac{\Delta X}{X}\right)}$$

Math FYI: Cobb Douglas Functions and Logs

• One of the (many) reasons why economists love Cobb-Douglas functions: $$Y=AL^{\alpha}K^{\beta}$$

• Taking logs, relationship becomes linear:

$$\ln(Y)=\ln(A)+\alpha \ln(L)+ \beta \ln(K)$$

• With data on \((Y, L, K)\) and linear regression, can estimate \(\alpha\) and \(\beta\)
  • \(\alpha\): elasticity of \(Y\) with respect to \(L\)
    • A 1% change in \(L\) will lead to an \(\alpha\)% change in \(Y\)
  • \(\beta\): elasticity of \(Y\) with respect to \(K\)
    • A 1% change in \(K\) will lead to a \(\beta\)% change in \(Y\)

Math FYI: Cobb Douglas Functions and Logs

• Taking logs:

$$\ln Y=\ln 2+0.75 \ln L + 0.25 \ln K$$

• A 1% change in \(L\) will yield a 0.75% change in output \(Y\)

• A 1% change in \(K\) will yield a 0.25% change in output \(Y\)

Logarithms in R I

• The log() function can easily take the logarithm

gapminder <- gapminder %>%
  mutate(loggdp = log(gdpPercap)) # log GDP per capita
gapminder %>% head() # look at it

Logarithms in R II

• Note, log() by default is the natural logarithm \(ln()\), i.e. base e
  • Can change base with e.g. log(x, base = 5)
  • Some common built-in logs: log10, log2

log10(100)

## [1] 2

log2(16)

## [1] 4

log(19683, base=3)

## [1] 9

Logarithms in R III

• Note when running a regression, you can pre-transform the data into logs (as I did above), or just add log() around a variable in the regression

lm(lifeExp ~ loggdp,
   data = gapminder) %>%
  tidy()

lm(lifeExp ~ log(gdpPercap),
   data = gapminder) %>%
  tidy()

Types of Logarithmic Models

• Three types of log regression models, depending on which variables we log

1. Linear-log model: \(Y_i=\beta_0+\beta_1 \color{#e64173}{\ln X_i}\)

2. Log-linear model: \(\color{#e64173}{\ln Y_i}=\beta_0+\beta_1X_i\)

3. Log-log model: \(\color{#e64173}{\ln Y_i}=\beta_0+\beta_1 \color{#e64173}{\ln X_i}\)

Linear-Log Model

• Linear-log model has an independent variable \((X)\) that is logged

$$\begin{align*} Y&=\beta_0+\beta_1 \color{#e64173}{\ln X_i}\\ \beta_1&=\cfrac{\Delta Y}{\big(\frac{\Delta X}{X}\big)}\\ \end{align*}$$

• Marginal effect of \(\mathbf{X \rightarrow Y}\): a 1% change in \(X \rightarrow\) a \(\frac{\beta_1}{100}\) unit change in \(Y\)

Linear-Log Model in R

lin_log_reg <- lm(lifeExp ~ loggdp,
                  data = gapminder)
library(broom)
lin_log_reg %>% tidy()

Linear-Log Model in R

lin_log_reg <- lm(lifeExp ~ loggdp,
                  data = gapminder)
library(broom)
lin_log_reg %>% tidy()

Linear-Log Model in R

lin_log_reg <- lm(lifeExp ~ loggdp,
                  data = gapminder)
library(broom)
lin_log_reg %>% tidy()

Linear-Log Model in R

lin_log_reg <- lm(lifeExp ~ loggdp,
                  data = gapminder)
library(broom)
lin_log_reg %>% tidy()

Linear-Log Model Graph I

ggplot(data = gapminder)+
  aes(x = gdpPercap,
      y = lifeExp)+
  geom_point(color="blue", alpha=0.5)+
  geom_smooth(method="lm",
              formula=y~log(x),
              color="orange")+
  scale_x_continuous(labels=scales::dollar,
                     breaks=seq(0,120000,20000))+
  scale_y_continuous(breaks=seq(0,100,10),
                     limits=c(0,100))+
  labs(x = "GDP per Capita",
       y = "Life Expectancy (Years)")+
  ggthemes::theme_pander(base_family = "Fira Sans Condensed",
           base_size=16)

Linear-Log Model Graph II

ggplot(data = gapminder)+
  aes(x = loggdp,
      y = lifeExp)+ 
  geom_point(color="blue", alpha=0.5)+
  geom_smooth(method="lm", color="orange")+
  scale_y_continuous(breaks=seq(0,100,10),
                     limits=c(0,100))+
  labs(x = "Log GDP per Capita",
       y = "Life Expectancy (Years)")+
  ggthemes::theme_pander(base_family = "Fira Sans Condensed",
           base_size=16)

Log-Linear Model

• Log-linear model has the dependent variable \((Y)\) logged

$$\begin{align*} \color{#e64173}{\ln Y_i}&=\beta_0+\beta_1 X\\ \beta_1&=\cfrac{\big(\frac{\Delta Y}{Y}\big)}{\Delta X}\\ \end{align*}$$

• Marginal effect of \(\mathbf{X \rightarrow Y}\): a 1 unit change in \(X \rightarrow\) a \(\beta_1 \times 100\) % change in \(Y\)

Log-Linear Model in R (Preliminaries)

gapminder <- gapminder %>%
  mutate(gdp_t = gdpPercap/1000, # first make GDP/capita in $1000s
         loglife = log(lifeExp)) # take the log of LifeExp
gapminder %>% head() # look at it

Log-Linear Model in R

log_lin_reg <- lm(loglife~gdp_t,
                data = gapminder)
log_lin_reg %>% tidy()

Log-Linear Model in R

log_lin_reg <- lm(loglife~gdp_t,
                data = gapminder)
log_lin_reg %>% tidy()

Log-Linear Model in R

log_lin_reg <- lm(loglife~gdp_t,
                data = gapminder)
log_lin_reg %>% tidy()

Log-Linear Model in R

log_lin_reg <- lm(loglife~gdp_t,
                data = gapminder)
log_lin_reg %>% tidy()

Linear-Log Model Graph I

ggplot(data = gapminder)+
  aes(x = gdp_t,
      y = loglife)+
  geom_point(color="blue", alpha=0.5)+
  geom_smooth(method="lm", color="orange")+
  scale_x_continuous(labels=scales::dollar,
                     breaks=seq(0,120,20))+
  labs(x = "GDP per Capita ($ Thousands)",
       y = "Log Life Expectancy")+
  ggthemes::theme_pander(base_family = "Fira Sans Condensed",
           base_size=16)

Log-Log Model

• Log-log model has both variables \((X \text{ and } Y)\) logged

$$\begin{align*} \color{#e64173}{\ln Y_i}&=\beta_0+\beta_1 \color{#e64173}{\ln X_i}\\ \beta_1&=\cfrac{\big(\frac{\Delta Y}{Y}\big)}{\big(\frac{\Delta X}{X}\big)}\\ \end{align*}$$

• Marginal effect of \(\mathbf{X \rightarrow Y}\): a 1% change in \(X \rightarrow\) a \(\beta_1\) % change in \(Y\)

• \(\beta_1\) is the elasticity of \(Y\) with respect to \(X\)!

Log-Log Model in R

log_log_reg <- lm(loglife ~ loggdp,
                  data = gapminder)
log_log_reg %>% tidy()

Log-Log Model in R

log_log_reg <- lm(loglife ~ loggdp,
                  data = gapminder)
log_log_reg %>% tidy()

Log-Log Model in R

log_log_reg <- lm(loglife ~ loggdp,
                  data = gapminder)
log_log_reg %>% tidy()

Log-Log Model in R

log_log_reg <- lm(loglife ~ loggdp,
                  data = gapminder)
log_log_reg %>% tidy()

Log-Log Model Graph I

ggplot(data = gapminder)+
  aes(x = loggdp,
      y = loglife)+
  geom_point(color="blue", alpha=0.5)+
  geom_smooth(method="lm", color="orange")+
  labs(x = "Log GDP per Capita",
       y = "Log Life Expectancy")+
  ggthemes::theme_pander(base_family = "Fira Sans Condensed",
           base_size=16)

Comparing Models I

• Hint: the variable that gets logged changes in percent terms, the variable not logged changes in unit terms
  • Going from units \(\rightarrow\) percent: multiply by 100
  • Going from percent \(\rightarrow\) units: divide by 100

Comparing Models II

library(huxtable)
huxreg("Life Exp." = lin_log_reg,
       "Log Life Exp." = log_lin_reg,
       "Log Life Exp." = log_log_reg,
       coefs = c("Constant" = "(Intercept)",
                 "GDP ($1000s)" = "gdp_t",
                 "Log GDP" = "loggdp"),
       statistics = c("N" = "nobs",
                      "R-Squared" = "r.squared",
                      "SER" = "sigma"),
       number_format = 2)

Comparing Models III

Linear-Log	Log-Linear	Log-Log
\(\hat{Y_i}=\hat{\beta_0}+\hat{\beta_1}\color{#e64173}{\ln X_i}\)	\(\color{#e64173}{\ln Y_i}=\hat{\beta_0}+\hat{\beta_1}X_i\)	\(\color{#e64173}{\ln Y_i}=\hat{\beta_0}+\hat{\beta_1}\color{#e64173}{\ln X_i}\)
\(R^2=0.65\)	\(R^2=0.30\)	\(R^2=0.61\)

When to Log?

Comparing Coefficients of Different Units I

Comparing Coefficients of Different Units II

• An easy way is to standardize^† the variables (i.e. take the \(Z\)-score)

$$X_Z=\frac{X_i-\overline{X}}{sd(X)}$$

^† Also called “centering” or “scaling.”

Comparing Coefficients of Different Units: Example

Standardizing in R

gapminder <- gapminder %>%
  mutate(life_Z = scale(lifeExp),
         gdp_Z = scale(gdpPercap))
std_reg <- lm(life_Z ~ gdp_Z, data = gapminder)
tidy(std_reg)

## # A tibble: 2 × 5
##   term        estimate std.error statistic   p.value
##   <chr>          <dbl>     <dbl>     <dbl>     <dbl>
## 1 (Intercept) 1.10e-16    0.0197  5.57e-15 1.00e+  0
## 2 gdp_Z       5.84e- 1    0.0197  2.97e+ 1 3.57e-156

Joint Hypothesis Testing I

• Maybe region doesn't affect wages at all?

• \(H_0: \beta_2=0, \, \beta_3=0, \, \beta_4=0\)

• This is a joint hypothesis to test

Joint Hypothesis Testing II

• A joint hypothesis tests against the null hypothesis of a value for multiple parameters: $$\mathbf{H_0: \beta_1= \beta_2=0}$$ the hypotheses that multiple regressors are equal to zero (have no causal effect on the outcome)

• Our alternative hypothesis is that: $$H_1: \text{ either } \beta_1\neq0\text{ or } \beta_2\neq0\text{ or both}$$ or simply, that \(H_0\) is not true

Types of Joint Hypothesis Tests

Joint Hypothesis Tests: F-statistic

• The F-statistic is the test-statistic used to test joint hypotheses about regression coefficients with an F-test

• This involves comparing two models:

  1. Unrestricted model: regression with all coefficients
  2. Restricted model: regression under null hypothesis (coefficients equal hypothesized values)

• \(F\) is an analysis of variance (ANOVA)

  • essentially tests whether \(R^2\) increases statistically significantly as we go from the restricted model$\rightarrow$unrestricted model

• \(F\) has its own distribution, with two sets of degrees of freedom

Joint Hypothesis F-test: Example I

• \(H_0: \beta_2=\beta_3=\beta_4=0\)

• \(H_a\): \(H_0\) is not true (at least one \(\beta_i \neq 0\))

Joint Hypothesis F-test: Example II

Calculating the F-statistic

Calculating the F-statistic

Calculating the F-statistic

Calculating the F-statistic

Calculating the F-statistic

Calculating the F-statistic

Calculating the F-statistic II

F-test Example I

• We'll use the wooldridge package's wage1 data again

# load in data from wooldridge package
library(wooldridge)
wages <- wage1
# run regressions
unrestricted_reg <- lm(wage ~ female + northcen + west + south, data = wages)
restricted_reg <- lm(wage ~ female, data = wages)

F-test Example II

• Unrestricted model:

$$\widehat{Wage_i}=\hat{\beta_0}+\hat{\beta_1}Female_i+\hat{\beta_2}Northeast_i+\hat{\beta_3}Northcen+\hat{\beta_4}South_i$$

• Restricted model:

$$\widehat{Wage_i}=\hat{\beta_0}+\hat{\beta_1}Female_i$$

• \(H_0: \beta_2 = \beta_3 = \beta_4 =0\)

• \(q = 3\) restrictions (F numerator df)

• \(n-k-1 = 526-4-1=521\) (F denominator df)

F-test Example III

# load car package for additional regression tools
library(car) 
# F-test
linearHypothesis(unrestricted_reg, c("northcen", "west", "south"))

## Linear hypothesis test
## 
## Hypothesis:
## northcen = 0
## west = 0
## south = 0
## 
## Model 1: restricted model
## Model 2: wage ~ female + northcen + west + south
## 
##   Res.Df    RSS Df Sum of Sq      F   Pr(>F)   
## 1    524 6332.2                                
## 2    521 6174.8  3    157.36 4.4258 0.004377 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Second F-test Example: Are Two Coefficients Equal?

Second F-test Example: Data

# load in data
heightwages <- read_csv("../data/heightwages.csv")
# make a "heights" variable as the sum of adolescent (height81) and adult (height85) height
heightwages <- heightwages %>%
  mutate(heights = height81 + height85)
height_reg <- lm(wage96 ~ height81 + height85 + male, data = heightwages)
height_restricted_reg <- lm(wage96 ~ heights + male, data = heightwages)

Second F-test Example: Data

• For second argument, set two variables equal, in quotes

linearHypothesis(height_reg, "height81 = height85") # F-test

## Linear hypothesis test
## 
## Hypothesis:
## height81 - height85 = 0
## 
## Model 1: restricted model
## Model 2: wage96 ~ height81 + height85 + male
## 
##   Res.Df     RSS Df Sum of Sq      F Pr(>F)
## 1   6591 5128243                           
## 2   6590 5127284  1     959.2 1.2328 0.2669

• Insufficient evidence to reject \(H_0\)!

• The effect of adolescent and adult height on wages is the same

All F-test I

summary(unrestricted_reg)

## 
## Call:
## lm(formula = wage ~ female + northcen + west + south, data = wages)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -6.3269 -2.0105 -0.7871  1.1898 17.4146 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   7.5654     0.3466  21.827   <2e-16 ***
## female       -2.5652     0.3011  -8.520   <2e-16 ***
## northcen     -0.5918     0.4362  -1.357   0.1755    
## west          0.4315     0.4838   0.892   0.3729    
## south        -1.0262     0.4048  -2.535   0.0115 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.443 on 521 degrees of freedom
## Multiple R-squared:  0.1376,    Adjusted R-squared:  0.131 
## F-statistic: 20.79 on 4 and 521 DF,  p-value: 6.501e-16

All F-test II

• Alternatively, if you use broom instead of summary():
  • glance() command makes table of regression summary statistics
  • tidy() only shows coefficients

library(broom)
glance(unrestricted_reg)

## # A tibble: 1 × 12
##   r.squared adj.r.squared sigma statistic  p.value    df logLik   AIC   BIC
##       <dbl>         <dbl> <dbl>     <dbl>    <dbl> <dbl>  <dbl> <dbl> <dbl>
## 1     0.138         0.131  3.44      20.8 6.50e-16     4 -1394. 2800. 2826.
## # … with 3 more variables: deviance <dbl>, df.residual <int>, nobs <int>

• statistic is the All F-test, p.value next to it is the p-value from the F test