The General Linear Model

$$\Large \boldsymbol{Y} \sim \mathcal{N}(\boldsymbol{Y}, \sigma^2\boldsymbol{I})$$ $$\Large \boldsymbol{\hat{Y}} = \boldsymbol{\beta X}$$

• This is the Matrix formulation of $\sum\beta_jx_j$

• This equation is huge. X can be anything - categorical, continuous, squared, sine, etc.

• There can be straight additivity, or interactions

• So far, the only model we've used with >1 predictor is ANOVA

So Many Ways to Build a Model

Combining Categorical Variables and Continuous Variables (Analysis of Covariance)

$$\Large \boldsymbol{Y} = \boldsymbol{\beta X}$$

translates to

$$\widehat{y_i} = \beta_0 + \beta_{1}x_1 + \sum_{j=2}^{k}\beta_{ij}x_{ij}$$
$$x_{ij} = 0,1$$

• Here, we have many levels of a group in $x_{ij}$

• Often used to correct for a gradient or some continuous variable affecting outcome

• OR used to correct a regression due to additional groups that may throw off slope estimates
  • e.g. Simpson's Paradox: A positive relationship between test scores and academic performance can be masked by gender differences

Neanderthals and Mixing Categorical and Continuous Variables

Who had a bigger brain: Neanderthals or us?

The Means Look the Same...

But there appears to be a Relationship Between Body and Brain Mass

And Mean Body Mass is Different

Mixing Categories and Continuous Variables: Analysis of Covariance

$$lnbrain_i \sim \mathcal{N}(\widehat{lnbrain_i}, \sigma_2)$$ $$\widehat{lnbrain_i} = \beta_0 + \beta_{1}lnmass_1 + \sum_{j=2}^{k}\beta_{ij}species_{ij}$$
$$species_{ij} = 0\; or\; 1\; if\; neandertal$$

Evaluate a categorical effect(s), controlling for a covariate (parallel lines)

Groups modify the intercept.

Fit with Our Engine: Ordinary Least Squares

neand_lm <- lm(lnbrain ~ species + lnmass, 
               data=neand)
neand_dat <- augment(neand_lm)

Does Your Model Match Your Data?

Are There Weird Patterns in Your Residuals?

Did You Modelboss Too Close to the Correlated Suns?

Did Our Predictors Matter?

What are the Association Between Predictors and Response?

What Does it Look Like to See the Unique Effect of One Driver?

• Intercept is the brain mass of a neanderthal with 0 body mass

• speciesrecent is the change in brain mass for a recent human, still assuming 0 body mass

• lnmass is the association of log body mass with log brain mass

What Does it Mean Visually?

We can run comparisons at equal body mass

So Many Ways to Build a Model

The Linear Effect of Cover on Richness

But... What's Up With That Bend?

How Do We Add Nonlinearities to our Linear Model?

$$y_{i} \sim N(\widehat{y_{i}}, \sigma^{2} )$$ $$\widehat{y_{i}} = \beta_{0} + \sum \beta_{j}x_{ij}$$

• What if $x_1$ is a linear term and $x_2 = x_1^2$?

• Adding nonlinear terms is just adding more predictors to a multiple linear regression model

• Sometimes to reduce collinearity/make our model make more sense, we need to center x - i.e. use $x-\bar{x}$ - to reduce SE in parameters

A Nonlinear Effect of Cover?

mod_sq <- lm(rich ~ cover + I(cover^2), data = keeley)

Which is Better? Maybe A Cubic? Cross-Validate with LOO!

2 or 4? Or Other Nonlinear Terms?

So Many Ways to Build a Model

What if Fire Severity Only Affects Species Richness in Old Stands?

Interaction Effects

• The effect of one predictor cannot be known without knowing the level of another

• Common in nature!

• Some are inevitable (e.g., the effects of disturbance depend on something being there to disturb)

• Some are.... quite tricky, but the spice of biological life!

• Exercise: Think of an interaction effect you have encountered in biology or elsewhere!

Interaction Effects in Models

$$y_{i} \sim N(\widehat{y_{i}}, \sigma^{2} )$$ $$\widehat{y_{i}} = \beta_{0} + \beta_{1}x_{1i} + \beta_{2}x_{2i} + \beta_{3}x_{1i}x_{2i}$$ $$x_{2i} = 0,1$$ If we consider $x_1 * x_2$ a variable in it's own right - i.e. $x_3$ then..... this is just.......

$$\widehat{y_{i}} = \beta_{0} + \sum\beta_{j}x_{ij}$$

IT'S ALL THE SAME THING!

Interaction Effects in Models - and F-Tests!

mod_int <- lm(rich ~ firesev * age_break, data = keeley)

Interaction Effects What does it mean?

• The intercept is the number of species when fire severity is 0 and stand age is young

• The firesev effect is the effect of fire for young stands

• The age effect is the increase in species in an old stand - but only if fire severity is 0

• The interaction is the change in the fire severity slope when the stand is old

This is Why I Hate Staring at Coefficients

Models with Continuous Interactions - The Same Model!

$$y_{i} \sim \mathcal{N}(\widehat{y_{i}}, \sigma^2)$$ $$\widehat{y_{i}} = \beta_{0} + \beta_{1}x_{1i} + \beta_{2}x_{2i} + \beta_{3}x_{1i}x_{2i}$$

Fit and Go!

mod_int_c <- lm(rich ~ firesev * age, data = keeley)

Did You Make Sure your Train Wasn't On Fire?

But Was This a Good Idea? AIC Edition

Coefficients are Clearer

• Intercept is when all predictors are 0
  
  
• Additive slopes are when the OTHER predictor is 0
  
  
• Interaction is how they modify each other

What if a Predictor Value of 0 Makes No Sense: Centering

• Additive coefficients are now the effect of a predictor at the mean value of the other predictors

• Intercepts are at the mean value of all predictors

• Visualization will keep you from getting confused!

Model Visualization: The Effect of One Predictor at Different Levels of the Other

Final Thoughts

• Whew! That's a lot of ways to build a model!

• Note, we are still thinking in terms of normal error - and an additive model

• There are plenty of extensions

  • nonlinear non-additive models
  • Generalized Linear Models with Non-Normal Error

• But, additive Gaussian models are a safe starting point

• AND extensions.... really all just have linear models at their core

• It's the BIOLOGY and your intuition of what model to build that should always be the core of your modeling process