Models with Single Predictors

class: center, middle

# Single Predictor Models

![:scale 55%](Images/lm/linear_regression_love.gif)

---

# What are we doing when we model data?

1. Start with a question we want to answer

2. Design a model of the part of the world we need to know about to answer the question

3. Acquire data appropriate to build that model

4. Chose an engine to fit the model

5. Chose an inferential framework

6. Use that framework to ask the fit model questions about the world

---
# Many models have one predictor
![:scale 75%](Images/lm/simple_train.jpg)

---

class: middle

# Many models have one predictor

---
# Model in Equation Form

.large[

`$$y_i \sim \mathcal{N}(\hat{y_i}, \sigma^2)\\
\hat{y_i} = \beta_0 + \beta_1  x_i$$`

]

- Response `$y_i$` is normally distributed around a fitted value with some residual error.

- The fitted value is determined by an intercept and a slope multiplied by a predictor `$x_i$`.

- That's it.

---
background-color:black
class:middle
![](Images/lm/why_linear_models.jpg)

---
# One Predictor Models
1. Linear Regression  
      - Did our model set its world on fire?  
      - Does our predictor matter?  
      - Describing associations between variables
      - What confidence do we have in our fit?  
      - How good are we at explaining our response

2. Comparing two means  
     - It's just linear regression with two values for X  
     - Assessing the difference between two estimated parameters

3. Comparing many means  
     - It's just a bunch of two-value linear regressions again  
     - Making pairwise comparisons

---

# Batesian Mimicry with Pufferfish

.pull-left[
- Pufferfish are toxic/harmful to predators  
<br>
- Batesian mimics gain protection from predation - why?
<br><br>
- Evolved response to appearance?
<br><br>
- Researchers tested with mimics varying in toxic pufferfish resemblance
]

.pull-right[
![:scale 80%](./Images/lm/puffer_mimics.jpg)
]

---
# Scale of Dissimilarity

![:scale 25%](./Images/lm/puffer_treatments.png)

.pull-left[ .small[Carey and Schluter 2003] ]

---

# The Data
<img src="one_predictor_models_files/figure-html/puffershow-1.png" style="display: block; margin: auto;" />

---

# Our Model
.large[

`$$visits_i \sim \mathcal{N}(\hat{visits_i}, \sigma^2)\\
\hat{visits_i} = \beta_0 + \beta_1  resemblance_i$$`

]

<img src="one_predictor_models_files/figure-html/dagmod-1.png" style="display: block; margin: auto;" />
---

# Fit with Our Engine: Ordinary Least Squares

### Simple model, simple solution!

.large[

```r
mod <- lm(predators ~ resemblance, data = puffer)
```

]

---

# Engine Makes No Difference Here

|engine |term        | estimate| std.error|
|:------|:-----------|--------:|---------:|
|OLS    |(Intercept) |    16.87|      1.66|
|OLS    |resemblance |    -2.99|      0.57|
|ML     |(Intercept) |    16.87|      1.66|
|ML     |resemblance |    -2.99|      0.57|
|Bayes  |(Intercept) |    16.92|      1.77|
|Bayes  |resemblance |    -3.00|      0.61|

---
background-color:black
class:middle

![:scale 65%](Images/lm/i-dont-always-5acc6f.jpg)

---

# Does Your Model Match Your Data?

---

# Are There Weird Patterns in Your Residuals?

---

# What To Do If You've Caused a Fire

- Lay down and cry

- Change your model to something more interesting
     - This includes simple changes like log or logit transforming

- Recognize that your coefficient estimates are still correct, you just cannot do any inferential *testing* or talk about *error*

---

# Did Our Predictor Matter?

- F = Variability due to Slope & Intercept / Variability due to Error

- We ask, what is the probability we obtain an F values like the one we see or an even bigger one if our predictor didn't matter given how much data we have?

- This is called a **p-value**

---

# Did Our Predictor Matter?
<br><br>
<table class="table table-striped" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> term </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> sumsq </th>
   <th style="text-align:right;"> meansq </th>
   <th style="text-align:right;"> statistic </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> resemblance </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 255.15 </td>
   <td style="text-align:right;"> 255.15 </td>
   <td style="text-align:right;"> 27.37 </td>
   <td style="text-align:right;"> 0 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Residuals </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 167.80 </td>
   <td style="text-align:right;"> 9.32 </td>
   <td style="text-align:right;"> NA </td>
   <td style="text-align:right;"> NA </td>
  </tr>
</tbody>
</table>

- p is so small it rounds to 0

- the probability of our result or a more extreme result given a null hypothesis that our result isn't distinguishable from noise is 0

- larger p-values mean a greater chance that we cannot reject this null

- what % chance are you comfortable with?

---

# What is the Association Between Predictor and Response?

- **Intercept:** If resemblance = 0, 16.87 predators will approach on average.

- **Slope:** A 1 unit increase in resemblance is associated with a decrease in ~3 predator visits

- The SE gives an estimate of the precision of our slope...

---

# How Precise Are Our Estimates?
### We can see by simulating new lines from our coefficients

---

# How Precise Are Our Estimates?
### We can see by with a 95% confidence interval of our estimate

<img src="one_predictor_models_files/figure-html/unnamed-chunk-3-1.png" style="display: block; margin: auto;" />
---

# How Strong is the Association Between Our Generative Model and Observed Data?

`$$R^2 = 1 - \frac{Residual \; Variation}{Total \; Variation \; in \; the \; Data}$$`
<img src="one_predictor_models_files/figure-html/resplot-1.png" style="display: block; margin: auto;" />

2. .red[ Comparing two means]  
     - It's just linear regression with two values for X

3. Comparing many means  
     - It's just a bunch of two-value linear regressions again  
     - Making pairwise comparisons  
     
---

# Comparing High and Low Similarity

---
class:middle
background-color: black

![](Images/lm/morpheus_regression.jpg)

---

# Our Model Is Unchanged
.large[

`$$visits_i \sim \mathcal{N}(\hat{visits_i}, \sigma^2)\\
\hat{visits_i} = \beta_0 + \beta_1  resemblance_i$$`

]

<img src="one_predictor_models_files/figure-html/dagmod-1.png" style="display: block; margin: auto;" />
---

# Comparing Means is Just a Line with a 0 and 1 as X

---

# Fit with OLS

- `lm` recodes treatment as 0/1 - a dummy variable!

.large[

```r
mod_t <- lm(predators ~ resemblance_trt, data = puffer_t)
```

]

---

# Does Your Model Match Your Data?

<img src="one_predictor_models_files/figure-html/densplot_t-1.png" style="display: block; margin: auto;" />
---

# Are There Weird Patterns in Your Residuals?

---

# Did Our Predictor Matter?
### F-Test!

<table class="table table-striped" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> term </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> sumsq </th>
   <th style="text-align:right;"> meansq </th>
   <th style="text-align:right;"> statistic </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> resemblance_trt </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 207.20 </td>
   <td style="text-align:right;"> 207.20 </td>
   <td style="text-align:right;"> 18.55 </td>
   <td style="text-align:right;"> 0 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Residuals </td>
   <td style="text-align:right;"> 10 </td>
   <td style="text-align:right;"> 111.71 </td>
   <td style="text-align:right;"> 11.17 </td>
   <td style="text-align:right;"> NA </td>
   <td style="text-align:right;"> NA </td>
  </tr>
</tbody>
</table>

Still basically no chance of observing our data or more extreme data

---

# Are the Groups *REALLY* Different?

- If we subtract one group mean from the other, the **null hypothesis** is that the answer will be 0

- It should fall in a distribution, though, given that this is a sample.

- This distribution was discovered by a former chief brewer at Guiness! He published under the name Student and called this the t distribution.

---

# Are the Groups *REALLY* Different?

<table class="table table-striped" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> contrast </th>
   <th style="text-align:right;"> estimate </th>
   <th style="text-align:right;"> std.error </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> statistic </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Not Similar - Similar </td>
   <td style="text-align:right;"> 8.43 </td>
   <td style="text-align:right;"> 1.96 </td>
   <td style="text-align:right;"> 10 </td>
   <td style="text-align:right;"> 4.31 </td>
   <td style="text-align:right;"> 0 </td>
  </tr>
</tbody>
</table>

---

# What do our Estimates Look Like?

2. Comparing two means  
     - It's just linear regression with two values for X

3. .red[ Comparing many means]  
     - It's just a bunch of two-value linear regressions again  
     - Making pairwise comparisons  
     
---

# Categorical Instead of Continuous Predictors

---
class:middle
background-color: black

![](Images/lm/linear_regression_everywhere.jpg)

---

# Our Model Changes...slightly....
.large[

`$$visits_{ij} \sim \mathcal{N}(\hat{visits_{ij}}, \sigma^2)\\
\hat{visits_{ij}} = \beta_0 + \sum\beta_j  resemblance_{ij}$$`

]

<Br><br>
- individual i in treatment j

- `$resemblance_{ij}$` is now 1 if you are in treatment j, otherwise 0

- `$\beta_j$` can be deviation from a grant mean, `$\beta_0$`

---
# But Conceptually It Is the Same

---

# It's just a bunch of 0's and 1s!

---

# What You Are Ultimately Fitting

```r
mod_aov <- lm(predators ~ resemblance_c, data = puffer)
```

---

# Still Have to Make Sure We Have Started No Fires

---

# Does Our Predictor Matter?
## This is often called an Analysis of Variance or ANOVA
### It's still just an F test

<br>
<table class="table table-striped" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> term </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> sumsq </th>
   <th style="text-align:right;"> meansq </th>
   <th style="text-align:right;"> statistic </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> resemblance_c </td>
   <td style="text-align:right;"> 3 </td>
   <td style="text-align:right;"> 270.24 </td>
   <td style="text-align:right;"> 90.08 </td>
   <td style="text-align:right;"> 9.44 </td>
   <td style="text-align:right;"> 0 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Residuals </td>
   <td style="text-align:right;"> 16 </td>
   <td style="text-align:right;"> 152.71 </td>
   <td style="text-align:right;"> 9.54 </td>
   <td style="text-align:right;"> NA </td>
   <td style="text-align:right;"> NA </td>
  </tr>
</tbody>
</table>

Compare to regression - note difference in F:
<table class="table table-striped" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> term </th>
   <th style="text-align:right;"> df </th>
   <th style="text-align:right;"> sumsq </th>
   <th style="text-align:right;"> meansq </th>
   <th style="text-align:right;"> statistic </th>
   <th style="text-align:right;"> p.value </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> resemblance </td>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:right;"> 255.15 </td>
   <td style="text-align:right;"> 255.15 </td>
   <td style="text-align:right;"> 27.37 </td>
   <td style="text-align:right;"> 0 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Residuals </td>
   <td style="text-align:right;"> 18 </td>
   <td style="text-align:right;"> 167.80 </td>
   <td style="text-align:right;"> 9.32 </td>
   <td style="text-align:right;"> NA </td>
   <td style="text-align:right;"> NA </td>
  </tr>
</tbody>
</table>

---

# What do our Estimates Look Like?
<img src="one_predictor_models_files/figure-html/cat-1.png" style="display: block; margin: auto;" />
---

# Which Groups *REALLY* Different?  Have some T...
<img src="one_predictor_models_files/figure-html/tukey-1.png" style="display: block; margin: auto;" />

Was this model a good way to ask the question?

---

# One Predictor and Go!
.large[

`$$y_i \sim \mathcal{N}(\hat{y_i}, \sigma^2)\\
\hat{y_i} = \beta_0 + \beta_1  x$$`

]

- There are many ways to use this model

- We can fit lines, compare two or even more means

- But it's just a model. Your questions have to guide how you use it.

- For communication with your audience, they don't need the nitty-gritty details - but they need to learn what the model can tell them that is new