# Midterm Cheat Sheet

<table border="1" id="bkmrk-linear-regression-mu" style="border-collapse: collapse; width: 100%;"><tbody><tr><td style="width: 49.9413%;">**Linear Regression**

[![image-1666104229251.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104229251.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104229251.png)

[![image-1666104239088.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104239088.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104239088.png)

[![image-1666104245811.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104245811.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104245811.png)

[![image-1666104252727.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104252727.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104252727.png)

[![image-1666104260540.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104260540.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104260540.png)

[![image-1666104271056.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104271056.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104271056.png)

[![image-1666104276418.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104276418.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104276418.png)

[![image-1666104282605.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104282605.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104282605.png)

[![image-1666104287023.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104287023.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104287023.png)

<span style="text-decoration: none;">Predicting a CI new obs adds a 1 to se(y):</span>

<span style="text-decoration: none;">𝛽<sub><span style="font-weight: normal;">0</span></sub><span style="font-weight: normal;"> </span><span style="font-weight: normal;">+</span><span style="font-weight: normal;"> 𝛽</span><sub><span style="font-weight: normal;">2</span></sub><span style="font-weight: normal;">x +/- t\*</span></span>[![image-1666104297248.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104297248.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104297248.png)

</td><td style="width: 49.9413%;">**Multiple Linear Regression and Estimation**

[![image-1666104323618.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104323618.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104323618.png)

[![image-1666104327905.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104327905.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104327905.png)

[![image-1666104332246.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104332246.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104332246.png)

[![image-1666104336362.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104336362.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104336362.png)

[![image-1666104345652.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104345652.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104345652.png)

[![image-1666104350789.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104350789.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104350789.png)

[![image-1666104355280.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104355280.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104355280.png)

[![image-1666104359234.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104359234.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104359234.png)

[![image-1666104363437.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104363437.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104363437.png)

[![image-1666290673808.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666290673808.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666290673808.png)

[![image-1666290691336.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666290691336.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666290691336.png)

𝐻0 : 𝛽1 = 𝛽2 = 𝛽3 = ⋯ = 𝛽𝑝 = 0   
v.s. 𝐻1 : not all 𝛽𝑘 = 0, 𝑘 = 1, … , 𝑝

[![image-1666104389032.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104389032.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104389032.png)[![image-1666104395117.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104395117.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104395117.png)

rejection rule of 𝑡 &gt;= t(1 − alpha/2; 𝑛 − 𝑝 − 1)

[![image-1666104407108.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104407108.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104407108.png)

[![image-1666104410593.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104410593.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104410593.png)

[![image-1666104415116.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104415116.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104415116.png)

[![image-1666104423017.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104423017.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104423017.png)

</td></tr><tr><td style="width: 49.9413%;">**Model Fitting: Inference**

[![image-1666104576887.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104576887.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104576887.png)

df<sub>Ω</sub> = n - p, and df<sub>𝜔</sub> = n – q

[![image-1666138531298.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666138531298.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666138531298.png)

Reject the null hypothesis if F &gt; Fα p - q, n – p

[![image-1666104643517.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104643517.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104643517.png)

[![image-1666138340542.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666138340542.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666138340542.png)

</td><td style="width: 49.9413%;">**Dummy Variables and Analysis of Covariance**  
Consider a Xi2 for which is 0 for – and 1 for +:

[![image-1666104607845.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104607845.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104607845.png)

An interaction between Xi1 and Xi2:

[![image-1666104615922.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104615922.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104615922.png)

A model with multiple categorical variables:

[![image-1666104625980.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104625980.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104625980.png)

[![image-1666104633030.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104633030.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104633030.png)

</td></tr><tr><td style="width: 49.9413%;">**Regression Diagnostics**  
Assumptions:  
 • Error: ~ N(0, SD2I);   
 ◦ Independent  
 ◦ Equal Variance  
 ◦ Normally Distributed  
 • Model: E\[y\] = Xβ is correct  
 • Unusual observations  
   
Leverage Points: data point with unusual x-value

[![image-1666104773344.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104773344.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104773344.png)

 The Hat Matrix – n\*n matrix  
h<sub>ii </sub>is the leverage of the i<sup>th</sup> case  
leverage &gt; 2p’/n should be looked at closely

  
Outliers: Unusual observation on x or y axis

[![image-1666104790374.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104790374.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104790374.png)

Calculate the t-test and compare abs with limit:  
abs(qt(.05/(n\*2), df = n - pprime - 1, lower.tail = T))

</td><td style="width: 49.9413%;">Influential Points: causes changes to regression  
 Difference in Fits:

[![image-1666104815733.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104815733.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104815733.png)

with a threshold of

[![image-1666104825747.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104825747.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104825747.png)

Where p’ is the number of parameters

 Cook's Distance:

[![image-1666104834542.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666104834542.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666104834542.png)

with a threshold of  
Di &gt; 4/n should be looked at  
Di &gt; .5 possible influence  
Di &gt;= 1 very influential

Error: a plot of e\_hat should  
 • have constant variance  
 • have no clear pattern  
 • H0: residuals are normal

<span style="text-decoration: none;">Shapiro-Wilk normality test</span>

<span style="text-decoration: none;">H0: Residuals are normally distributed</span>

<span style="text-decoration: none;">Bonferroni Correction: Divide alpha by n</span>

</td></tr><tr><td style="width: 49.9413%;">**Variable Selection**

<span style="font-weight: normal;">Backwards Elimination:</span>

1. <span style="font-weight: normal;">Start model with all the predictors</span>
2. <span style="font-weight: normal;">Remove the predictor with highest p-value greater than alpha</span>
3. <span style="font-weight: normal;">Refit the model</span>
4. <span style="font-weight: normal;">Remove the remaining least significant predictor provided its p-value is greater than alpha</span>
5. <span style="font-weight: normal;">Repeat 3 and 4 until all "non-significant" predictors are removed</span>

<span style="font-weight: normal;">Cutoff p significance can be 15-20% for testing</span>

Forward Selection:

1. <span style="font-weight: normal;">Start model with no predictors</span>
2. <span style="font-weight: normal;">For predictors not in the model, check the p-value if they are added to the model. We choose the one with lowest p-value less than alpha</span>
3. <span style="font-weight: normal;">Continue until no new predictors can be added</span>

<span style="font-weight: normal;">Stepwise regression: A combination of the tw</span>o

</td><td style="width: 49.9413%;">Selection Criteria:  
Akaike Information Criterion (AIC):   
 • -2 max log-likelihood + 2p'   
 • n\*log(RSS/n) + 2p'  
Bayes Information Criterion (BIC):   
 • -2 max log-likelihood + p' log(n)   
 • n\*log(RSS/n) + log(n) \* p'   
Adjusted R2:  
R<sup>2</sup> = 1 – RSS/SSY

[![image-1666202623504.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666202623504.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666202623504.png)

Mallow’s C<sub>p</sub> Statistic: Avg MSE of prediction

[![image-1666202639730.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666202639730.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666202639730.png)If a p-predictor fits then:

[![image-1666202661358.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666202661358.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666202661358.png)

We desire models with small p and Cp around or less than p

</td></tr></tbody></table>

**R Code Snippets**

<table border="1" id="bkmrk-%23-model-with-only-be" style="border-collapse: collapse; width: 100%;"><tbody><tr><td style="width: 49.9383%;"><span style="font-family: Courier, monospace;"><span style="font-size: small;">\# Model with only beta\_0</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span style="font-weight: normal;">sr\_lm0 &lt;- lm(</span><span style="font-weight: normal;">y</span><span style="font-weight: normal;"> ~ 1, data=s</span><span style="font-weight: normal;">r</span><span style="font-weight: normal;">)</span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">\# Full model </span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span style="font-weight: normal;">sr\_lm1 &lt;- lm(</span><span style="font-weight: normal;">y </span><span style="font-weight: normal;">~ ., data=s</span><span style="font-weight: normal;">r</span><span style="font-weight: normal;">)</span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">sr\_syy &lt;- sum((savings$sr - mean(savings$sr))^2)</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">sr\_rss &lt;- deviance(sr\_lm1)</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">\# F = ((SYY -RSS)/((n-1) - (n-2))) / (RSS / (n - 1))</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">sr\_num &lt;- (sr\_syy - sr\_rss)/(df.residual(sr\_lm0) - df.residual(sr\_lm1))</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">sr\_den &lt;- sr\_rss / df.residual(sr\_lm1)</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">sr\_f &lt;- sr\_num / sr\_den</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">\# dfΩ = n - p, and df𝜔 = n - q</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">pf(sr\_f, df.residual(sr\_lm0) - df.residual(sr\_lm1), df.residual(sr\_lm1), lower.tail = F)</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span style="font-weight: normal;">\# β=(X</span><sup><span style="font-weight: normal;">I</span></sup><span style="font-weight: normal;"> </span><span style="font-weight: normal;">X)</span><sup><span style="font-weight: normal;">−1</span></sup><span style="font-weight: normal;"> X</span><sup><span style="font-weight: normal;">I</span></sup><span style="font-weight: normal;">Y</span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">beta &lt;- solve(t(x)%\*%x)%\*%(t(x)%\*%y)</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">\# Pearson's</span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">cor(lin\_reg$fitted.values, lin\_reg$residuals, method="pearson")</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# Stratify variables by a factor</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">by(depress, depress$publicassist, summary)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# Welsh's Two Sample T-test </span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# </span></span></span><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">For difference in means</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">t.test(assist$cesd, noassist$cesd) </span></span></span><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# or t.test(data.y ~ factor)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# </span></span></span><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">CI of LS means based on covariates</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">library(lsmeans)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">lsmeans(reg, ~Type)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# Apply a mean function to an array </span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# split on a factor</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">tapply(assist$cesd, assist$assist, mean)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# When a regression factor has </span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# </span></span></span><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">more than two categories</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">reg &lt;- lm(Pulse1 ~ Height + Sex + Smokes + as.factor(Exercise))</span></span></span></span>

</td><td style="width: 49.9383%;"><span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# Cook's Distance </span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">cook &lt;- cooks.distance(reg)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">cook\[cook &gt; 4/n\]</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# Shapiro Test for normallity</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">shapiro.test(reg$residuals)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# Studentized residuals</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">stud &lt;- rstudent(reg)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# Threshold for lower tail of </span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# studentized resids with correction</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">lim = abs(qt(.05/(n\*2), df = n - pprime - 1, lower.tail = T))</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">stud\[which(abs(stud) &gt; lim)\]</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">\# Hat values</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">hat &lt;- hatvalues(reg)</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">lev &lt;- 2 \* pprime / n</span></span></span></span>

<span style="font-size: small;"><span style="font-family: Courier, monospace;"><span lang="zxx"><span style="font-weight: normal;">hat\[hat &gt; lev\]</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">\# </span></span><span lang="zxx"><span style="font-weight: normal;">Forward </span></span><span lang="zxx"><span style="font-weight: normal;">selection</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">forward &lt;- ~ year + unemployed + femlab + marriage + birth + military</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">m0 &lt;- lm(divorce ~ 1, data = usa)</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">reg.forward.AIC &lt;- step(m0, scope = forward, direction = "forward", k = 2)</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">n &lt;- nrow(usa)</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">\# AIC = n\*log(RSS/n) + 2p'</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">n\*log(162.1228/n)+2\*6 </span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">extractAIC(reg.forward.AIC, k=2)</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">\# BIC</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">reg.forward.BIC &lt;- step(m0, scope = forward, direction = "forward", k = log(n))</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">extractAIC(reg.forward,k=log(n)) </span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">\# BIC = n\*log(RSS/n) + p'\*log\*n)</span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;"><span lang="zxx"><span style="font-weight: normal;">n\*log(162.1228/n)+6\*log(n) </span></span></span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">library(leaps)</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">leaps &lt;- regsubsets(divorce ~ .)</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">rs &lt;- summary(leaps)</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">par(mfrow=c(1,2))</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">plot(2:7, rs$cp, xlab="No. of parameters", ylab="Cp Statistic")</span></span>

<span style="font-family: Courier, monospace;"><span style="font-size: small;">abline(0,1)</span></span>

</td></tr></tbody></table>

[![image-1666105013557.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666105013557.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666105013557.png)

[![image-1666105036225.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666105036225.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666105036225.png)[![image-1666105020734.png](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/scaled-1680-/image-1666105020734.png)](https://bookstack.mitchellhenschel.com/uploads/images/gallery/2022-10/image-1666105020734.png)