Advanced Machine Learning
Recall that in an ordinary multiple linear regression, we have a set of p predictor variables measuring some response variables to fit a model like:
$$ Y = /deltabeta_0 + /beta_1 X_1 ... _ /beta_n X_n + /epsilon $$
Least Absolute Shrinkage and Selection Operator (LASSO)
Lasso regression is a regularization technique for linear regression models. Regularization is a statistical method to reduce errors caused by overfitting on training data.
The process:
- Start with full model (all possible features)
- "Shrink" some coefficients to 0 (exactly)
- Non-zero coefficents indicate "selected" features
Traditional ridge regression:
Total Cost = Measure of Fit [AKA RSS(w)] +