Discussion Forum & Activities

In the formula of the Ridge regression, which is \( min_w ||y-X_w||^2_2 + \alpha*||w||^2_2 \),  \( \alpha \) stands for the regularization term and it should avoid overfitting.

With small values for \( \alpha \) the ridge regression overfits the dataset with high accuracy both in training set and test set, while with higher values for \( \alpha \) the ridge regression ends up with an approximation function very far from the target.