Scikit-Learn Ridge Regression Example

Introduction

This lab demonstrates how to use Ridge Regression for estimating collinear coefficients of an estimator. Ridge Regression is a type of linear regression that applies L2 regularization to the model.

In this example, we will generate a 10x10 Hilbert matrix and use Ridge Regression to estimate the coefficients of the matrix.

VM Tips

After the VM startup is done, click the top left corner to switch to the Notebook tab to access Jupyter Notebook for practice.

Sometimes, you may need to wait a few seconds for Jupyter Notebook to finish loading. The validation of operations cannot be automated because of limitations in Jupyter Notebook.

If you face issues during learning, feel free to ask Labby. Provide feedback after the session, and we will promptly resolve the problem for you.

Skills Graph

Import Required Libraries

In this step, we will import the required libraries for this example.

import numpy as np
import matplotlib.pyplot as plt
from sklearn import linear_model

Generate Data

In this step, we will generate a 10x10 Hilbert matrix and set the target variable y to be a vector of ones.

X = 1.0 / (np.arange(1, 11) + np.arange(0, 10)[:, np.newaxis])
y = np.ones(10)

Compute Ridge Regression Paths

In this step, we will compute the Ridge Regression paths for different regularization strengths.

n_alphas = 200
alphas = np.logspace(-10, -2, n_alphas)

coefs = []
for a in alphas:
    ridge = linear_model.Ridge(alpha=a, fit_intercept=False)
    ridge.fit(X, y)
    coefs.append(ridge.coef_)

Visualize Results

In this step, we will visualize the results of the Ridge Regression paths.

ax = plt.gca()

ax.plot(alphas, coefs)
ax.set_xscale("log")
ax.set_xlim(ax.get_xlim()[::-1])  ## reverse axis
plt.xlabel("alpha")
plt.ylabel("weights")
plt.title("Ridge coefficients as a function of the regularization")
plt.axis("tight")
plt.show()

Summary

In this lab, we demonstrated how to use Ridge Regression for estimating collinear coefficients of an estimator. We generated a 10x10 Hilbert matrix and used Ridge Regression to estimate the coefficients of the matrix. We then visualized the results of the Ridge Regression paths. Ridge Regression is useful for reducing the variation (noise) in highly ill-conditioned matrices. By setting a certain regularization strength, we can balance the effect of the regularization and the squared loss function.