Classify Handwritten Digits with MLP Classifier

Introduction

This tutorial will show you how to create an MLPClassifier using Scikit-learn to classify handwritten digits from the MNIST dataset. We will also visualize the weights of the first layer of the MLP to gain insight into the learning behavior.

VM Tips

After the VM startup is done, click the top left corner to switch to the Notebook tab to access Jupyter Notebook for practice.

Sometimes, you may need to wait a few seconds for Jupyter Notebook to finish loading. The validation of operations cannot be automated because of limitations in Jupyter Notebook.

If you face issues during learning, feel free to ask Labby. Provide feedback after the session, and we will promptly resolve the problem for you.

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL sklearn(("`Sklearn`")) -.-> sklearn/UtilitiesandDatasetsGroup(["`Utilities and Datasets`"]) sklearn(("`Sklearn`")) -.-> sklearn/ModelSelectionandEvaluationGroup(["`Model Selection and Evaluation`"]) sklearn(("`Sklearn`")) -.-> sklearn/CoreModelsandAlgorithmsGroup(["`Core Models and Algorithms`"]) ml(("`Machine Learning`")) -.-> ml/FrameworkandSoftwareGroup(["`Framework and Software`"]) sklearn/UtilitiesandDatasetsGroup -.-> sklearn/datasets("`Datasets`") sklearn/UtilitiesandDatasetsGroup -.-> sklearn/exceptions("`Exceptions and Warnings`") sklearn/ModelSelectionandEvaluationGroup -.-> sklearn/model_selection("`Model Selection`") sklearn/CoreModelsandAlgorithmsGroup -.-> sklearn/neural_network("`Neural Network Models`") ml/FrameworkandSoftwareGroup -.-> ml/sklearn("`scikit-learn`") subgraph Lab Skills sklearn/datasets -.-> lab-49216{{"`Classify Handwritten Digits with MLP Classifier`"}} sklearn/exceptions -.-> lab-49216{{"`Classify Handwritten Digits with MLP Classifier`"}} sklearn/model_selection -.-> lab-49216{{"`Classify Handwritten Digits with MLP Classifier`"}} sklearn/neural_network -.-> lab-49216{{"`Classify Handwritten Digits with MLP Classifier`"}} ml/sklearn -.-> lab-49216{{"`Classify Handwritten Digits with MLP Classifier`"}} end

Import Libraries

We will start by importing the necessary libraries for this project.

import warnings
import matplotlib.pyplot as plt
from sklearn.datasets import fetch_openml
from sklearn.exceptions import ConvergenceWarning
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split

Load Data

Next, we will load the MNIST dataset using Scikit-learn's fetch_openml function.

X, y = fetch_openml(
    "mnist_784", version=1, return_X_y=True, as_frame=False, parser="pandas"
)

Preprocess Data

We will normalize the data by dividing each pixel value by 255.0, which is the maximum pixel value.

X = X / 255.0

Split Data

We will split the dataset into a training set and a test set using the train_test_split function.

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0, test_size=0.7)

Train MLPClassifier

We will create an MLPClassifier with a single hidden layer containing 40 neurons. We will train the MLP for only 8 iterations due to resource constraints. We will also catch the ConvergenceWarning that will be thrown because the model won't converge within the limited number of iterations.

mlp = MLPClassifier(
    hidden_layer_sizes=(40,),
    max_iter=8,
    alpha=1e-4,
    solver="sgd",
    verbose=10,
    random_state=1,
    learning_rate_init=0.2,
)

with warnings.catch_warnings():
    warnings.filterwarnings("ignore", category=ConvergenceWarning, module="sklearn")
    mlp.fit(X_train, y_train)

Evaluate the Model

We will evaluate the MLPClassifier by computing its accuracy on the training and test sets.

print("Training set score: %f" % mlp.score(X_train, y_train))
print("Test set score: %f" % mlp.score(X_test, y_test))

Visualize Weights

Finally, we will visualize the weights of the first layer of the MLP. We will create a 4x4 grid of subplots and display each weight as a 28x28 pixel grayscale image.

fig, axes = plt.subplots(4, 4)
vmin, vmax = mlp.coefs_[0].min(), mlp.coefs_[0].max()
for coef, ax in zip(mlp.coefs_[0].T, axes.ravel()):
    ax.matshow(coef.reshape(28, 28), cmap=plt.cm.gray, vmin=0.5 * vmin, vmax=0.5 * vmax)
    ax.set_xticks(())
    ax.set_yticks(())

plt.show()

Summary

In this tutorial, we learned how to create an MLPClassifier using Scikit-learn to classify handwritten digits from the MNIST dataset. We also visualized the weights of the first layer of the MLP to gain insight into the learning behavior.