Digit Recogniser 1.0

A web application which guesses the digit you have drawn using Machine Learning

How to use this application?

1. Click on '(Re)Train' when you open the website for first time

2. Draw on the canvas, it is the black square

3. Click on 'Predict' and the model guesses what you have drawn

click here to jump to the drawing canvas


Note: This application works best only on desktop, sorry

How was this app built?

Short answer: tensorflow-js


This app was built using the tensorflow-js library. There is a codelab link which dives in great detail regarding the model and how it is built here
In a nutshell:
Data:The data is the MNIST dataset . In this dataset, you have a bunch of handwritten digits which were captured on a single channel (hence they're black and white)


Sample Data


Model: we build a 2 layer CNN model. To understand CNNs in detail, click here (build a separate page which explains CNN and its architecture)
Training and accuracy: To check the accuracy of ṭhe model, we look at the accuracy per class and also the confusion matrix. These are over here

Cannot see the confusion matrix?: please click the (re)train button over here

What happens when you click train or predict?

Train: The model is re-instantiated and re-trained on the MNIST dataset.
     This means, everytime you click the (re)train button, a new model is trained in the browser using new parameters
Predict:The image from the canvas is transformed and fed into the model (remember the MNIST data consists of 28x28 images but the canvas is 300x300 pixels).
     The model then predicts what the drawn image might be

Model Architecture


Overview of the model


tensorflow-js gives us the ability to train models inside the browser itself. Once trained, this model is saved in the local storage, so you don't have to re-train it everytime the browser is refreshed.
We have 2-layers of Convolutional filters, which end with a fully connected layer with 10 nodes . The reason we have 10 nodes is because we have 10 classes from 0 - 9.

Why are some predictions bad?

If you play long enough with the canvas, you will eventually see that some of the predictions are consistently wrong. One of the major reasons for these wrong predictions is the classic case of overfitting

Overfitting: refers to the phenonmenon where the ML model works really well on the trained and validation data, but does not perform on unseen data

In our model, we trained our model on the MNIST data, but we predict using the canvas drawing as an input.
     If we see few examples from MNIST and compare it with the canvas drawings, we can see that there are certain stroke patterns which make canvas data diferent from MNIST data (apart from the image sizes, of course)


sample MNIST data


sample canvas input

You can see that the canvas produces even strokes but as the MNIST data is handwritten, the stroke pattern is different.
     Hence, as the canvas data is different from the MNIST data, this unseen data cause the predictions to be poor.

Can we make the model better?

Yes, overfitting can be corrected by expanding the training data. This means we need to increase the sample of the training data to include canvas drawn images.
    This model will be trained offline and will be called statically in the browser. tensorflow-js also has the ability to train models offline and just call the parameters in the browser, this helps reducing training times and save memory
     This implmentation will be part of digit recogniser ver 2.0, which will essentially make the predictions better, as it will be trained on both MNIST and canvas data.

You can view this model at this link