Neural Net Observatory

Two multilayer perceptrons share their hidden layers and differ only at the output: 784 to 128 to 64 to 10 on the MNIST digits, and 784 to 128 to 64 to 26 on the EMNIST letters. ReLU, softmax cross-entropy, He-normal initialisation, Adam with cosine decay from 1e-3 to 1e-5, batch 128. Nothing exotic, deliberately. The architecture is not the interesting part; the fact that every derivative in it was worked out and written down rather than imported is.

There are two independent implementations of the same maths. The backward pass in numpy is derived by hand, and the TypeScript one is written again from scratch, including gradients with respect to the input rather than only the parameters. That duplication is what makes the tests mean anything: logits agree to a measured maximum difference of 5e-11, and predictions agree image by image across both full test sets.

The gradients themselves are checked against central finite differences in float64 on every parameter tensor, with a worst relative error of 7.4e-7, and a 64-image overfit test drives the loss from 2.343 to 0.0003. Between them those two catch nearly every way a hand-derived backward pass goes subtly wrong, which is the failure mode where everything still trains and simply ends up worse.

Nothing is synthesised and nothing is taken on trust. MNIST is tried against four mirrors in turn, and every file is checked against its published byte size and MD5, its IDX magic number and shape, and its per-class label counts before anything trains on it. EMNIST is the official 561,753,746 byte NIST archive, verified by size and MD5 before it is opened and then by zip CRC, header, mapping file and the published count of 4,800 training and 800 test images per letter. If any of that disagrees, the script stops instead of continuing with a smaller number.

Two things about EMNIST are easy to get wrong and both fail silently. Every image is stored transposed relative to MNIST, so a loader that does not transpose back trains perfectly happily on sideways letters and simply scores worse than it should. The test for it does not compare arrays: it checks on the real data that the mean L leans on its left stroke with its foot at the bottom and the mean T carries its bar on top, which the untransposed layout fails. Separately the TypeScript tests transpose the raw test file themselves and reproduce Python's prediction on all 20,800 images, so both implementations would have to be wrong in the same direction to agree.

The other is the labels, stored 1 to 26 with 1 meaning A. They are shifted so class k is chr(65 + k), and the mapping file has to say exactly that or the build stops. Upper and lower case share one class in EMNIST Letters, so the model answers A to Z whichever case you draw, and the page says so beside the prediction rather than letting a lowercase g scoring as G look like a bug.

A network trained on MNIST has never seen a 280 pixel mouse stroke. It has seen 28 by 28 images framed by one specific rule, and reproducing that rule is most of the difference between a model that works on your handwriting and one that does not.

The two datasets frame differently and the app reproduces each. MNIST crops to the ink's bounding box, scales the long side to 20 pixels with exact area-weighted resampling, then shifts by whole pixels until the centre of mass lands on (14, 14). Real MNIST centres of mass sit at (13.996, 14.007) with a spread of 0.29 pixels, which is just whole-pixel rounding. EMNIST crops, scales the long side to 24 pixels and puts the bounding box centre on the field centre, because across its 124,800 training letters the box centre holds to 0.2 pixels while the centre of mass wanders by 1.3.

The training script measures that framing on the training set and writes it into the exported metadata, and a test checks the TypeScript constants against what Python measured, so the two cannot drift apart. Taking real test images, rendering them large and re-framing them back through the browser's own canvas keeps 9,957 of 10,000 digit predictions and 20,364 of 20,800 letter predictions. Framing either dataset by the other's rule changes many more, which is also a test.

Augmentation closes some of what is left. Every mini-batch is shifted by up to 2 pixels, rotated up to 12 degrees and scaled by up to 10 per cent, and one image in five gets a soft stroke thickening, so that the chunky strokes a finger makes are not completely outside what the model has seen.

The page opens on the trained weights. Home resets the network to snapshot 0, which is the real He-normal initialisation and not a decorative scramble, and from there the transport plays 60 snapshots: step 0, then 59 log-spaced to the end, because nearly everything worth watching happens in the first few hundred steps. The digits model is at 9.03 per cent, then 69.36 by step 16, then 98.94 by step 28,080.

Each snapshot carries int8 weights with one symmetric scale per tensor, float32 biases, train and test loss, the full confusion matrix over the whole test set, and the 3D PCA coordinates of the second hidden layer's activations for a fixed set of test images, Procrustes-aligned to the previous snapshot so the cloud turns smoothly instead of flipping. The history is 6.3 MiB gzipped for digits and 6.5 for letters.

Everything follows the scrubber: the network, the prediction and its confidence bars, the inspector, the loss and accuracy curves, the confusion matrix, the embedding cloud and the first-layer receptive fields. A status line always names what is on screen, Trained or Untrained or Checkpoint 34 of 59, with that snapshot's test accuracy beside it, because a network that looks half-learned should say which half.

The int8 snapshots are not a rough stand-in for the story. The accuracy printed beside each one is computed with the dequantised int8 weights, so the number on screen is the number those exact weights produce, and the final int8 copy scores 98.92 against the float32 model's 98.94.

Layers are planes: the input a 28 by 28 pixel grid, hidden units nodes whose brightness is their activation relative to the layer, outputs glowing with their softmax probability. Edges are the strongest contributions, activation times weight, red where a unit raises the next one and blue where it lowers it, additively blended through a bloom pass. Only the top 1,600, 560 and 220 per layer pair are drawn, because 109,386 at once is a fog with no information in it.

Clicking a neuron pulls it out of that fog. It gets a ring and a name tag, its own 40 strongest inputs and outputs are drawn whether or not they made the global cut, and everything unconnected to it recedes. A first-layer neuron shows its incoming weights as an image, a second-layer neuron the classes it votes for and against, an output its probability, its logit and the hidden units that moved it most. Each of those is a link to the neuron it names, so you can walk backwards through whatever produced the answer.

Learned Patterns asks the question the other way round: for each class, what input would these weights score most strongly as that class. It is 260 steps of gradient ascent on the log probability from a blank image, through the same TypeScript backward pass the tests check, with an L2 penalty, a small blur every other step and a seeded shift. It runs in a worker, because 26 classes is a few seconds of float maths, and it is recomputed per snapshot, so during a replay you watch the model's idea of a letter assemble itself out of noise.

Accuracy on what you draw is not the figure in the headline. Those are the full test sets, which are scanned pen on paper. A mouse or a finger is a different distribution, and the framing and augmentation narrow that gap rather than closing it.

The letters model is genuinely worse than the digits model, for a reason the confusion matrix makes plain. Its most frequent confusions are I against L on 353 test images, G against Q on 181 and U against V on 57, and in EMNIST a lowercase l and a capital I are frequently the same vertical stroke. Some of those pairs are not separable from the image alone, which is a property of the dataset rather than something a better model would fix.

The architecture is deliberately dull. A small convolutional network would beat both of these comfortably, and would also be far harder to draw as individual weights, which is the entire purpose here. Nothing was tuned on a test set either: the digits hyperparameters were fixed before training except the epoch count, and the letters run reuses that recipe unchanged.

Training is reproducible rather than roughly reproducible. Seeds are fixed for initialisation, data order, augmentation and the evaluation subsets, and re-running the digits training after a refactor reproduced the shipped model, snapshot and curve files byte for byte on the same machine and numpy build.