ConvNetJS CIFAR-10 demo 卷积神经网络分类demo

ConvNetJS CIFAR-10 demo

Description

This demo trains a Convolutional Neural Network on the CIFAR-10 dataset in your browser, with nothing but Javascript. The state of the art on this dataset is about 90% accuracy and human performance is at about 94% (not perfect as the dataset can be a bit ambiguous). I used this python script to parse the original files (python version) into batches of images that can be easily loaded into page DOM with img tags.

This dataset is more difficult and it takes longer to train a network. Data augmentation includes random flipping and random image shifts by up to 2px horizontally and verically.

By default, in this demo we're using Adadelta which is one of per-parameter adaptive step size methods, so we don't have to worry about changing learning rates or momentum over time. However, I still included the text fields for changing these if you'd like to play around with SGD+Momentum trainer.

Report questions/bugs/suggestions to @karpathy.

Training Stats

Forward time per example: 23ms
Backprop time per example: 24ms
Classification loss: 2.24887
L2 Weight decay loss: 0.00085
Training accuracy: 0.16
Validation accuracy: -1
Examples seen: 141

Learning rate:     
Momentum:     
Batch size:     
Weight decay:   

 

Loss:

Instantiate a Network and Trainer

Network Visualization

Activations:

input (32x32x3)

max activation: 0.42549, min: -0.5
max gradient: 0.01467, min: -0.01771

Activations:

Activation Gradients:

Weights:

Weight Gradients:

conv (32x32x16)

filter size 5x5x3, stride 1
max activation: 1.13915, min: -0.9202
max gradient: 0.0151, min: -0.01904
parameters: 16x5x5x3+16 = 1216

Activations:

Activation Gradients:

relu (32x32x16)

max activation: 1.13915, min: 0
max gradient: 0.01867, min: -0.01904

Activations:

Activation Gradients:

pool (16x16x16)

pooling size 2x2, stride 2
max activation: 1.13915, min: 0
max gradient: 0.01867, min: -0.01904

Activations:

Activation Gradients:

Weights:
()()()()()()()()()()()()()()()()()()()()
Weight Gradients:
()()()()()()()()()()()()()()()()()()()()

conv (16x16x20)

filter size 5x5x16, stride 1
max activation: 0.74686, min: -1.94339
max gradient: 0.05836, min: -0.05125
parameters: 20x5x5x16+20 = 8020

Activations:

Activation Gradients:

relu (16x16x20)

max activation: 0.74686, min: 0
max gradient: 0.0788, min: -0.06547

Activations:

Activation Gradients:

pool (8x8x20)

pooling size 2x2, stride 2
max activation: 0.74686, min: 0
max gradient: 0.0788, min: -0.06547

Activations:

Activation Gradients:

Weights:
()()()()()()()()()()()()()()()()()()()()
Weight Gradients:
()()()()()()()()()()()()()()()()()()()()

conv (8x8x20)

filter size 5x5x20, stride 1
max activation: 0.46723, min: -0.37553
max gradient: 0.11546, min: -0.15106
parameters: 20x5x5x20+20 = 10020

Activations:

Activation Gradients:

relu (8x8x20)

max activation: 0.46723, min: 0
max gradient: 0.11546, min: -0.15106

Activations:

Activation Gradients:

pool (4x4x20)

pooling size 2x2, stride 2
max activation: 0.46723, min: 0
max gradient: 0.11546, min: -0.15106

Activations:

Activation Gradients:

fc (1x1x10)

max activation: 0.42779, min: -0.56877
max gradient: 0.11617, min: -0.82716
parameters: 10x320+10 = 3210

Activations:

softmax (1x1x10)

max activation: 0.17284, min: 0.0638
max gradient: 0, min: 0

Example predictions on Test set

test accuracy based on last 200 test images: 0

/from: http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html/

car

cat

frog

airplane

truck

car

car

cat

frog

car

cat

airplane