[GitHub] vishaalkapoor commented on issue #8601: Why is the accuracy overestimated?

[GitBox <gi...@apache.org>]

vishaalkapoor commented on issue #8601: Why is the accuracy overestimated?
URL: https://github.com/apache/incubator-mxnet/issues/8601#issuecomment-415176400
 
 
   @mstokes42 If your confusion matrix is representative of your test/train set, it looks like you're using very few (~ 100) training examples with what appears to be LeNet-5. This is about 2 orders of magnitude too little data; in [Lecun98](http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf) LeNet-5 was trained using MNIST (60k 32x32 training images).
   
   The reason you're getting 100% accuracy on your train set is that LeNet-5 is memorizing your training data and not learning enough to generalize to your test set. 
   
   Take a look at the Convolutional Neural networks section for some examples with training sets:
   https://gluon.mxnet.io/chapter04_convolutional-neural-networks/cnn-scratch.html
   E.g.
   `
   batch_size = 64
   num_inputs = 784
   num_outputs = 10
   def transform(data, label):
       return nd.transpose(data.astype(np.float32), (2,0,1))/255, label.astype(np.float32)
   train_data = gluon.data.DataLoader(gluon.data.vision.MNIST(train=True, transform=transform),
                                         batch_size, shuffle=True)
   test_data = gluon.data.DataLoader(gluon.data.vision.MNIST(train=False, transform=transform),
                                        batch_size, shuffle=False)
   `
   Additionally, if this is the issue, please feel free to close the issue. Thanks!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services