[GitHub] jwfromm opened a new issue #8978: Very Low Accuracy When Using Pretrained Model

[GitBox <gi...@apache.org>]

jwfromm opened a new issue #8978: Very Low Accuracy When Using Pretrained Model
URL: https://github.com/apache/incubator-mxnet/issues/8978
 
 
   ## Description
   Pretrained models dont seem to be working well with gluon, specifically datasets build with Dataloader and ImageRecords or ImageFolders.
   
   As an example, here I load the ImageNet validation dataset and feed it into alexnet downloaded from gluon model zoo
   
   ```
   ctx = mx.gpu()
   batch_size = 64
   def transformer(data, label):
       data = mx.image.imresize(data, 224, 224)
       data = mx.nd.transpose(data, (2,0,1))
       data = data.astype(np.float32)
       return data/255, label
   
   test_data = gluon.data.DataLoader(gluon.data.vision.ImageFolderDataset(root="/data2/imagenet/val/", transform=transformer),
                                         batch_size, shuffle=False)
   
   model = gluon.model_zoo.vision.alexnet(pretrained=True, ctx=ctx)
   
   def evaluate_accuracy(data_iterator, net):
       acc = mx.metric.Accuracy()
       for d, l in data_iterator:
           data = d.as_in_context(ctx)
           label = l.as_in_context(ctx)
           output = net(data)
           predictions = nd.argmax(output, axis=1)
           acc.update(preds=predictions, labels=label)
       return acc.get()[1]
   
   evaluate_accuracy(test_data, model, ctx=ctx)
   [0.12393999999999999]
   ```
   
   The 12% accuracy shows the issue is probably that the transforms used to train the model dont exactly align with the transforms presented in the Gluon tutorial. It would be nice if an example showing how to properly do this using the new gluon functions were added.
   
   ## Environment info (Required)
   Python 3.6

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services