[GitHub] [incubator-mxnet] austinmw opened a new issue #18026: [Feature Request] Documentation on mx.np and gluon datasets

[GitBox <gi...@apache.org>]

austinmw opened a new issue #18026: [Feature Request] Documentation on mx.np and gluon datasets
URL: https://github.com/apache/incubator-mxnet/issues/18026
 
 
   ## Description
   I'm trying to determine the recommended way in mxnet 1.6.0 to calculate the statistics of my gluon image Dataset, but there's no clear documentation.
   
   I thought that it was preferred to use `mx.np` now, but since samples from this dataset are loaded as `mx.ndarray`s this doesn’t seem to be right.
   
   For example:
   
   ```python
   from mxnet import gluon
   from mxnet.gluon.data.vision import datasets
   from mxnet import np
   from multiprocessing import cpu_count
   
   fmnist_train = datasets.FashionMNIST(train=True)
   
   def get_mean_and_std(dataset):
     '''Compute the mean and std value of dataset.'''
     dataloader = gluon.data.DataLoader(dataset.transform_first(transforms.ToTensor()), 
                                        batch_size=1, shuffle=True, num_workers=cpu_count())
     mean = np.zeros(3)
     std = np.zeros(3)
     for X, y in dataloader:
       for i in range(3):
         mean[i] +=  np.mean(X[:,i,:,:])
         std[i] +=  np.std(X[:,i,:,:])
     mean /= len(dataset)
     std /= len(dataset)
     return mean, std
   
   get_mean_and_std(fmnist_train)
   ```
   
   I’m not sure what is the recommended way to do this:
   
   - stop using `mx.np`
   - convert my datasets to `mx.np.array`s
   - use `npx.set_np()` and/or `as_np_ndarray()`
   - use a premade convenience function that I haven’t found yet
   - something else entirely
   
   Any help regarding the current recommended way to do this would be greatly appreciated! Also additional documentation for current best practices would go a long way.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-mxnet] austinmw closed issue #18026: [Feature Request] Documentation on mx.np and gluon datasets

[GitBox <gi...@apache.org>]

austinmw closed issue #18026: [Feature Request] Documentation on mx.np and gluon datasets
URL: https://github.com/apache/incubator-mxnet/issues/18026
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services