[GitHub] Ishitori commented on issue #14089: Load dataset directly to vRAM

[GitBox <gi...@apache.org>]

Ishitori commented on issue #14089: Load dataset directly to vRAM
URL: https://github.com/apache/incubator-mxnet/issues/14089#issuecomment-464247795
 
 
   I don't think it is possible at the moment. The data always loaded first in RAM, and then you need to manually move it to the proper GPU memory with either `split_and_load()` or `as_in_context()`. 
   
   If you can load the batch of 16 items in RAM, but would like to load more into GPUs before doing a forward pass, then I would:
   
   1. Write a custom loop which would go through ImageRecordIter as many times as you need (let's say you want each GPU to work with 32 items: for 8 GPUs you need to do 8 iterations)
   2. On each iteration do `split_and_load()` and concatenate the result for each GPU with the data that is already there if any.
   
   Something like this (I haven't run the code myself, so it might need adjustments):
   
   ```
   import mxnet as mx
   from mxnet import gluon
   
   num_gpus = 8
   max_items_for_ram = 16
   max_items_per_gpu = 32
   ctx = [mx.gpu(i) for i in range(num_gpus)]
   
   data_train = mx.io.ImageRecordIter(
       path_imgrec='dataset_train.rec',
       data_shape=(3, 256, 256),
       batch_size=max_items_for_ram,
       label_width = 22
   )
   
   gpu_items = []
   
   iters_to_load_max_items_per_gpu = max_items_per_gpu / (max_items_for_ram / num_gpus)
   
   for iteration in range(int(iters_to_load_max_items_per_gpu)):
       for batch in data_train:
           items = gluon.utils.split_and_load(batch, ctx)
       
           if not gpu_items:
               gpu_items = items
           else:
               for i in range(len(ctx)):
                   gpu_items[i] = mx.nd.concat(gpu_items[i], items[i])
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services