[GitHub] [incubator-mxnet] mightydeveloper opened a new pull request #14607: split_and_load can now handle num_ctx > num_data. Github Issue #13909

[GitBox <gi...@apache.org>]

mightydeveloper opened a new pull request #14607: split_and_load can now handle num_ctx > num_data. Github Issue #13909
URL: https://github.com/apache/incubator-mxnet/pull/14607
 
 
   ## Description ##
   Handles Issue #13909
   when last batch size is smaller than the number of contexts, 
   the previous utility function` gluon.utils.split_and_load` throws an exception `ValueError: Too many slices for data with shape ....`
   However, we can just put one data per context and ignore the remaining contexts.
   This integrates nicely with the given reproducing example code. (User does not need to modify the code)
   ```
   for d, l in zip(data, label):
           with autograd.record():
               out = net(d)
               losses.append(loss_fn(out, l))
       for loss in losses:
           loss.backward()
   ```
   where `data` and `label` will only output data that exists in the first few of the given context.
   the forward / backward pass is only done in the contexts where needed. (remaining contexts does not need to do a fake forward/backward pass)
   My concern is people can make mistakes when calculating mean of the losses.
   
   So it would be nice if we add this behavior to the documentation. (`split_and_load` can output a list that has a size less than number of contexts when even_split=False)
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   -> I believe this PR is just tiny change
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services