[GitHub] [incubator-mxnet] chandana1332 opened a new pull request #15195: Fixed a bug in Gluon DataLoader.

[GitBox <gi...@apache.org>]

chandana1332 opened a new pull request #15195: Fixed a bug in Gluon DataLoader.
URL: https://github.com/apache/incubator-mxnet/pull/15195
 
 
   Issue: https://github.com/apache/incubator-mxnet/issues/15025
   Fix: Broadened the scope of worker pool to iterators. Passed a reference of dataloader to the multi worker iterator
   
   ## Description ##
   Bug: Gluon DataLoader terminates the process pool early while _MultiWorkerIter is operating on the pool.
   Cause: https://github.com/apache/incubator-mxnet/pull/13537/files
   As seen in the patch, the process pool is terminated when DataLoader is garbage collected but the scope of the process pool goes beyond the DataLoader until _MultiWorkerIter
   
   Fix: Broadened the scope of worker pool to iterators. Passed a reference of dataloader to the multi worker iterator
   
   For more details: https://github.com/apache/incubator-mxnet/issues/15025
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - [x] Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
   - [x] Code is well-documented: 
   - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - Broadened the scope of worker pool to iterators. Passed a reference of dataloader to the multi worker iterator. Added a test for the same.
   
   ## Comments ##
   - If this change is a backward incompatible change, why must this change be made.
   - Interesting edge cases to note here
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services