[GitHub] mfiore commented on issue #13945: dataloader crashes with threads and slow downs with processes

[GitBox <gi...@apache.org>]

mfiore commented on issue #13945: dataloader crashes with threads and slow downs with processes
URL: https://github.com/apache/incubator-mxnet/issues/13945#issuecomment-459270811
 
 
   @zhreshold  thanks for your answer!
   The dataset is about 178k examples so with a batch size of 32 it's more than 5k batches.
   Putting the mutex in the line you pointed solves the crash problem. Now the behaviors with threads or multiprocess is the same, with slow downs every few batches.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services