[GitHub] ThomasDelteil commented on issue #13447: Rewrite dataloader, improves responsiveness and reliability

[GitBox <gi...@apache.org>]

ThomasDelteil commented on issue #13447: Rewrite dataloader, improves responsiveness and reliability
URL: https://github.com/apache/incubator-mxnet/pull/13447#issuecomment-445433076
 
 
   Yes I think the problem with multi-threading is to make sure the user understand that get a real parallel processing boost, it needs to be done in the underlying libraries like opencv to bypass the GIL. I think there's a case for it but it shouldn't be the default and has appropriate warning in the documentation. I believe different pickling protocols can be chosen to trade off compute for memory @leezu ? Maybe we could also expose that to the users

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services