[GitHub] [incubator-mxnet] liav-teichner commented on issue #15486: mxnet profiler yields exception when multithreaded mxndarray IO operations occur in the background

[GitBox <gi...@apache.org>]

liav-teichner commented on issue #15486: mxnet profiler yields exception when multithreaded mxndarray IO operations occur in the background 
URL: https://github.com/apache/incubator-mxnet/issues/15486#issuecomment-515259583
 
 
   Hi,
   I am a co-worker of Aviv and would like to ask a further question: while we target moving to Gluon API and Gluon data loading, our current code uses Module API and DataIter classes. To achieve acceptable data loading performance we spawn child processes that load data from the disk, augment the data and use NDArray.save to save it to shared memory. The main process that executes the training loop has a background thread that continuously calls a single MXNet function - NDArray.load to load NDArray from shared memory.
   Will this result in unexpected behaviour? (I.e. having a single background thread that calld NDArray.load dyring training). If it does, what's the easiest way to replace it with code that won't result in unexpected behaviour?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services