[GitHub] DickJC123 commented on issue #14006: Dual stream cudnn Convolution backward() with MXNET_GPU_WORKER_NSTREAMS=2.

[GitBox <gi...@apache.org>]

DickJC123 commented on issue #14006: Dual stream cudnn Convolution backward() with MXNET_GPU_WORKER_NSTREAMS=2.
URL: https://github.com/apache/incubator-mxnet/pull/14006#issuecomment-458656852
 
 
   I think we're not at the point of having the framework be smart enough to make this trade-off, which involves a potential performance increase at the expense of a larger model global memory footprint.  I think users would be upset if suddenly they have to drop their batchsize due to an out-of-memory error (and lose perf) because the framework was ill-advisedly stretching for the ultimate efficiency.  On the other hand, allowing experts to fine-tune performance for production uses and demonstrators is important.  I think a next step toward what you're asking is a storage allocation that is allowed to return a null pointer (rather than immediately exiting).  That would allow operators to take sensible fall-back approaches if memory is exhausted.  This would be a different PR after some discussion.
   
   In the meantime, I still recommend adding this knob in the form of the MXNET_GPU_WORKER_NSTREAMS environment variable.  I think it natural to first introduce a facility with a manual control knob, then evolve to a point where the framework picks the best setting.  The control knob could be retained quietly in the background to support testing and to prove that the automatic selection is performing correctly.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services