[GitHub] [incubator-mxnet] zburning opened a new issue #17105: Bug with fusion

[GitBox <gi...@apache.org>]

zburning opened a new issue #17105: Bug with fusion
URL: https://github.com/apache/incubator-mxnet/issues/17105
 
 
   ## Description
   I'm running the script at https://github.com/dmlc/gluon-nlp/tree/master/scripts/language_model/run_glue.py. The command line to reproduce:
   python run_glue.py --task MRPC --gpu 8 --batch_size 32
    
   I have the following observation:
   1. With model.hybridize()
   With mxnet-cu100 >= 1.6.0b20191102, the time cost of the first batch is more than 283s with a single GPU, more than 1000s with 4 GPUs. 
   With mxnet-cu100==1.6.0b20191101, the time cost of the first batch is about 11s  with a single GPU.
   
   2. With model.hybridize() and  os.environ['MXNET_USE_FUSION'] = '0'
   for both mxnet-cu100 == 1.6.0b201911102 and 1.6.0b20191215, the time cost of the first batch is around 11s.
   
   2. Without hybridize():
   The performance are nearly the same.
   
   
   
   
   ### Error Message
   (Paste the complete error message. Please also include stack trace by setting environment variable `DMLC_LOG_STACK_TRACE_DEPTH=10` before running your script.)
   
   ## To Reproduce
   (If you developed your own code, please provide a short script that reproduces the error. For existing examples, please provide link.)
   
   ### Steps to reproduce
   (Paste the commands you ran that produced the error.)
   
   1.
   2.
   
   ## What have you tried to solve it?
   
   1.
   2.
   
   ## Environment
   
   We recommend using our script for collecting the diagnositc information. Run the following command and paste the outputs below:
   ```
   curl --retry 10 -s https://raw.githubusercontent.com/dmlc/gluon-nlp/master/tools/diagnose.py | python
   
   # paste outputs here
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services