You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/02/27 05:30:22 UTC
[GitHub] [incubator-mxnet] eric-haibin-lin opened a new issue #17705: mkldnn
quantized FC is slow
eric-haibin-lin opened a new issue #17705: mkldnn quantized FC is slow
URL: https://github.com/apache/incubator-mxnet/issues/17705
quantized BERT model with int8 is 2x slower than float32.
Download trained SST params from: https://dist-bert.s3.amazonaws.com/demo/finetune/sst.params
Clone is install gluon-nlp v0.9
```
# calibration
KMP_AFFINITY=granularity=fine,noduplicates,compact,1,0 OMP_NUM_THREADS=1 numactl --physcpubind=0 --membind=0 python3 finetune_classifier.py --task_name SST --only_calibration --model_parameters sst.params
# fp32 inference
python3 finetune_classifier.py --task_name SST --epoch 1 --only_inference --model_parameters sst.params --round_to 128 --dev_batch_size 1
# int8 inference
python3 finetune_classifier.py --task_name SST --epoch 1 --only_inference --model_prefix ./output_dir/model_bert_SST_quantized_customize --deploy --round_to 128 --dev_batch_size 1
```
I'm using c5.12xlarge. I tried to set OMP_NUM_THREAD=8, but int8 is still slower than float32.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] pengzhao-intel commented on issue #17705: mkldnn
quantized FC is slow
Posted by GitBox <gi...@apache.org>.
pengzhao-intel commented on issue #17705: mkldnn quantized FC is slow
URL: https://github.com/apache/incubator-mxnet/issues/17705#issuecomment-591789973
@wuxun-zhang @ciyongch
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ciyongch commented on issue #17705: mkldnn
quantized FC is slow
Posted by GitBox <gi...@apache.org>.
ciyongch commented on issue #17705: mkldnn quantized FC is slow
URL: https://github.com/apache/incubator-mxnet/issues/17705#issuecomment-591791543
Thanks for reporting this, I will take a look.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ciyongch commented on issue #17705: mkldnn
quantized FC is slow
Posted by GitBox <gi...@apache.org>.
ciyongch commented on issue #17705: mkldnn quantized FC is slow
URL: https://github.com/apache/incubator-mxnet/issues/17705#issuecomment-591868495
@eric-haibin-lin I just created a PR https://github.com/apache/incubator-mxnet/pull/17707 to address this issue, please take a review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] pengzhao-intel commented on issue #17705: mkldnn
quantized FC is slow
Posted by GitBox <gi...@apache.org>.
pengzhao-intel commented on issue #17705: mkldnn quantized FC is slow
URL: https://github.com/apache/incubator-mxnet/issues/17705#issuecomment-597046840
Feel free to reopen if the issue is not resolved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] pengzhao-intel closed issue #17705: mkldnn
quantized FC is slow
Posted by GitBox <gi...@apache.org>.
pengzhao-intel closed issue #17705: mkldnn quantized FC is slow
URL: https://github.com/apache/incubator-mxnet/issues/17705
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services