You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2021/04/14 08:59:01 UTC
[GitHub] [incubator-mxnet] bgawrych opened a new pull request #20163: [Feature] Add oneDNN support for interleaved_matmul_selfatt_* operators (fp32/int8)
bgawrych opened a new pull request #20163:
URL: https://github.com/apache/incubator-mxnet/pull/20163
## Description ##
This change adds oneDNN support for two operators:
- _contrib_interleaved_matmul_selfatt_qk
- _contrib_interleaved_matmul_selfatt_valatt
Both operators will be used when backend MKLDNN/MKLDNN_QUANTIZE will be chosen - there is no change in terms of performance between MKL fp32 vs. oneDNN fp32, but the main advantage is utilizing int8 data type
10 iterations of BERT-Large (gluon-nlp v0.10.x) [Intel(R) Core(TM) i9-9940X CPU @ 3.30GHz]:
**MKL implementation (fp32 as int8 is not supported):**
**oneDNN implementation (int8):**
_contrib_interleaved_matmul_selfatt_qk => _sg_mkldnn_selfatt_qk
_contrib_interleaved_matmul_selfatt_qk => _sg_mkldnn_selfatt_valatt
We can observe that this change positively influenced other operators as there is less dequantization/quantization overhead and memory reorders
Great contribution of @grygielski to this change
## Checklist ##
### Essentials ###
- [x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
- [x] Changes are complete (i.e. I finished coding on this PR)
- [x] All changes have test coverage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha merged pull request #20163: [Feature] Add oneDNN support for interleaved_matmul_selfatt_* operators (fp32/int8)
Posted by GitBox <gi...@apache.org>.
szha merged pull request #20163:
URL: https://github.com/apache/incubator-mxnet/pull/20163
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on pull request #20163: [Feature] Add oneDNN support for interleaved_matmul_selfatt_* operators (fp32/int8)
Posted by GitBox <gi...@apache.org>.
szha commented on pull request #20163:
URL: https://github.com/apache/incubator-mxnet/pull/20163#issuecomment-823270420
@bgawrych thanks for the contribution
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20163: [Feature] Add oneDNN support for interleaved_matmul_selfatt_* operators (fp32/int8)
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20163:
URL: https://github.com/apache/incubator-mxnet/pull/20163#issuecomment-820412392
Jenkins CI successfully triggered : [windows-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #20163: [Feature] Add oneDNN support for interleaved_matmul_selfatt_* operators (fp32/int8)
Posted by GitBox <gi...@apache.org>.
bgawrych commented on pull request #20163:
URL: https://github.com/apache/incubator-mxnet/pull/20163#issuecomment-823005459
@szha Can we merge it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on pull request #20163: [Feature] Add oneDNN support for interleaved_matmul_selfatt_* operators (fp32/int8)
Posted by GitBox <gi...@apache.org>.
bgawrych commented on pull request #20163:
URL: https://github.com/apache/incubator-mxnet/pull/20163#issuecomment-820412320
@mxnet-bot run ci [windows-cpu]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20163: [Feature] Add oneDNN support for interleaved_matmul_selfatt_* operators (fp32/int8)
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20163:
URL: https://github.com/apache/incubator-mxnet/pull/20163#issuecomment-819356165
Hey @bgawrych , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [unix-cpu, clang, unix-gpu, windows-cpu, centos-gpu, website, centos-cpu, sanity, miscellaneous, windows-gpu, edge]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org