You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/04/25 08:00:16 UTC

[GitHub] [tvm] yangulei opened a new pull request, #11111: [BYOC] Enable bfloat16 in DNNL BYOC

yangulei opened a new pull request, #11111:
URL: https://github.com/apache/tvm/pull/11111

   Enable [bfloat16](https://discuss.tvm.apache.org/t/rfc-add-bfloat16-data-type/6778) in [DNNL BYOC](https://discuss.tvm.apache.org/t/rfc-byoc-intel-r-onednn-integration/11582) following the path:
   
   > [float32 graph] --> \<[AMP](https://discuss.tvm.apache.org/t/rfc-relay-fp32-fp16-model-support/9994)\> --> [bfloat16 graph] --> \<BYOC\> --> [TVM + oneDNN module]
   
   Main work include:
   - Enable more data types in DNNL json runtime (only bfloat16 has been tested so far).
   - Consider dtype while querying optimal DNNL layout.
   - Add tests for bf16 DNNL BYOC.
   
   With those improvements, a float32 graph could be converted to bfloat16 through AMP, and then be lowered by native codegen or consumed by oneDNN and finally inference in bfloat16 mode now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] yangulei commented on pull request #11111: [BYOC] Enable bfloat16 in DNNL BYOC

Posted by GitBox <gi...@apache.org>.
yangulei commented on PR #11111:
URL: https://github.com/apache/tvm/pull/11111#issuecomment-1138259115

   Thanks a lot.
   We are working on the e2e performance optimization in DNNL-BYOC 😄 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi commented on a diff in pull request #11111: [BYOC] Enable bfloat16 in DNNL BYOC

Posted by GitBox <gi...@apache.org>.
masahi commented on code in PR #11111:
URL: https://github.com/apache/tvm/pull/11111#discussion_r882396542


##########
tests/python/contrib/test_dnnl.py:
##########
@@ -37,6 +37,8 @@
     ids=["compile", "run"],
 )
 
+bf16_supported = "avx512" in open("/proc/cpuinfo", "r").read()

Review Comment:
   Probably need more precise detection, but ok.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi merged pull request #11111: [BYOC] Enable bfloat16 in DNNL BYOC

Posted by GitBox <gi...@apache.org>.
masahi merged PR #11111:
URL: https://github.com/apache/tvm/pull/11111


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] yangulei commented on pull request #11111: [BYOC] Enable bfloat16 in DNNL BYOC

Posted by GitBox <gi...@apache.org>.
yangulei commented on PR #11111:
URL: https://github.com/apache/tvm/pull/11111#issuecomment-1138114180

   Modifications since the original PR:
   - #11140
   - #11369
   - [change FATAL to WARNING for invalid layout transforms to pass the corresponding tests](https://github.com/apache/tvm/pull/11111/commits/a42764a7db1f1cb2596a224cee55eb59ee59eae1)
   - [skip bf16 tests if AVX512 is unavailable to prevent failure in test_dnnl.py](https://github.com/apache/tvm/pull/11111/commits/4caede47b08f6f5eded9e8d8bcc15da0edba4c68)
   
   All the CI checks have passed now, what a long journey.
   @masahi Could you please help to review this PR, thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi commented on pull request #11111: [BYOC] Enable bfloat16 in DNNL BYOC

Posted by GitBox <gi...@apache.org>.
masahi commented on PR #11111:
URL: https://github.com/apache/tvm/pull/11111#issuecomment-1138254585

   cc @AndrewZhaoLuo this is cool (the first e2e run of AMP + bf16!!) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org