You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "tqchen (via GitHub)" <gi...@apache.org> on 2023/03/26 16:57:28 UTC

[GitHub] [tvm] tqchen opened a new pull request, #14405: [TIR] Refactor BF16Legalize

tqchen opened a new pull request, #14405:
URL: https://github.com/apache/tvm/pull/14405

   This PR refactors BF16Legalize to enable more f32 computations. We also split the BF16Legalize into two steps.
   
   BF16ComputeLegalize changes all computation to f32 while keeping the external BF16 storages.
   
   BF16StorageLegalize changes all storage to u16. Now BF16 kernels accept tvm.nd.array that are created as bfloat16 type.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] junrushao merged pull request #14405: [TIR] Refactor BF16Legalize

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao merged PR #14405:
URL: https://github.com/apache/tvm/pull/14405


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] junrushao commented on a diff in pull request #14405: [TIR] Refactor BF16Legalize

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #14405:
URL: https://github.com/apache/tvm/pull/14405#discussion_r1149924075


##########
tests/python/frontend/onnx/test_forward.py:
##########
@@ -99,6 +99,14 @@ def get_tvm_output_with_vm(
             freeze_params=freeze_params,
             convert_config=convert_config,
         )
+        # handle the bfloat16 so we explicitly allocate
+        # bfloat16 arrays as input
+        for i, param in enumerate(mod["main"].params):
+            if param.type_annotation.dtype == "bfloat16":
+                input_data[i] = tvm.nd.empty(input_data[i].shape, "bfloat16").copyfrom(
+                    input_data[i]
+                )

Review Comment:
   why are we adding ONNX tests in this PR though? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] tvm-bot commented on pull request #14405: [TIR] Refactor BF16Legalize

Posted by "tvm-bot (via GitHub)" <gi...@apache.org>.
tvm-bot commented on PR #14405:
URL: https://github.com/apache/tvm/pull/14405#issuecomment-1484157144

   <!---bot-comment-->
   
   Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
   
   <!--bot-comment-ccs-start-->
    * cc @Hzfengsy, @junrushao, @quic-sanirudh, @shingjan <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
   
   <sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] junrushao commented on pull request #14405: [TIR] Refactor BF16Legalize

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on PR #14405:
URL: https://github.com/apache/tvm/pull/14405#issuecomment-1485791796

   Retriggering failed Android tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] tqchen commented on pull request #14405: [TIR] Refactor BF16Legalize

Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #14405:
URL: https://github.com/apache/tvm/pull/14405#issuecomment-1485775223

   note: the android failure is not related to this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] junrushao commented on a diff in pull request #14405: [TIR] Refactor BF16Legalize

Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #14405:
URL: https://github.com/apache/tvm/pull/14405#discussion_r1149923464


##########
src/target/llvm/codegen_llvm.cc:
##########
@@ -828,6 +828,10 @@ void CodeGenLLVM::CreateSerialFor(llvm::Value* begin, llvm::Value* end, llvm::Va
 llvm::Value* CodeGenLLVM::CreateCast(DataType from, DataType to, llvm::Value* value) {
   llvm::Type* target = DTypeToLLVMType(to);
   if (value->getType() == target) return value;
+  // TODO(tvm-team): consider add native support
+  ICHECK(!from.is_bfloat16()) << "BF16 needs to be storaged lowered first";
+  ICHECK(!to.is_bfloat16()) << "BF16 needs to be storaged lowered first";
+

Review Comment:
   seems possible to support software BF16 in upstream LLVM/MLIR world. leaving it to future work



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] tqchen commented on pull request #14405: [TIR] Refactor BF16Legalize

Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #14405:
URL: https://github.com/apache/tvm/pull/14405#issuecomment-1484157882

   cc @yangulei @Menooker 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] tqchen commented on pull request #14405: [TIR] Refactor BF16Legalize

Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #14405:
URL: https://github.com/apache/tvm/pull/14405#issuecomment-1485790017

   cc @vinx13 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] tqchen commented on a diff in pull request #14405: [TIR] Refactor BF16Legalize

Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on code in PR #14405:
URL: https://github.com/apache/tvm/pull/14405#discussion_r1149962830


##########
tests/python/frontend/onnx/test_forward.py:
##########
@@ -99,6 +99,14 @@ def get_tvm_output_with_vm(
             freeze_params=freeze_params,
             convert_config=convert_config,
         )
+        # handle the bfloat16 so we explicitly allocate
+        # bfloat16 arrays as input
+        for i, param in enumerate(mod["main"].params):
+            if param.type_annotation.dtype == "bfloat16":
+                input_data[i] = tvm.nd.empty(input_data[i].shape, "bfloat16").copyfrom(
+                    input_data[i]
+                )

Review Comment:
   This is needed to patch up onnx converting bf16 to uint16



##########
src/target/llvm/codegen_llvm.cc:
##########
@@ -828,6 +828,10 @@ void CodeGenLLVM::CreateSerialFor(llvm::Value* begin, llvm::Value* end, llvm::Va
 llvm::Value* CodeGenLLVM::CreateCast(DataType from, DataType to, llvm::Value* value) {
   llvm::Type* target = DTypeToLLVMType(to);
   if (value->getType() == target) return value;
+  // TODO(tvm-team): consider add native support
+  ICHECK(!from.is_bfloat16()) << "BF16 needs to be storaged lowered first";
+  ICHECK(!to.is_bfloat16()) << "BF16 needs to be storaged lowered first";
+

Review Comment:
   Good point, already left as a todo



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org