[GitHub] [tvm] srkreddy1238 commented on pull request #13325: [TRANSFORM] Fix virtual device annotation issue with BYOC subgraphs

[GitBox <gi...@apache.org>]

srkreddy1238 commented on PR #13325:
URL: https://github.com/apache/tvm/pull/13325#issuecomment-1308171709

   
   PushVirtualDevice and PopVirtualDevice maintains the current VirtualDevice information on stack ```expr_virtual_devices_ ```. These calls are invoked across a block parsing (FunctionNodes, CallNode ..etc). The stack (expr_virtual_devices_) grows as we nest into the functions.
   
   ```
   def @tvmgen_default_clml_main_17(%clml_17_i0: Tensor[(1, 2048), float32] /* ty=Tensor[(1, 2048), float32] */, Inline=1, Compiler="clml", global_symbol="tvmgen_default_clml_main_17", Primitive=1) -> Tensor[(1, 1000), float32] {
     %245 = fn (%FunctionVar_0_01: Tensor[(1, 2048), float32] /* ty=Tensor[(1, 2048), float32] */, PartitionedFromPattern="nn.dense_nn.bias_add_", Composite="clml.dense") -> Tensor[(1, 1000), float32] {
       %244 = nn.dense(%FunctionVar_0_01, meta[relay.Constant][257] /* ty=Tensor[(1000, 2048), float32] */, units=1000) /* ty=Tensor[(1, 1000), float32] */;
       add(%244, meta[relay.Constant][258] /* ty=Tensor[(1, 1000), float32] */) /* ty=Tensor[(1, 1000), float32] */
     } /* ty=fn (Tensor[(1, 2048), float32]) -> Tensor[(1, 1000), float32] */;
     %245(%clml_17_i0) /* ty=Tensor[(1, 1000), float32] */
   }
   
   ```
   
   In the above IR the composite function ```%245 = fn (%FunctionVar_0_01``` will not have a VirtualDevice definition (``` VirtualDevice(?)```) and the PushVirtualDevice fails. Whereas at end of this function parsing the PopVirtualDevice removes the VirtualDevice corresponding to the our scope function. From here all annotations fail due to empty stack.
   
   
   I figured it out as part of large network. I am working on to facilitate a test case for the same.
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org