[GitHub] [incubator-tvm] masahi edited a comment on issue #5306: [Torch] Support Python list, more realistic recurrent networks

[GitBox <gi...@apache.org>]

masahi edited a comment on issue #5306: [Torch] Support Python list, more realistic recurrent networks
URL: https://github.com/apache/incubator-tvm/pull/5306#issuecomment-612441129
 
 
   > LGTM. Is torchscript list immutable or mutable like python's list?
   
   Yes it is mutable. List `append` is mapped to `aten::append` in Torchscript and it is entirely side-effecting operation. See below for a simple module that only does list append and its Torchscript representation.
   
    Even though variables that are updated in the loop are supposed to be passed to `prim::Loop` op to become loop variables, this does not apply to side effecting operations like list append. Translating this module to Relay is complicated because also in Relay loop variables are the only ones that can be updated between iteration. If we naively translate it, the list `outputs.1` below appears free in the loop and can not be updated.
   
   ```Py
   class ListAppend(nn.Module):
       def forward(self, input):
           # type: (Tensor) -> List[Tensor]
           outputs = []
           for i in range(input.size(0)):
               outputs.append(input)
           return outputs
   ```
   
   ```
   graph(%self : __torch__.ListAppend,
         %input.1 : Tensor):
     %8 : bool = prim::Constant[value=1]() # rnn_test.py:142:8
     %4 : int = prim::Constant[value=0]() # rnn_test.py:142:34
     %outputs.1 : Tensor[] = prim::ListConstruct()
     %5 : int = aten::size(%input.1, %4) # rnn_test.py:142:23
      = prim::Loop(%5, %8) # rnn_test.py:142:8
       block0(%i : int):
         %12 : Tensor[] = aten::append(%outputs.1, %input.1) # rnn_test.py:143:12
         -> (%8)
     return (%outputs.1)
   ```
   
   To workaround the difficulty of list append, I use list concat to append one element at the tail of a list. The original LSTM models in Pytorch repo do not use list append either and use concat instead, probably for the same reason. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services