You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/02/17 18:55:17 UTC

[GitHub] [tvm] masahi opened a new pull request #7463: [TOPI, Relay] Support roi_align NHWC layout

masahi opened a new pull request #7463:
URL: https://github.com/apache/tvm/pull/7463


   I've been working on optimizing MaskRCNN/FasterRCNN, one I thing I found was that Ansor generates better code for NHWC layout. So I'm looking at improving NHWC end to end support / performance.
   
   `roi_align` only supports NCHW layout, so `layout_transform` is inserted before/after each `roi_align`. This layout transform turned out expensive on GPU, so I added NHWC impl to `roi_align` to remove `layout_transform`. This cuts Faster RCNN runtime by 8 milli second.
   
   Now, with Ansor NHWC tuning + `roi_align` improvement in this PR, we are beating pytorch by a large margin:
   ```
   TVM NHWC(Ansor) + cublas: 0.0589 milli sec
   PyTorch: 0.0738 milli sec
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi merged pull request #7463: [TOPI, Relay] Support roi_align NHWC layout

Posted by GitBox <gi...@apache.org>.

masahi merged pull request #7463:
URL: https://github.com/apache/tvm/pull/7463


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] comaniac commented on pull request #7463: [TOPI, Relay] Support roi_align NHWC layout

Posted by GitBox <gi...@apache.org>.

comaniac commented on pull request #7463:
URL: https://github.com/apache/tvm/pull/7463#issuecomment-780788665


   BTW, did you use ConvertLayout pass to convert the model layout? Would you need to add the corresponding layout converter (e.g., `register_convert_op_layout` or `InferCorrectLayout`) for RoI align to eliminate the `layout_transform`?
   
   Also out of curiousity, what GPU did you use the get the performance number?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] comaniac commented on pull request #7463: [TOPI, Relay] Support roi_align NHWC layout

Posted by GitBox <gi...@apache.org>.

comaniac commented on pull request #7463:
URL: https://github.com/apache/tvm/pull/7463#issuecomment-780796357


   Ah I see. That makes sense.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi commented on pull request #7463: [TOPI, Relay] Support roi_align NHWC layout

Posted by GitBox <gi...@apache.org>.

masahi commented on pull request #7463:
URL: https://github.com/apache/tvm/pull/7463#issuecomment-780792776


   @comaniac GPU is geforce 1070 ti
   
   Yeah, roi_align NHWC was like half done in https://github.com/apache/tvm/pull/6443 where `ConvertLayout` support was added but implementation was not. So until now we can get a relay graph with NHWC `roi_align` op in it, but we cannot compile it. It is only useful for e.g BYOC.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org