You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "youxiudeshouyeren (via GitHub)" <gi...@apache.org> on 2023/09/06 12:34:46 UTC

[GitHub] [tvm] youxiudeshouyeren opened a new pull request, #15682: [Relay] Fuse clip operators in yolov5

youxiudeshouyeren opened a new pull request, #15682:
URL: https://github.com/apache/tvm/pull/15682

   When I specify the ReLU6 activation function in yolov5, This redundancy occurs.
   For example:
   ![image](https://github.com/apache/tvm/assets/41790911/add17bfa-2f3c-4553-b8fe-7c140e50d847)
   It generates this IR:
   ```
     %233 = nn.conv2d(%231, meta[relay.Constant][54] /* ty=Tensor[(256, 256, 1, 1), int8] */, padding=[0, 0, 0, 0], channels=256, kernel_size=[1, 1], out_dtype="int32") /* ty=Tensor[(1, 256, 16, 16), int32] */;
     %234 = add(%233, meta[relay.Constant][55] /* ty=Tensor[(256, 1, 1), int32] */) /* ty=Tensor[(1, 256, 16, 16), int32] */;
     %235 = fixed_point_multiply(%234, multiplier=2097356939, shift=-6) /* ty=Tensor[(1, 256, 16, 16), int32] */;
     %236 = clip(%235, a_min=-128f, a_max=127f) /* ty=Tensor[(1, 256, 16, 16), int32] */;
     %237 = cast(%236, dtype="int8") /* ty=Tensor[(1, 256, 16, 16), int8] */;
     %238 = clip(%237, a_min=0f, a_max=48f) /* ty=Tensor[(1, 256, 16, 16), int8] */;
   ```
   
   The redundancy pattern is:
   ```
    Example:
     %1 = clip(%0, a_min=0f, a_max=255f) [type=int32]
     %2 = cast(%1, dtype="uint8") [type=uint8]
     %3 = clip(%2, a_min=20f, a_max=66f) [type=uint8]
   ```
   
   It can be optimized to:
   ```
    %1 = clip(%0, a_min=20f, a_max=66f) [type=uint8]
    %2 = cast(%1, dtype="uint8") [type=uint8]
   ```
   
   This PR is to simplify `clip->cast->clip` to `clip->cast`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] youxiudeshouyeren commented on pull request #15682: [Relay] Fuse clip operators in yolov5

Posted by "youxiudeshouyeren (via GitHub)" <gi...@apache.org>.
youxiudeshouyeren commented on PR #15682:
URL: https://github.com/apache/tvm/pull/15682#issuecomment-1711006449

   
   
   
   
   > Thanks for the PR! @youxiudeshouyeren. It seems that this example should have been optimized by `SimplifyClip`/`SimplifyClipAndConsecutiveCast` passes, but it was not executed. Perhaps you need to prioritize checking why existing passes have not been optimized to this pattern.
   
   They are different optimizations and both have different patterns. SimplifyClipAndConsecutiveCast targets clip(int32)-cast(uint8)->cast(int32). This simplification targets clip(int32)->cast(int8)->clip(int8). I'm sure the pass SimplifyClipAndConsecutiveCast is enabled, but it doesn't eliminate this redundancy in yolov5.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] youxiudeshouyeren commented on a diff in pull request #15682: [Relay] Fuse clip operators in yolov5

Posted by "youxiudeshouyeren (via GitHub)" <gi...@apache.org>.
youxiudeshouyeren commented on code in PR #15682:
URL: https://github.com/apache/tvm/pull/15682#discussion_r1319297402


##########
src/relay/transforms/simplify_expr.cc:
##########
@@ -326,6 +326,56 @@ static std::vector<int> GetTransposeAxisOrder(const Call& call, int ndim) {
   return std::move(attr_axes);
 }
 
+/*!
+ * \brief SimplifyClipCastClip matches the pattern clip->cast->clip and fuse clips based on Clip
+ *    min/max values.
+ *
+ * Example:
+ *   %1 = clip(%0, a_min=0f, a_max=255f) [type=int32]
+ *   %2 = cast(%1, dtype="uint8") [type=uint8]
+ *   %3 = clip(%2, a_min=20f, a_max=66f) [type=uint8]
+ *
+ * Optimized to (fuse Clips):
+ *   %1 = clip(%0, a_min=20f, a_max=66f) [type=uint8]
+ *   %2 = cast(%1, dtype="uint8") [type=uint8]
+ */

Review Comment:
   I realize it was my example that was written incorrectly, the first data type is int32. It should be:
   ```
    * Optimized to (fuse Clips):
    *   %1 = clip(%0, a_min=20f, a_max=66f) [type=int32]
    *   %2 = cast(%1, dtype="uint8") [type=uint8]
    */
   ```
   I'll correct the example in the comment.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] jikechao commented on a diff in pull request #15682: [Relay] Fuse clip operators in yolov5

Posted by "jikechao (via GitHub)" <gi...@apache.org>.
jikechao commented on code in PR #15682:
URL: https://github.com/apache/tvm/pull/15682#discussion_r1319294898


##########
src/relay/transforms/simplify_expr.cc:
##########
@@ -326,6 +326,56 @@ static std::vector<int> GetTransposeAxisOrder(const Call& call, int ndim) {
   return std::move(attr_axes);
 }
 
+/*!
+ * \brief SimplifyClipCastClip matches the pattern clip->cast->clip and fuse clips based on Clip
+ *    min/max values.
+ *
+ * Example:
+ *   %1 = clip(%0, a_min=0f, a_max=255f) [type=int32]
+ *   %2 = cast(%1, dtype="uint8") [type=uint8]
+ *   %3 = clip(%2, a_min=20f, a_max=66f) [type=uint8]
+ *
+ * Optimized to (fuse Clips):
+ *   %1 = clip(%0, a_min=20f, a_max=66f) [type=uint8]
+ *   %2 = cast(%1, dtype="uint8") [type=uint8]
+ */

Review Comment:
   Thanks for the PR! @youxiudeshouyeren. This optimization behavior seems not thorough enough; It can be further optimized to  `%1 = clip(%0, a_min=20f, a_max=66f) [type=uint8]`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Relay] Fuse clip operators in yolov5 [tvm]

Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen closed pull request #15682: [Relay] Fuse clip operators in yolov5
URL: https://github.com/apache/tvm/pull/15682


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] youxiudeshouyeren commented on pull request #15682: [Relay] Fuse clip operators in yolov5

Posted by "youxiudeshouyeren (via GitHub)" <gi...@apache.org>.
youxiudeshouyeren commented on PR #15682:
URL: https://github.com/apache/tvm/pull/15682#issuecomment-1708277499

   Code Review: @Hzfengsy @junrushao @Lunderberg Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] youxiudeshouyeren commented on pull request #15682: [Relay] Fuse clip operators in yolov5

Posted by "youxiudeshouyeren (via GitHub)" <gi...@apache.org>.
youxiudeshouyeren commented on PR #15682:
URL: https://github.com/apache/tvm/pull/15682#issuecomment-1708406335

   @tvm-bot re-run


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] jikechao commented on pull request #15682: [Relay] Fuse clip operators in yolov5

Posted by "jikechao (via GitHub)" <gi...@apache.org>.
jikechao commented on PR #15682:
URL: https://github.com/apache/tvm/pull/15682#issuecomment-1711368687

   @youxiudeshouyeren Thanks for your explanation! 
   I mean that this redundancy may be caused by a bug in the implementation of `SimplifyClip`. 
   Definitely, adding a brand new pass in this PR can solve this problem. However, It will be better to fix this bug rather than add the new pass.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] jikechao commented on a diff in pull request #15682: [Relay] Fuse clip operators in yolov5

Posted by "jikechao (via GitHub)" <gi...@apache.org>.
jikechao commented on code in PR #15682:
URL: https://github.com/apache/tvm/pull/15682#discussion_r1319294898


##########
src/relay/transforms/simplify_expr.cc:
##########
@@ -326,6 +326,56 @@ static std::vector<int> GetTransposeAxisOrder(const Call& call, int ndim) {
   return std::move(attr_axes);
 }
 
+/*!
+ * \brief SimplifyClipCastClip matches the pattern clip->cast->clip and fuse clips based on Clip
+ *    min/max values.
+ *
+ * Example:
+ *   %1 = clip(%0, a_min=0f, a_max=255f) [type=int32]
+ *   %2 = cast(%1, dtype="uint8") [type=uint8]
+ *   %3 = clip(%2, a_min=20f, a_max=66f) [type=uint8]
+ *
+ * Optimized to (fuse Clips):
+ *   %1 = clip(%0, a_min=20f, a_max=66f) [type=uint8]
+ *   %2 = cast(%1, dtype="uint8") [type=uint8]
+ */

Review Comment:
   BTW, this optimization behavior seems not thorough enough; It can be further optimized to  `%1 = clip(%0, a_min=20f, a_max=66f) [type=uint8]`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org