[GitHub] [tvm] kfeng123 commented on pull request #15068: [Relay] Improve the "clip" op optimization in simplify expr pass

["kfeng123 (via GitHub)" <gi...@apache.org>]

kfeng123 commented on PR #15068:
URL: https://github.com/apache/tvm/pull/15068#issuecomment-1591430778

   > By the way, what is motivation for this change? New case came from the real life DNN model... or this is required for some kind of synthetic test?
   
   The motivation is just to improve simplify_expr pass. Did not come from a real model or a test case...
   
   Nevertheless, this improvement may be practical after I develop further improvements. (I plan to give a series of PR to improve simplify_expr and related things shortly. There is still a lot of room for improvement in simplify_expr.)
   
   Suppose we already have an improved optimization for "cast" op (although we do not have for now). Consider the following model fragment:
   ```
   %input // uint8 
   %0 = cast(%input, "float32") // float32
   %1 = transpose(%0, axes = [1,0]) // float32
   %2 = clip(%0, 0f, 255f) // float32
   
   ```
   Note that the “cast” and "transpose" are commutative, and a lazy cast make transpose exected in uint8, which is faster:
   ```
   %input // uint8 
   %0 = transpose(%input, axes = [1,0]) // uint8
   %1 = cast(%0, "float32") // float32
   %2 = clip(%0, 0f, 255f) // float32
   ```
   
   Next,  note that the “cast” and "clip" are commutative, which leads to
   ```
   
   %input // uint8 
   %0 = transpose(%input, axes = [1,0]) // uint8
   %1 = clip(%0, 0f, 255f) // uint8
   %2 = cast(%0, "float32") // float32
   
   ```
   For now, we can use the pass in this PR to remove clip. (Note that the original pass can not remove the clip in this case!)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org