[GitHub] [tvm] Laurawly edited a comment on pull request #6839: [ONNX] NMS in ONNX

[GitBox <gi...@apache.org>]

Laurawly edited a comment on pull request #6839:
URL: https://github.com/apache/tvm/pull/6839#issuecomment-742728283


   > @Laurawly I took the mxnet example you provided and ran it with the debug runtime. It required a little bit of editing, APIs have changed slightly since that tutorial was written. Anyway, this is what I get on my 1070 TI with Thrust enabled.
   > 
   > main:
   > 
   > ```
   > Ops                                                                                                     Time(us)    Time(%)  Shape
   > ---                                                                                                     --------    -------  ----- 
   > fused_vision_non_max_suppression                                                                        139329.0    74.66    (1, 122640, 6)
   > fused_vision_get_valid_counts                                                                           124.255     0.067    (1, 122640, 6)     
   > ```
   > 
   > this PR:
   > 
   > ```
   > fused_vision_get_valid_counts                                                                           46138.3    50.891   (1, 122640, 6)  
   > fused_vision_non_max_suppression                                                                        12319.8    13.589   (1, 122640, 6)
   > ```
   > 
   > The get valid counts function slow down, but I'm actually seeing the total runtime of these ops decrease from 139.3ms to 58.5ms
   > 
   > My modifications to the example can be found here: https://gist.github.com/mbrookhart/df25427cbbfb3c73ed16be72c8525610
   
   The time measurement is not as fast as before because of this PR: https://github.com/apache/tvm/pull/7005. If you reverse back this one, you should get fairly good performance without any effect on the correctness. Because after my improvement, nms is not a bottleneck of the ssd model anymore while in your measurement, it seems it still is. So my point is your changes to PR #7005 here: https://github.com/apache/tvm/blob/f332512629e94fd8d761f3dd002fd3aa91673ce0/python/tvm/topi/cuda/nms.py#L470-L483 should be faster and thus gets the performance improvement. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org