[GitHub] ptrendx opened a new pull request #14290: Optimize NMS

[GitBox <gi...@apache.org>]

ptrendx opened a new pull request #14290: Optimize NMS
URL: https://github.com/apache/incubator-mxnet/pull/14290
 
 
   ## Description ##
   Current Box NMS forward path is very slow on the GPU, because it launches potentially thousands very tiny kernels with exposed launch latency. This PR introduces a new kernel for this slow part of the NMS op, reducing the time significantly.
   
   On single GPU, training Faster RCNN from GluonCV model zoo, the time to perform this portion of NMS improved by 50x, from 100 ms to 2.1 ms.
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
   - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [x] GPU path of Box NMS forward op changed from previous nms_impl kernel to the new optimized kernel 
   
   ## Comments ##
   - While this was the biggest problem with NMS op, there are other ones, now exposed because the biggest offender got significantly faster. Namely: using thrust for sorting introduces syncs, allocations and deallocations in the op; there are 2 Slice operations that in the case of FRCNN take together ~18ms (which is 20% of the end to end time in FP16 training). I don't have time currently to try to fix them though :-(.  @zhreshold FYI

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services