[GitHub] ptrendx opened a new pull request #13471: Optimization of metric evaluation

[GitBox <gi...@apache.org>]

ptrendx opened a new pull request #13471: Optimization of metric evaluation
URL: https://github.com/apache/incubator-mxnet/pull/13471
 
 
   ## Description ##
   Currently metrics are mostly evaluated on CPU using NumPy. Due to Python GIL, they are evaluated in single thread, sequentially, which may become a problem once number of used GPUs is large enough.
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] Changes are complete (i.e. I finished coding on this PR)
   - [x] All changes have test coverage
   - [x] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
   
   ### Changes ###
   - [x] Changed TopKAccuracy metric implementation from using numpy.argsort to numpy.argpartition, which from my experiments on NDArrays of shape (208,1000) and top_k=5 is ~4x faster 
   - [x] Added global statistics to metrics. We noticed the problem with metrics speed after PR #12182 which introduced computing the metrics twice (once for giving the immediate values for accuracy and once for computing accuracy over the entire epoch). This is wasteful, since the calculations needed are exactly the same in this case. This PR introduces additional fields in EvalMetric class (global_sum_metric and global_num_inst), accessors for them (get_global and get_global_name_values) and function reset_local which performs reset only on non-global versions of statistics. It also modifies all current metrics to be able to use the global statistics. That way, while the code is backward compatible (one can still just use get/get_name_values/reset functions), it also eliminates overhead introduced by calculating statistics twice in fit function.
   
   CC @vandanavk for comments. 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services