[GitHub] [incubator-mxnet] reminisce opened a new issue #17097: [mxnet 2.0][item 10.1] MXNet Imperative Op Invocation Overhead

[GitBox <gi...@apache.org>]

reminisce opened a new issue #17097: [mxnet 2.0][item 10.1] MXNet Imperative Op Invocation Overhead
URL: https://github.com/apache/incubator-mxnet/issues/17097
 
 
   MXNet imperative operator invocation overhead is as large as 30-60us, which is significant compared to the official NumPy operators with ~600ns overhead. This has negatively impacted the performance of applying MXNet to the models where many operators' kernel runtime duration is short, especially in the area of classic machine learning. We plan to address the problem in two steps:
   
   1. Short term: Use pybind11 to replace Python op API and ctypes/c api. Preliminary experiments show that the pure Python-C++ turnaround time by using Pybind is between 400-600ns, while the current Python op API using ctypes/c api costs more than 10us. We believe with the correct implementation, we can reduce the op invocation overhead to 2us including the time on FFI and engine.
   
   2. Long term: Adopt Python's C extension interface. NumPy did this by developing its own C API. This provides considerably less overhead compared to other solutions. However, it would cost much more engineering efforts by integrating this with our existing operator workflow in C++.
   
   @hzfan @hgt312

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services