[GitHub] KellenSunderland opened a new issue #10692: Executor's set_monitor_callback leaks memory in high-level languages.

[GitBox <gi...@apache.org>]

KellenSunderland opened a new issue #10692: Executor's set_monitor_callback leaks memory in high-level languages.
URL: https://github.com/apache/incubator-mxnet/issues/10692
 
 
   ## Description
   Executor's set_monitor_callback leaks memory in high-level languages.
   
   ## Environment info (Required)
   OSX, pip installed MXNet.
   
   ## Minimum reproducible example
   ```python
   import mxnet as mx
   count = 0
   
   def mon_callback(*args, **kwargs):
       global count
       count = count + 1
   
   def main():
       while True:
           process_request()[0].wait_to_read()
   
   def process_request():
       x = mx.sym.Variable('x')
       y = mx.sym.FullyConnected(x, num_hidden=1024)
       for i in range(0, 4):
           y = x + 1
       executor = y.bind(mx.cpu(), {'x': mx.nd.zeros((1024, ))})
       executor.set_monitor_callback(mon_callback)
       return executor.forward(is_train=False)
   
   if __name__ == "__main__":
       main()
   ```
   
   ## Comments
   *  Looks like the leak is due to this implementation: https://github.com/apache/incubator-mxnet/blob/master/src/executor/graph_executor.cc#L1461 which copies output ndarrays and then passes back raw pointers to them.  This might make sense for native languages, but if the language accessing the callback is python for example it becomes difficult to properly delete this memory.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services