You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mxnet.apache.org by "Chris Olivier (JIRA)" <ji...@apache.org> on 2018/03/12 17:06:00 UTC

[jira] [Comment Edited] (MXNET-11) Multithreaded Inference

    [ https://issues.apache.org/jira/browse/MXNET-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395518#comment-16395518 ] 

Chris Olivier edited comment on MXNET-11 at 3/12/18 5:05 PM:
-------------------------------------------------------------

I suppose the multiple threads would call the C API.  Python is a bit tricky to get it to do multithreading, so I wouldn't be concerned with a python entry point at this point.

So above, you launched many processes, loaded a model and ran inference in parallel?

Or you loaded a model in many threads and ran inference through those?

The main problem right now is that there's not a good way to *share* parameters between graphs, so that would probably be some large bulk of the work. There's actually several other use-cases for this, including Keras integration.

This item relates to that:  https://issues.apache.org/jira/browse/MXNET-28

 


was (Author: cjolivier01):
I suppose the multiple threads would call the C API.  Python is a bit tricky to get it to do multithreading, so I wouldn't be concerned with a python entry point at this point.

So above, you launched many processes, loaded a model and ran inference in parallel?

Or you loaded a model in many threads and ran inference through those?

The main problem right now is that there's not a good way to *share* parameters between graphs, so that would probably be some large bulk of the work. There's actually several other use-cases for this, including Keras integration.

 

 

> Multithreaded Inference
> -----------------------
>
>                 Key: MXNET-11
>                 URL: https://issues.apache.org/jira/browse/MXNET-11
>             Project: Apache MXNet
>          Issue Type: Epic
>          Components: MXNet Engine
>            Reporter: Chris Olivier
>            Priority: Major
>              Labels: inference
>
> Add the ability to do multithreaded inference without using fork() or using multiple copies of a given model



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org