You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mxnet.apache.org by "Chris Olivier (JIRA)" <ji...@apache.org> on 2018/03/12 17:06:00 UTC
[jira] [Comment Edited] (MXNET-11) Multithreaded Inference
[ https://issues.apache.org/jira/browse/MXNET-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395518#comment-16395518 ]
Chris Olivier edited comment on MXNET-11 at 3/12/18 5:05 PM:
-------------------------------------------------------------
I suppose the multiple threads would call the C API. Python is a bit tricky to get it to do multithreading, so I wouldn't be concerned with a python entry point at this point.
So above, you launched many processes, loaded a model and ran inference in parallel?
Or you loaded a model in many threads and ran inference through those?
The main problem right now is that there's not a good way to *share* parameters between graphs, so that would probably be some large bulk of the work. There's actually several other use-cases for this, including Keras integration.
This item relates to that: https://issues.apache.org/jira/browse/MXNET-28
was (Author: cjolivier01):
I suppose the multiple threads would call the C API. Python is a bit tricky to get it to do multithreading, so I wouldn't be concerned with a python entry point at this point.
So above, you launched many processes, loaded a model and ran inference in parallel?
Or you loaded a model in many threads and ran inference through those?
The main problem right now is that there's not a good way to *share* parameters between graphs, so that would probably be some large bulk of the work. There's actually several other use-cases for this, including Keras integration.
> Multithreaded Inference
> -----------------------
>
> Key: MXNET-11
> URL: https://issues.apache.org/jira/browse/MXNET-11
> Project: Apache MXNet
> Issue Type: Epic
> Components: MXNet Engine
> Reporter: Chris Olivier
> Priority: Major
> Labels: inference
>
> Add the ability to do multithreaded inference without using fork() or using multiple copies of a given model
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org