You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by sandeep krishnamurthy <sa...@gmail.com> on 2019/02/21 19:11:18 UTC
[Design Review Request] Extending model save/load API
Hello MXNet community,
I have put up a design proposal to extend the MXNet model saving and
loading APIs to solve the following 2 problems:
1. Currently, model export/save APIs do not allow you to export the data
transformations as part of the model. By allowing this, we greatly simplify
the user experience during inference across all language bindings and also
we see a good performance gains.
2. Currently, model export/save APIs do not save input/output signature
like input data names, shapes etc.. By saving it as part of the model, we
simplify again the inference and model deployment logic.
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103090063
Thanks to our fellow community members (Zhi, Naveen, Pinar and Jake) for
their initial feedback. I request the community to review and provide
feedback on the proposal.
--
Sandeep Krishnamurthy
Re: [Design Review Request] Extending model save/load API
Posted by Sheng Zha <sz...@gmail.com>.
Hi Sandeep,
> users need to know input name
This information is already in the files from Gluon export API. Here's how
you get them:
In [1]: import mxnet as mx
...: net = mx.gluon.model_zoo.vision.alexnet(pretrained=True)
...: net.hybridize()
...: net(mx.nd.ones((1,3,224,224)))
...: net.export('alexnet')
...: net_sym = mx.sym.load('alexnet-symbol.json')
...: net_params = mx.nd.load('alexnet-0000.params')
...: print('input is {}'.format([n for n in net_sym.list_inputs() if
'arg:{}'.format(n) not in net_params.keys()]))
...: print('output is {}'.format(net_sym.list_outputs()))
...:
...:
input is ['data']
output is ['alexnet0_dense2_fwd_output']
> shape
It's true that shape needs to be specified before binding. The catch for
coding it into the symbol file is that many networks support variable size
data, such as RNN and CNN. Having static size makes the symbol file
strictly less useful. Also, those who develop the inference code would have
to know the shape at the time of binding because otherwise they won't know
if the data they feed is "legal".
IMO the proposed additional metadata seems to be coupling the model
development and deployment instead of decoupling them.
-sz
On Thu, Feb 21, 2019 at 2:36 PM sandeep krishnamurthy <
sandeep.krishna98@gmail.com> wrote:
> Hi Sheng,
>
> By stating unusable out of the box, I meant, when loading a model for
> inference, today, users need to know input name, shape for binding. This
> information if part of the model, could have simplified model training to
> model deployment path much easier and decoupled.
>
> Best,
> Sandeep
>
> On Thu, Feb 21, 2019 at 2:10 PM Sheng Zha <sz...@gmail.com> wrote:
>
> > Hi Sandeep,
> >
> > In the design doc, you stated that
> > > Input/Output signature are not part of the model: Saved model missing
> the
> > information about the input/output descriptions, like name/shape, making
> > the saved model unusable out of the box.
> >
> > Could you elaborate why you think this is the case?
> >
> > -sz
> >
> > On Thu, Feb 21, 2019 at 11:11 AM sandeep krishnamurthy <
> > sandeep.krishna98@gmail.com> wrote:
> >
> > > Hello MXNet community,
> > >
> > > I have put up a design proposal to extend the MXNet model saving and
> > > loading APIs to solve the following 2 problems:
> > > 1. Currently, model export/save APIs do not allow you to export the
> data
> > > transformations as part of the model. By allowing this, we greatly
> > simplify
> > > the user experience during inference across all language bindings and
> > also
> > > we see a good performance gains.
> > > 2. Currently, model export/save APIs do not save input/output signature
> > > like input data names, shapes etc.. By saving it as part of the model,
> we
> > > simplify again the inference and model deployment logic.
> > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103090063
> > >
> > > Thanks to our fellow community members (Zhi, Naveen, Pinar and Jake)
> for
> > > their initial feedback. I request the community to review and provide
> > > feedback on the proposal.
> > >
> > > --
> > > Sandeep Krishnamurthy
> > >
> >
>
>
> --
> Sandeep Krishnamurthy
>
Re: [Design Review Request] Extending model save/load API
Posted by sandeep krishnamurthy <sa...@gmail.com>.
Hi Sheng,
By stating unusable out of the box, I meant, when loading a model for
inference, today, users need to know input name, shape for binding. This
information if part of the model, could have simplified model training to
model deployment path much easier and decoupled.
Best,
Sandeep
On Thu, Feb 21, 2019 at 2:10 PM Sheng Zha <sz...@gmail.com> wrote:
> Hi Sandeep,
>
> In the design doc, you stated that
> > Input/Output signature are not part of the model: Saved model missing the
> information about the input/output descriptions, like name/shape, making
> the saved model unusable out of the box.
>
> Could you elaborate why you think this is the case?
>
> -sz
>
> On Thu, Feb 21, 2019 at 11:11 AM sandeep krishnamurthy <
> sandeep.krishna98@gmail.com> wrote:
>
> > Hello MXNet community,
> >
> > I have put up a design proposal to extend the MXNet model saving and
> > loading APIs to solve the following 2 problems:
> > 1. Currently, model export/save APIs do not allow you to export the data
> > transformations as part of the model. By allowing this, we greatly
> simplify
> > the user experience during inference across all language bindings and
> also
> > we see a good performance gains.
> > 2. Currently, model export/save APIs do not save input/output signature
> > like input data names, shapes etc.. By saving it as part of the model, we
> > simplify again the inference and model deployment logic.
> >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103090063
> >
> > Thanks to our fellow community members (Zhi, Naveen, Pinar and Jake) for
> > their initial feedback. I request the community to review and provide
> > feedback on the proposal.
> >
> > --
> > Sandeep Krishnamurthy
> >
>
--
Sandeep Krishnamurthy
Re: [Design Review Request] Extending model save/load API
Posted by Sheng Zha <sz...@gmail.com>.
Hi Sandeep,
In the design doc, you stated that
> Input/Output signature are not part of the model: Saved model missing the
information about the input/output descriptions, like name/shape, making
the saved model unusable out of the box.
Could you elaborate why you think this is the case?
-sz
On Thu, Feb 21, 2019 at 11:11 AM sandeep krishnamurthy <
sandeep.krishna98@gmail.com> wrote:
> Hello MXNet community,
>
> I have put up a design proposal to extend the MXNet model saving and
> loading APIs to solve the following 2 problems:
> 1. Currently, model export/save APIs do not allow you to export the data
> transformations as part of the model. By allowing this, we greatly simplify
> the user experience during inference across all language bindings and also
> we see a good performance gains.
> 2. Currently, model export/save APIs do not save input/output signature
> like input data names, shapes etc.. By saving it as part of the model, we
> simplify again the inference and model deployment logic.
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103090063
>
> Thanks to our fellow community members (Zhi, Naveen, Pinar and Jake) for
> their initial feedback. I request the community to review and provide
> feedback on the proposal.
>
> --
> Sandeep Krishnamurthy
>