Proposal for optimizing Gluon dynamic models for seamless deployment

["Zheng, Da" <dz...@amazon.com>]

Hello,

Scientists like to develop models with Gluon or Pytorch and hand the models over to engineer for deployment. It takes a lot of effort to deploy these models because engineers usually need to reimplement the models (this is especially for NLP and speech models). Recently, Pytorch announced their next release v1.0 in a near future, which will integrate Pytorch and Caffe2 for easy deployment of any models in Pytorch. Although Gluon is heading towards this direction, it currently doesn’t hybridize and export any models for deployment, especially the ones with control flows.

Previously, I proposed to add symbolic control flow operators to MXNet. I would like to extend the previous proposal and advance Gluon to hybridize dynamic models with control flows and deploy them seamlessly. The details of the proposal can be found here: https://cwiki.apache.org/confluence/display/MXNET/Optimize+dynamic+neural+network+models+with+control+flow+operators

Please let me know if you have any comments and suggestions.

Thanks,
Da