[GitHub] II-Matto opened a new issue #8908: How to use an LSTM Layer in CNN?

[GitBox <gi...@apache.org>]

II-Matto opened a new issue #8908: How to use an LSTM Layer in CNN?
URL: https://github.com/apache/incubator-mxnet/issues/8908
 
 
   I would like to use an LSTM layer in CNN. Specifically, the network will first have a few convolutional layers, producing output `y1` from input `x`. Then `y1` is interpreted as a sequence and fed into an LSTM, producing output `y2` at the last step. Finally `y1` and `y2` is fed together into one or more layers, which can be convolutional, fully-connected or any other reasonable layers, to produce the final output `z`. The whole network is supposed to be trained jointly in an end-to-end manner.
   ```
   x ---(CONV)---> y1 ---(LSTM)---> y2
   [y1, y2] ---(CONV/FC/...)---> z
   ```
   How can I create the symbol of such a network with MXNet?
   
   I can conveniently achieve this in Caffe. The only thing that I need to do is adding an LSTM layer in prototxt just like adding any other layers.
   
   (BTW, the size of inputs of the network will be determined on the fly, and module binding is performed each time the input size changes.)
   
   Currently I am considering to use the [RNN Cell Symbol API](https://mxnet.incubator.apache.org/api/python/symbol/rnn.html). But it seems that problems exist, e.g. I won't be able to `reset the cell when unrolling/stepping for a new sequence by calling lstm_cell.reset()`, since the `LSTMCell` is only used when defining the symbol of the network. If using the RNN Cell Symbol API is a feasible solution, can someone provide an accessible example?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services