[GitHub] [incubator-mxnet] xcgoner opened a new issue #18669: Inconsistency of output and states in ZoneoutCell

[GitBox <gi...@apache.org>]

xcgoner opened a new issue #18669:
URL: https://github.com/apache/incubator-mxnet/issues/18669


   ## Description
   
   In the implementation of ZoneoutCell:
   https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/rnn/rnn_cell.py
   The output and states are using 2 different masks.
   However, if we use an RNNCell (e.g., LSTMCell or GRUCell) as the base cell, output is also a part of states (typically the first element of states).
   As a result, there will be cases like output is using the previous output, while the output inside states is the next output, which causes inconsistency.
   A correct implementation should be: applying a mask to the output first, and then replace the output inside states with the new output, then apply another mask to the remaining part of states.
   Maybe a better way to fix this is to directly implement LSTMCell and GRUCell with Zoneout, instead of using ModifierCell.
   
   Note that the symbolic version of ZoneoutCell has the same issue, though I only report the issue of the gluon version here.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org