[GitHub] [incubator-mxnet] Wallart opened a new issue #16926: autograd.is_training() does not work on hybridized networks.

[GitBox <gi...@apache.org>]

Wallart opened a new issue #16926: autograd.is_training() does not work on hybridized networks.
URL: https://github.com/apache/incubator-mxnet/issues/16926
 
 
   Hello everyone,
   I might have found an issue about autograd.is_training().
   I've implemented a transformer. At training time my network is returning predictions and attention weights for plotting. During inference it's only returning predictions. After the training phase I'm doing validation on a new dataset. Here is a basic description of the problem
   
   Training loop : 
   `
   # training part
   with autograd.record():
     output, attn_w_1, attn_w_2 = transformerNet(someInputs)
     # loss compute, backward pass, trainer step, etc.
   # validation part
   output = transformerNet(someInputs) 
   # compute some score
   `
   Transformer hybrid_forward end :
   `
   # some stuff
   if autograd.is_training():
      return logits, attn_w_1, attn_w_2
   return logits
   `
   
   Everything is working fine except when I hybridize the network. During validation autograd.is_training() is always true and the method returns 3 variables instead of 1.
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services