[GitHub] [incubator-tvm] lfengad commented on issue #4990: [TF][Relay] BatchNorm support with run-time mean and variance calculation

[GitBox <gi...@apache.org>]

lfengad commented on issue #4990: [TF][Relay] BatchNorm support with run-time mean and variance calculation
URL: https://github.com/apache/incubator-tvm/pull/4990#issuecomment-595147201
 
 
   > I think we needn't add `_empty_for_training_mode_inference`. If we find `mean` / `variance` is `VarNode`, we should call `Mean` and `Variance`.
   > 
   > I don't think we should add `is_training` flag to relay `BatchNorm`. This should be done by users to make sure TF's model `BatchNorm`'s `is_training` flag be false. However, we still have user cases like you mention, so we could support as current implementation and don't add attribute to `BatchNorm`.
   
   Thank you so much for the quick reply!  
   Yeah, our current implementation is just to check whether `mean` / `variance` is empty `VarNode` (with zero dimension),  and then call `Mean` and `Variance` in BatchNormToInferUnpack.  Also as I understand, if `mean` / `variance` is `VarNode` but with non-zero dimension, it still has the possibility to hold the given pre-defined constant values and thus cannot be replaced with `Mean` \ `Variance`.   
   Thank you for the discussion!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services