[GitHub] [incubator-mxnet] leezu edited a comment on issue #16869: Cannot manipulate gradients during training in estimator

[GitBox <gi...@apache.org>]

leezu edited a comment on issue #16869: Cannot manipulate gradients during training in estimator
URL: https://github.com/apache/incubator-mxnet/issues/16869#issuecomment-555986191
 
 
   Thanks @liuzh91 for raising the issue. I agree the current `fit_batch` is buggy as it cannot support this standard use-case.
   
   What do you mean with "handlers are not supposed to apply gradients on network parameters"?
   
   If we allow handlers to apply gradient to network parameter, an alternative solution to your problem could be to remove the `trainer.step` from the current `def fit_batch` implementation and introduce a `GradientUpdateHandler`? Similar to how `LoggingHandler` and `MetricHandler` are constructed by default in `estimator.fit`, if not provided by the user, the `GradientUpdateHandler` would be default implement the `estimator.step` behavior.
   
   Users would be free to provide their own subclass of `GradientUpdateHandler` that could extend the handler to do gradient clipping or gradient accumulation.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services