You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/03/31 00:10:28 UTC
[GitHub] [incubator-mxnet] djaym7 opened a new issue #17939: Ways of
Freezing part of parameter and not the whole layer.
djaym7 opened a new issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939
Using grad_req == null freezes whole layer, i want to freeze certain weights in the current parameter, how to do this ?
Eg: Freezing 1 of 5 kernels in conv parameter during training..
Thanks
@zhreshold help ?
#question
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] djaym7 closed issue #17939: Ways of Freezing part
of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
djaym7 closed issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] zhreshold commented on issue #17939: Ways of
Freezing part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
zhreshold commented on issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939#issuecomment-606856072
Unless you separate the e.g. conv layer to multiple one and concat you can't freeze partial weights.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] djaym7 closed issue #17939: Ways of Freezing part
of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
djaym7 closed issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu edited a comment on issue #17939: Ways of
Freezing part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
leezu edited a comment on issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939#issuecomment-606348300
You can't freeze part of a parameter. However, you can overwrite the gradient arrays, once computed, accordingly and set those parts to 0 that you like frozen.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu commented on issue #17939: Ways of Freezing
part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939#issuecomment-611116192
It may not have the same performance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] djaym7 commented on issue #17939: Ways of
Freezing part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
djaym7 commented on issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939#issuecomment-607365929
Instead of freezing, i can use a custom layer and slice them back in the weights in forward and backward passes, wouldn't it serve the same purpose ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] djaym7 opened a new issue #17939: Ways of
Freezing part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
djaym7 opened a new issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939
Using grad_req == null freezes whole layer, i want to freeze certain weights in the current parameter, how to do this ?
Eg: Freezing 1 of 5 kernels in conv parameter during training..
Thanks
@zhreshold help ?
#question and not #featureRequest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] djaym7 edited a comment on issue #17939: Ways of
Freezing part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
djaym7 edited a comment on issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939#issuecomment-607365929
Instead of freezing, i can use a custom layer and slice them back in the weights in forward and backward passes so that they won't be counted in calculating gradients (trying to prune them), wouldn't it serve the same purpose ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu commented on issue #17939: Ways of Freezing
part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939#issuecomment-606348300
You can't freeze part of a parameter. However, you can overwrite the gradient arrays, once computed, accordingly and set those parts to 0 that you like "freezed".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] djaym7 commented on issue #17939: Ways of
Freezing part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
djaym7 commented on issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939#issuecomment-606945552
Okay, thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] djaym7 edited a comment on issue #17939: Ways of
Freezing part of parameter and not the whole layer.
Posted by GitBox <gi...@apache.org>.
djaym7 edited a comment on issue #17939: Ways of Freezing part of parameter and not the whole layer.
URL: https://github.com/apache/incubator-mxnet/issues/17939#issuecomment-607365929
Instead of freezing, i can use a custom layer and slice them back in the weights in forward and backward passes so that they won't be counted in calculating gradients, wouldn't it serve the same purpose ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services