You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2019/10/30 23:43:51 UTC
[GitHub] [incubator-mxnet] samskalicky commented on a change in pull request
#16408: Add MXNet Ops for fast multihead attention
samskalicky commented on a change in pull request #16408: Add MXNet Ops for fast multihead attention
URL: https://github.com/apache/incubator-mxnet/pull/16408#discussion_r340906555
##########
File path: src/operator/contrib/transformer-inl.h
##########
@@ -34,6 +34,18 @@
namespace mxnet {
namespace op {
+struct InterleavedMatMulParam : public dmlc::Parameter<InterleavedMatMulParam> {
+ int heads;
+ bool bwd_ignore_zero_init;
+ DMLC_DECLARE_PARAMETER(InterleavedMatMulParam) {
+ DMLC_DECLARE_FIELD(heads)
+ .describe("Set number of heads");
+ DMLC_DECLARE_FIELD(bwd_ignore_zero_init)
+ .describe("Make backward pass ignore AddTo and not init to 0.")
Review comment:
can you add more of what you said to the description? this is good info
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services