[GitHub] [incubator-mxnet] eric-haibin-lin edited a comment on issue #16408: Add MXNet Ops for fast multihead attention

[GitBox <gi...@apache.org>]

eric-haibin-lin edited a comment on issue #16408: Add MXNet Ops for fast multihead attention
URL: https://github.com/apache/incubator-mxnet/pull/16408#issuecomment-548921959
 
 
   I am still not comfortable with the `bwd_ignore_zero_init ` flag. The `MXNET_ENABLE_EXEC_ADDTO ` env var does not exist nor take any effect in the current version of mxnet. I'd suggest we remove this flag and related code for this PR. After `MXNET_ENABLE_EXEC_ADDTO ` is properly introduced by other PRs, we can add relevant optimization around it. 
   
   > I do see there are some useful discussions about the design and flags used in this PR. Also seems you already have considerations about future improvement with MHA from cuDNN, dynamic customized operator or with subgraph and fusion pass. So I would hope you can prepare a design proposal to cleanly describe the design, the limitations, user suggestions, and your further plans. Please also cross post the proposal to dev@ so it can get fully discussed and I also think the community can learn much from that. If possible, some guys can help to fill the missing part on CPU side.
   > BTW, what does that mean to MXNet Gluon if the CPU implementation is missing for an operator? @eric-haibin-lin
   
   I agree a design proposal is helpful, but it is not required for a contrib op. The purpose for contrib ops is that ppl who are interested in this op can try and use the op in gluonnlp Block as @Caenorst suggested, and developers can gather timely feedbacks. 
   
   The work on official multi-head attention block API / operator API, would definitely require a proposal and discussion on dev@ since it involves developers from different hardwares including CPU and GPU. 
   
   As far as gluon-nlp is concerned, I will not consider integrating this op (without CPU support) to the official API for now. However, we will redirect users who are interested to the example usage provided by @Caenorst if they only care about using GPUs. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services