[GitHub] [incubator-mxnet] TaoLv edited a comment on issue #16408: Add MXNet Ops for fast multihead attention

[GitBox <gi...@apache.org>]

TaoLv edited a comment on issue #16408: Add MXNet Ops for fast multihead attention
URL: https://github.com/apache/incubator-mxnet/pull/16408#issuecomment-549006049
 
 
   Thank you @eric-haibin-lin for clarifying the impact on MXNet Gluon. My change request is not only for that but also for an elaborative proposal about these new operators. Although they will live under `contrib op` category, I didn't see a clear scope definition about `contrib op` in MXNet.
   To make it clear, I hope below items can be described in the proposal:
   1. What's the current design? APIs, features, formulas, limitations, performance, ...
   2. Why cuDNN MHT, dynamic operator, or subgraph doesn't help here?
   3. Do we need call for contribution for CPU implementation?
   4. How long will these operators exist? If I understand correctly, once they're in, less likely they will be removed before next major release. So `contrib op` is not a trivial thing.
   5. Will they be replaced with other implementations? with the same operator interfaces? If not, how about backward compatibility?
   6. Do we encourage users and downstream projects to use them?
   7. Known issues or further plan to improve?
   
   I hope the proposal and more discussions can help other developers as well as users to understand these operators better and also make MXNet development more open.
   I don't want to be a hostile here. So it's not necessary for you to finish the proposal before merging this PR.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services