[GitHub] [incubator-mxnet] Caenorst edited a comment on issue #16408: Add MXNet Ops for fast multihead attention

[GitBox <gi...@apache.org>]

Caenorst edited a comment on issue #16408: Add MXNet Ops for fast multihead attention
URL: https://github.com/apache/incubator-mxnet/pull/16408#issuecomment-545583954
 
 
   @TaoLv I agree that the dynamic custom operators feature would be a great fit here, but since it is not yet ready contrib is the next best place for such operator, also I did not modified any core component of MXNet or any existing model. Of course having an automatic fusion is better, but you first need the Op before having the fusion in a follow up PR :)
   
   @eric-haibin-lin We intend to provide as a follow up a wrapper op that could be used for this and cudnn multihead attention that would unify this, but we don't want to wait for the cudnn multihead attention to cover all the usecases we needed right now.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services