[GitHub] [incubator-mxnet] TaoLv commented on issue #16408: Add MXNet Ops for fast multihead attention

[GitBox <gi...@apache.org>]

TaoLv commented on issue #16408: Add MXNet Ops for fast multihead attention
URL: https://github.com/apache/incubator-mxnet/pull/16408#issuecomment-545247834
 
 
   Actually I personally don't like this kind of super fused operator which doesn't have explicit semantics and also lacks the flexibility as a general operator. I'm not sure if we should encourage our users to call these strange and half-baked operators in their models and how to do that if they already implemented their models? Besides, it also increase the burden for the community to maintain the complicated logic in the code.
   I totally agree that fusions can bring performance improvement and make our framework outperforms other competing products. But I still think it would be better if fusions can be done in following ways:
   1. Graph fusion pass: we have subgraph in NNVM and more features in the coming Relay IR;
   2. Accelerating library support: integrate APIs and primitives from library and make them an operator is more acceptable. Just like what we did for fused RNN layers?
   3. Maintain your customized fused operators in a separated repository and teach users how to integrate them into MXNet if they really want. I think the dynamic custom operator feature may help in this situation?
   Just my two cents.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services