[GitHub] [incubator-mxnet] agrabows commented on a change in pull request #19971: [FEATURE] Fusing Leaky Relu operator in Fully Connected

[GitBox <gi...@apache.org>]

agrabows commented on a change in pull request #19971:
URL: https://github.com/apache/incubator-mxnet/pull/19971#discussion_r618315195



##########
File path: src/operator/subgraph/mkldnn/mkldnn_fc_property.h
##########
@@ -102,6 +102,16 @@ class SgMKLDNNFCSelector : public SubgraphSelector {
             return true;
           }
         }
+        if (new_node.op() == Op::Get("LeakyReLU")) {
+          const LeakyReLUParam &param = nnvm::get<LeakyReLUParam>(new_node.attrs.parsed);
+          if (param.act_type == leakyrelu::kLeakyReLU ||
+              param.act_type == leakyrelu::kELU ||
+              param.act_type == leakyrelu::kGELU) {
+            matched_list_.push_back(&new_node);
+            status_ = kSuccess;
+            return true;
+          }

Review comment:
       Those were activation types that can be easily fused with FC in this PR. We plan to enable other activation fuses in the next PR as they need different approach.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org