[GitHub] TccccD commented on a change in pull request #12440: Add stable nrm2 for L2 normalization

[GitBox <gi...@apache.org>]

TccccD commented on a change in pull request #12440: Add stable nrm2 for L2 normalization
URL: https://github.com/apache/incubator-mxnet/pull/12440#discussion_r238906809
 
 

 ##########
 File path: src/operator/l2_normalization-inl.h
 ##########
 @@ -87,22 +88,24 @@ class L2NormalizationOp : public Operator {
     Stream<xpu> *s = ctx.get_stream<xpu>();
     TShape orig_shape = in_data[l2_normalization::kData].shape_;
     if (param_.mode == l2_normalization::kInstance) {
+      TShape small = out_data[1].shape_;
+      ReduceAxesComputeImpl<xpu, mxnet::op::mshadow_op::nrm2, false,
+        mxnet::op::mshadow_op::identity>(ctx, in_data, req,
+        { out_data[l2_normalization::kNorm] }, small);
       Shape<2> dshape = Shape2(orig_shape[0],
         orig_shape.ProdShape(1, orig_shape.ndim()));
       Tensor<xpu, 2, DType> data = in_data[l2_normalization::kData]
         .get_with_shape<xpu, 2, DType>(dshape, s);
       Tensor<xpu, 2, DType> out = out_data[l2_normalization::kOut]
         .get_with_shape<xpu, 2, DType>(dshape, s);
       Tensor<xpu, 1, DType> norm = out_data[l2_normalization::kNorm].get<xpu, 1, DType>(s);
-      norm = sumall_except_dim<0>(F<mxnet::op::mshadow_op::square>(data));
-      MXNET_ASSIGN_REQ_SWITCH(req[0], Req, {
-        mxnet_op::Kernel<mxnet_op::op_with_req<mxnet::op::mshadow_op::plus, Req>, xpu>::Launch(
-          s, norm.size(0), norm.dptr_, norm.dptr_, DType(param_.eps));
-      });
-      norm = F<mxnet::op::mshadow_op::square_root>(norm);
-      out = data / broadcast<0>(norm, out.shape_);
+      out = data / mshadow::expr::broadcast<0>(norm, out.shape_);
 
 Review comment:
   If I use
   `mxnet::op::ReduceAxesComputeImpl`
   it will got an error:
   `src/operator/./l2_normalization-inl.h:91:7: error: ‘ReduceAxesComputeImpl’ is not a member of ‘mxnet::op’`
   I think it may be because the **L2_norm** compilation order is before **broadcast_reduce_op.h**
   So, I use
   `#include "./tensor/broadcast_reduce_op.h"`
   and this requires that I must use **mshadow::expr::** again, otherwise conflicts will occur.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services