[GitHub] [incubator-tvm] ajtulloch commented on a change in pull request #4277: [ARM][Topi] Improving Int8 Perf in Spatial Conv2D schedule.

[GitBox <gi...@apache.org>]

ajtulloch commented on a change in pull request #4277: [ARM][Topi] Improving Int8 Perf in Spatial Conv2D schedule.
URL: https://github.com/apache/incubator-tvm/pull/4277#discussion_r343949837
 
 

 ##########
 File path: topi/python/topi/arm_cpu/conv2d_spatial_pack.py
 ##########
 @@ -93,24 +93,35 @@ def conv2d_spatial_pack_nchw(cfg, data, kernel, strides, padding, dilation,
     ovshape = (N, CO // VC, OH // VH, OW // VW, VH, VW, VC)
     oshape = (N, CO, OH, OW)
 
+    # For Integer convs, upcasting to int16 leads to faster implementation
+    # because LLVM is able to better interleave vmlal.s16 and vldr instructions,
+    # leading to higher CPU utilization.
+    adjusted_dtype = data.dtype
+    if 'int8' in data.dtype and 'int8' in kernel.dtype and out_dtype == 'int32':
+        adjusted_dtype = 'int16'
+
     if dilation_h != 1 or dilation_w != 1:
         # undilate input data
         dvshape = (N, OH // VH, OW // VW, CI, KH, KW, VH, VW)
         data_vec = tvm.compute(dvshape, lambda n, h, w, ci, kh, kw, vh, vw:
                                data_pad[n][ci][(h*VH+vh)*HSTR+kh*dilation_h]
-                               [(w*VW+vw)*WSTR+kw*dilation_w],
+                               [(w*VW+vw)*WSTR+kw*dilation_w].astype(adjusted_dtype),
                                name='data_vec_undilated')
     else:
         dvshape = (N, OH // VH, OW // VW, CI, VH*HSTR + KH-1, VW*WSTR + KW-1)
         data_vec = tvm.compute(dvshape, lambda n, h, w, ci, vh, vw:
-                               data_pad[n][ci][h*VH*HSTR+vh][w*VW*WSTR+vw],
+                               data_pad[n][ci][h*VH*HSTR+vh][w*VW*WSTR+vw].astype(adjusted_dtype),
                                name='data_vec')
 
     if pre_packed:
         kernel_vec = kernel
+        if adjusted_dtype != kernel.dtype:
+            kernel_vec = tvm.compute(kvshape, lambda co, ci, kh, kw, vc:
 
 Review comment:
   is this scheduled correctly? presumably we want this `kernel_vec` computation inlined in this case?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services