Re: [I] [Bug] Tensorization breaks when TIR one dimension is a unit iterator [tvm]

["patschmidt2 (via GitHub)" <gi...@apache.org>]

patschmidt2 commented on issue #16566:
URL: https://github.com/apache/tvm/issues/16566#issuecomment-2066754902

   @LeiWang1999 I tried using your commit but I still receive an error:
   ```
   Error message: The stmt tir.For#0 doesn't match the tensor intrin
   The pattern attempting to be matched:
   for y_o in range(T.int64(32)):
       k_o = T.int64()
       with T.block("res_update"):
           v_y_o_i = T.axis.spatial(T.int64(32), y_o)
           v_k_o_i = T.axis.reduce(T.int64(64), k_o)
           res = T.Buffer((T.int64(1), T.int64(32)), "int8")
           v_x_o_o = T.int64()
           p0 = T.Buffer((T.int64(1), T.int64(64)), "int8")
           fused_constant = T.Buffer((64, 32), "int8")
           fused_constant_1 = T.Buffer((32,), "int32")
           T.reads(res[v_x_o_o, v_y_o_i], p0[v_x_o_o, v_k_o_i], fused_constant[v_k_o_i, v_y_o_i], fused_constant_1[v_y_o_i])
           T.writes(res[v_x_o_o, v_y_o_i])
           T.block_attr({"scale": T.float32(0.0007562367245554924)})
           res[v_x_o_o, v_y_o_i] = res[v_x_o_o, v_y_o_i] + (p0[v_x_o_o, v_k_o_i] * fused_constant[v_k_o_i, v_y_o_i] + T.Cast("int8", fused_constant_1[v_y_o_i]))
   Does not match the tensorize description:
   for j in range(T.int64(32)):
       k = T.int64()
       with T.block(""):
           vii = T.axis.spatial(T.int64(1), T.int64(0))
           vjj = T.axis.spatial(T.int64(32), j)
           vkk = T.axis.reduce(T.int64(64), k)
           C = T.Buffer((1, T.int64(32)), "int8", offset_factor=1)
           A = T.Buffer((1, T.int64(64)), "int8", offset_factor=1)
           B = T.Buffer((T.int64(64), T.int64(32)), "int8", offset_factor=1)
           Bias = T.Buffer((T.int64(32),), "int32", offset_factor=1)
           T.reads(C[T.int64(0), vjj], A[T.int64(0), vkk], B[vkk, vjj], Bias[vjj])
           T.writes(C[T.int64(0), vjj])
           C[T.int64(0), vjj] = C[T.int64(0), vjj] + (A[T.int64(0), vkk] * B[vkk, vjj] + T.Cast("int8", Bias[vjj]))
   CompareArray array size mismatch. lhs.size()=2 vs rhs.size()=3
   BlockRealizeNode iter_values do not match: op->iter_values=[y_o, k_o] vs rhs->iter_values=[T.int64(0), j, k]
   ```
   
   So that is still the original error. I have found a way to change the `DeriveBlockBinding` function in order to keep all iter_values but that leads to this new error: 
   
   ```
   Error message: The stmt tir.For#0 doesn't match the tensor intrin
   The pattern attempting to be matched:
   for y_o in range(T.int64(32)):
       x_o = T.int64()
       k_o = T.int64()
       with T.block("res_update"):
           v_x_o_i = T.axis.spatial(T.int64(1), x_o)
           v_y_o_i = T.axis.spatial(T.int64(32), y_o)
           v_k_o_i = T.axis.reduce(T.int64(64), k_o)
           res = T.Buffer((T.int64(1), T.int64(32)), "int8")
           p0 = T.Buffer((T.int64(1), T.int64(64)), "int8")
           fused_constant = T.Buffer((64, 32), "int8")
           fused_constant_1 = T.Buffer((32,), "int32")
           T.reads(res[T.int64(0), v_y_o_i], p0[T.int64(0), v_k_o_i], fused_constant[v_k_o_i, v_y_o_i], fused_constant_1[v_y_o_i])
           T.writes(res[T.int64(0), v_y_o_i])
           T.block_attr({"scale": T.float32(0.00083669449668377638)})
           res[T.int64(0), v_y_o_i] = res[T.int64(0), v_y_o_i] + (p0[T.int64(0), v_k_o_i] * fused_constant[v_k_o_i, v_y_o_i] + T.Cast("int8", fused_constant_1[v_y_o_i]))
   Does not match the tensorize description:
   for j in range(T.int64(32)):
       k = T.int64()
       with T.block(""):
           vii = T.axis.spatial(T.int64(1), T.int64(0))
           vjj = T.axis.spatial(T.int64(32), j)
           vkk = T.axis.reduce(T.int64(64), k)
           C = T.Buffer((1, T.int64(32)), "int8", offset_factor=1)
           A = T.Buffer((1, T.int64(64)), "int8", offset_factor=1)
           B = T.Buffer((T.int64(64), T.int64(32)), "int8", offset_factor=1)
           Bias = T.Buffer((T.int64(32),), "int32", offset_factor=1)
           T.reads(C[T.int64(0), vjj], A[T.int64(0), vkk], B[vkk, vjj], Bias[vjj])
           T.writes(C[T.int64(0), vjj])
           C[T.int64(0), vjj] = C[T.int64(0), vjj] + (A[T.int64(0), vkk] * B[vkk, vjj] + T.Cast("int8", Bias[vjj]))
   Expression mismatch: x_o vs T.int64(0)
   BlockRealizeNode iter_values do not match: op->iter_values=[x_o, y_o, k_o] vs rhs->iter_values=[T.int64(0), j, k]
   ```
   
   The variable is preserved, but the Simplifier applied to the intrin description replaces the unit loop with a constant. Is there a similar simplifier that I could apply to the Block? Because that seems to be the only difference remaining here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org