[GitHub] [tvm] comaniac commented on a change in pull request #8457: [CUDA] Improve injective schedule to enable half2

[GitBox <gi...@apache.org>]

comaniac commented on a change in pull request #8457:
URL: https://github.com/apache/tvm/pull/8457#discussion_r668341809



##########
File path: python/tvm/topi/cuda/injective.py
##########
@@ -54,6 +64,26 @@ def schedule_injective_from_existing(sch, out):
 
     try:
         const_size = utils.get_const_int(out_len)
+
+        # Adjust block and thread to make sure they are dividable so that vectorize can be
+        # correctly applied.
+        if vector_width > 1 and const_size % vector_width == 0:
+            remain_total_size = const_size // vector_width
+            cand_sizes = []
+            for max_size in [num_thread, max_block]:
+                cand_sizes.append(
+                    max_size
+                    if remain_total_size % max_size == 0
+                    else find_nearest_small_factor(remain_total_size, max_size)
+                )
+                remain_total_size //= cand_sizes[-1]
+
+            # If the product of candidate dividable (block * thread) is too small,
+            # then the performance may be worse even half2 is enabled. Note that 0.7
+            # is just a heuristic ratio and may not be optimal for all workloads.
+            if np.prod(cand_sizes) / (max_block * num_thread) >= 0.7:
+                max_block, num_thread = cand_sizes

Review comment:
       Good catch. Sorry for the mistake.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org