[PR] [Fix][Relax] Fix top-p/top-k sampling kernel [tvm]

["MasterJH5574 (via GitHub)" <gi...@apache.org>]

MasterJH5574 opened a new pull request, #16703:
URL: https://github.com/apache/tvm/pull/16703

   This PR fixes a typo in the samping kernel of top-p/top-k sampling op. Prior to this PR, the kernel has out-of-bound global memory access due to a miss when introducing `sample_indices` in #16675.
   
   The correctness pass did not reveal this issue by directly running the test or running through pytest. But actually, if we use compute-sanitizer from NVIDIA, it will report the illegal memory access:
   ```
   > compute-sanitizer --tool memcheck --print-limit=5 --launch-timeout 3600 python tests/python/relax/test_frontend_nn_op.py
   ========= COMPUTE-SANITIZER
   ========= Invalid __global__ read of size 8 bytes
   =========     at 0x4e90 in get_index_from_sorted_kernel
   =========     by thread (7,0,0) in block (0,0,0)
   =========     Address 0x7fe35ac00238 is out of bounds
   =========     and is 9 bytes after the nearest allocation at 0x7fe35ac00200 of size 48 bytes
   =========     Saved host backtrace up to driver entry point at kernel launch time
   ...
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Fix][Relax] Fix top-p/top-k sampling kernel [tvm]

["tqchen (via GitHub)" <gi...@apache.org>]

tqchen merged PR #16703:
URL: https://github.com/apache/tvm/pull/16703


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org