[PR] [Unity][Support] Sample from top-p supports offset [tvm]

["MasterJH5574 (via GitHub)" <gi...@apache.org>]

MasterJH5574 opened a new pull request, #16069:
URL: https://github.com/apache/tvm/pull/16069

   This PR enhances the `SampleTopPFromProb` function, adding a new `unit_offset` parameter. With this PR, the input prob array can contain multiple prob distributions. The offset is used to specify which probability to sample from.
   
   Prior to this PR, this function assumes the input prob array only contains a single probability distribution.
   This assumption, though effective, prevents efficient sampling for batching scenarios, since it forces us to split the batched prob array before sampling.
   
   This PR eliminates this issue by introducing the offset.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Unity][Support] Sample from top-p supports offset [tvm]

["junrushao (via GitHub)" <gi...@apache.org>]

junrushao merged PR #16069:
URL: https://github.com/apache/tvm/pull/16069


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org