[PR] [Web] Compatibility with PagedKVCache in WebGPU [tvm]

["CharlieFRuan (via GitHub)" <gi...@apache.org>]

CharlieFRuan opened a new pull request, #16554:
URL: https://github.com/apache/tvm/pull/16554

   This PR introduces various WebGPU changes to accommodate the new `PagedKVCache` interface. All changes below are essential for making models that use PagedKVCache runnable under WebGPU:
   
   - Require exactly same-dtype matching for WebGPU smem reuse in `storage_rewrite.cc`
   - Rename `AttentionKVCache` to `AttentionKVCacheLegacy` for the old KVcache interface in `lm_support.cc`; include `paged_kv_cache.cc` when making `wasm_runtime` subsequently
   - In WebGPU codegen:
     - Declare local variables within the function scope rather than the module scope
     - Generate `while (true)` rather than `while (1)`
   - Require 10 `maxStorageBuffersPerShaderStage` rather than the default 8 from the WebGPU device when initializing runtime; this is required for new kernels introduced in PagedKVCache
   - In `deviceCopyToCPU()`, when raw bytes to write are not multiples of 4, we pad them, as required by WebGPU's `writeBuffer()`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Web] Compatibility with PagedKVCache in WebGPU [tvm]

["MasterJH5574 (via GitHub)" <gi...@apache.org>]

MasterJH5574 merged PR #16554:
URL: https://github.com/apache/tvm/pull/16554


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org