You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "CharlieFRuan (via GitHub)" <gi...@apache.org> on 2024/04/21 04:46:04 UTC
[PR] [Web] Support string[] in setPackedFunc() and exceptionally long arrays [tvm]
CharlieFRuan opened a new pull request, #16910:
URL: https://github.com/apache/tvm/pull/16910
There are two changes in this PR.
#### Change 1: Support `string[]` in `setPackedFunc()`
Prior to this PR, we cannot pass in `string[]` from typescript to a TVM PackedFunc and need to convert it to `TVMArray<TVMString>` (for instance in `getParamsFromCacheByName()`). This may not be the most convenient if the PackedFunc's caller is not internal to tvmjs. Thus, this PR moves the conversion to `setPackedFunc()` instead.
#### Change 2: Support exceptionally long TVM arrays
The second change is dealing with exceptionally long TVM arrays. In cases like passing in a token table, we need to pass in a long `string[]` (in Llama-3's case, of size 128000), leading to JS error `RangeError: Maximum call stack size exceeded` since we treat each string element as an argument, shown in `this.ctx.arrayMake(...inputs)`.
This PR sets an empirical call stack limit of 30000 and chunks the array elements in `makeTVMArray()`, converting each chunk to its own TVMArray. Then we concatenate them with the newly implemented `runtime.ArrayConcat` that concatenates N TVMArrays.
Tested end-to-end in WebLLM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Web] Support string[] in setPackedFunc() and exceptionally long arrays [tvm]
Posted by "MasterJH5574 (via GitHub)" <gi...@apache.org>.
MasterJH5574 merged PR #16910:
URL: https://github.com/apache/tvm/pull/16910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Web] Support string[] in setPackedFunc() and exceptionally long arrays [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on code in PR #16910:
URL: https://github.com/apache/tvm/pull/16910#discussion_r1573734081
##########
src/runtime/container.cc:
##########
@@ -49,6 +49,23 @@ TVM_REGISTER_GLOBAL("runtime.Array").set_body([](TVMArgs args, TVMRetValue* ret)
*ret = Array<ObjectRef>(data);
});
+// Concatenate n TVMArrays
+TVM_REGISTER_GLOBAL("runtime.ArrayConcat").set_body([](TVMArgs args, TVMRetValue* ret) {
Review Comment:
given this is not as a commonly used function, let us move it to https://github.com/apache/tvm/blob/main/web/emcc/wasm_runtime.cc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Web] Support string[] in setPackedFunc() and exceptionally long arrays [tvm]
Posted by "CharlieFRuan (via GitHub)" <gi...@apache.org>.
CharlieFRuan commented on code in PR #16910:
URL: https://github.com/apache/tvm/pull/16910#discussion_r1573844188
##########
src/runtime/container.cc:
##########
@@ -49,6 +49,23 @@ TVM_REGISTER_GLOBAL("runtime.Array").set_body([](TVMArgs args, TVMRetValue* ret)
*ret = Array<ObjectRef>(data);
});
+// Concatenate n TVMArrays
+TVM_REGISTER_GLOBAL("runtime.ArrayConcat").set_body([](TVMArgs args, TVMRetValue* ret) {
Review Comment:
Makes sense; just moved and renamed to `tvmjs.runtime.ArrayConcat`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org