You are viewing a plain text version of this content. The canonical link for it is here.
Posted to discuss-archive@tvm.apache.org by "professor-moody (via GitHub)" <gi...@apache.org> on 2026/07/13 17:46:26 UTC

[PR] [Runtime][VM] Bounds-check per-parameter byteOffset/nbytes in tensor-cache loader [tvm]

professor-moody opened a new pull request, #19997:
URL: https://github.com/apache/tvm/pull/19997

   ### Motivation
   
   `TensorCacheMetadata::FileRecord::ParamRecord::Load` copies each parameter out of the shard buffer using `byte_offset` and `nbytes` read from `tensor-cache.json`. `FileRecord::Load` validates only the shard total (`TVM_FFI_CHECK_EQ(this->nbytes, raw_data_buffer->length())`), not each parameter's range, so a cache directory whose per-parameter `byteOffset`/`nbytes` are inconsistent with the shard reads out of bounds of the shard buffer (both the `f32-to-bf16` `memcpy` path and the `CopyTensorFromBytes` path).
   
   Bounds-checking the file-controlled offsets before the copy is cheap, self-documenting defensive hardening: it turns a hard-to-debug out-of-bounds read into a clear, actionable error when a cache directory is truncated or inconsistent.
   
   ### Change
   
   Add an overflow-safe bounds check at the top of `ParamRecord::Load`, raising `ValueError` (matching the existing `TVM_FFI_CHECK_EQ` shard-total check in `FileRecord::Load`) with the offending range:
   
   - `byte_offset >= 0 && nbytes >= 0 && byte_offset <= shard_len && nbytes <= shard_len - byte_offset`
   
   The condition is written as `nbytes <= shard_len - byte_offset` (rather than `byte_offset + nbytes <= shard_len`) to avoid signed overflow.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org


Re: [PR] [Runtime][VM] Bounds-check per-parameter byteOffset/nbytes in tensor-cache loader [tvm]

Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen closed pull request #19997: [Runtime][VM] Bounds-check per-parameter byteOffset/nbytes in tensor-cache loader
URL: https://github.com/apache/tvm/pull/19997


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org


Re: [PR] [Runtime][VM] Bounds-check per-parameter byteOffset/nbytes in tensor-cache loader [tvm]

Posted by "professor-moody (via GitHub)" <gi...@apache.org>.
professor-moody commented on PR #19997:
URL: https://github.com/apache/tvm/pull/19997#issuecomment-4961601624

   Thanks for the review. Addressed both points: added the null check on the shard buffer, and an `nbytes % 2 == 0` guard on the `f32-to-bf16` path — the latter also prevents the odd-`nbytes` buffer overwrite you flagged, since `buffer(nbytes / 2)` would otherwise be one byte short of the `memcpy`. Both fold into the same tensor-cache bounds hardening.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org


Re: [PR] [Runtime][VM] Bounds-check per-parameter byteOffset/nbytes in tensor-cache loader [tvm]

Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #19997:
URL: https://github.com/apache/tvm/pull/19997#discussion_r3572902702


##########
src/runtime/vm/tensor_cache_support.cc:
##########
@@ -153,6 +153,16 @@ void CopyTensorFromBytes(Tensor param, const void* data, size_t nbytes,
 
 Tensor TensorCacheMetadata::FileRecord::ParamRecord::Load(
     Device device, const std::string* raw_data, ffi::Optional<Tensor>* staging_buffer) const {
+  // `byte_offset` and `nbytes` come from tensor-cache.json; FileRecord::Load only validates the shard
+  // total, not each parameter's range. Bounds-check this parameter against the shard buffer before the
+  // copy so a malformed/inconsistent cache fails with a clear error instead of reading out of bounds.
+  // (Written to avoid signed overflow on `byte_offset + nbytes`.)
+  int64_t shard_len = static_cast<int64_t>(raw_data->length());
+  TVM_FFI_CHECK(byte_offset >= 0 && nbytes >= 0 && byte_offset <= shard_len &&
+                    nbytes <= shard_len - byte_offset,
+                ValueError)
+      << "Parameter byte range [" << byte_offset << ", " << (byte_offset + nbytes)
+      << ") is out of bounds of the " << shard_len << "-byte shard buffer.";

Review Comment:
   ![security-high](https://www.gstatic.com/codereviewagent/security-high-priority.svg) ![high](https://www.gstatic.com/codereviewagent/high-priority.svg)
   
   There are two potential issues here:
   1. **Null Pointer Dereference**: `raw_data` is a pointer and is not checked for `nullptr` before calling `raw_data->length()`. If a caller passes `nullptr`, this will result in a crash.
   2. **Buffer Overflow**: In the `f32-to-bf16` format path below, `std::vector<uint16_t> buffer(nbytes / 2)` is allocated, and `std::memcpy(buffer.data(), raw_data->data() + byte_offset, nbytes)` is called. If `nbytes` is odd, `nbytes / 2` will allocate a buffer of size `nbytes - 1` bytes, leading to an out-of-bounds write (buffer overflow) during `std::memcpy`.
   
   We should add a null check for `raw_data` and ensure `nbytes` is even when using the `f32-to-bf16` format.
   
   ```c
     TVM_FFI_CHECK(raw_data != nullptr, ValueError) << "raw_data pointer cannot be nullptr.";
     int64_t shard_len = static_cast<int64_t>(raw_data->length());
     TVM_FFI_CHECK(byte_offset >= 0 && nbytes >= 0 && byte_offset <= shard_len &&
                       nbytes <= shard_len - byte_offset,
                   ValueError)
         << "Parameter byte range [" << byte_offset << ", " << (byte_offset + nbytes)
         << ") is out of bounds of the " << shard_len << "-byte shard buffer.";
     if (dtype == DLDataType{kDLFloat, 32, 1} && format == "f32-to-bf16") {
       TVM_FFI_CHECK(nbytes % 2 == 0, ValueError)
           << "Parameter nbytes must be even for f32-to-bf16 format, but got " << nbytes;
     }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org