[PR] [rust] Coalesce scalar BLOB range reads [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi opened a new pull request, #880:
URL: https://github.com/apache/paimon-rust/pull/880

   ### Purpose
   
   Reduce scalar inline BLOB read amplification. The reader previously issued one FileRead request per selected BLOB entry, so a 16,000-row scan performed roughly 16,000 small range reads even when entries were adjacent.
   
   ### Changes
   
   - Coalesce nearby scalar BLOB entry ranges with the same defaults as the Python reader: 1 MiB maximum gap and 8 MiB maximum span.
   - Fetch merged spans with the configured blob parallelism, then restore the requested row order.
   - Preserve per-entry magic, embedded-length, and CRC32 validation after slicing each merged response.
   - Strengthen the data-evolution fallback test to assert exact payload ranges instead of request counts.
   
   ### Performance
   
   Release Python binding, local filesystem, 16,000 rows, 4 KiB payloads, 8 splits, 5 measured iterations:
   
   - native parallelism=1, blob_parallelism=1: 152.1 ms -> 17.5 ms
   - native parallelism=4, blob_parallelism=1: 136.8 ms -> 5.8 ms
   - native parallelism=4, blob_parallelism=4: 134.6 ms -> 6.0 ms
   
   For 512 rows with 128 KiB payloads in one split, increasing blob_parallelism from 1 to 4 reduced the native median from 16.5 ms to 13.8 ms, confirming that separate 8 MiB spans still read concurrently.
   
   All benchmark results matched Python row counts, schemas, and checksums.
   
   ### Verification
   
   - cargo fmt --all -- --check
   - cargo clippy --locked --all-targets --workspace --features fulltext,vortex -- -D warnings
   - cargo test --locked -p paimon --all-targets --features fulltext,vortex
     - 3009 unit tests passed, 2 ignored
     - all integration and example targets passed
   - complete BLOB module tests: 38 passed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [rust] Coalesce inline BLOB reads with bounded amplification [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #880:
URL: https://github.com/apache/paimon-rust/pull/880


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org