[PR] Reuse probed lists in batch reader search [paimon-vector-index]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi opened a new pull request, #14:
URL: https://github.com/apache/paimon-vector-index/pull/14

   ## Summary
   
   Tighten the batch reader search path by grouping only actually probed lists and reusing the shared reader list scan helper. This avoids allocating and scanning `nlist` empty buckets when `nprobe * nq` is much smaller than `nlist`, while keeping list reads shared across queries.
   
   ## Changes
   
   - Replace the `Vec<Vec<_>>` sized by `nlist` with a map plus an ordered `unique_lists` list for probed lists only.
   - Iterate only probed unique lists instead of scanning every IVF list.
   - Reuse `scan_reader_list` from the single-query reader path so transposed/4-bit/precomputed scan behavior stays aligned.
   - Add a batch-vs-single reader search consistency test.
   
   ## Testing
   
   - `cargo fmt --all -- --check`
   - `cargo test -p paimon-vindex-core batch_reader -- --nocapture`
   - `cargo check -p paimon-vindex-core`
   - `cargo test -p paimon-vindex-core`
   - `cargo clippy --all-targets --workspace -- -D warnings`
   - `cargo build --all-targets`
   - `cargo test --workspace`
   - `git diff --check`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Reuse probed lists in batch reader search [paimon-vector-index]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #14:
URL: https://github.com/apache/paimon-vector-index/pull/14


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org