[PR] feat(vector-search): split with_filter into partition + data parts for the residual [paimon-rust]

["JunRuiLee (via GitHub)" <gi...@apache.org>]

JunRuiLee opened a new pull request, #609:
URL: https://github.com/apache/paimon-rust/pull/609

   ### Purpose
   
   Primary-key vector search reused the whole `with_filter` predicate as the
   per-row residual applied after ANN recall — even though the same predicate is
   already pushed into scan planning (`PkVectorScan`), where partition-only
   conjuncts prune partitions/files. Re-checking partition conjuncts per row is
   redundant, and re-reading partition columns for the residual is wasted I/O.
   
   ### Change
   
   Derive the residual from the DATA (non-partition) conjuncts only, via
   `split_partition_and_data_predicates`:
   
   - A partition-only filter (e.g. `dt = 'a'`) now needs no residual at all — the
     residual read is skipped entirely (planning already enforces the partition).
   - `AND(partition, data)` residual-checks only the data conjunct (one fewer
     column re-read per candidate file).
   - A mixed conjunct (e.g. `dt = 'a' OR id > 5`) is not partition-only, so it
     stays whole in the residual and evaluates against the materialized partition
     column (primary-key data files carry partition columns) — results unchanged.
   
   Planning still receives the whole filter, so partition pruning is unaffected.
   Mirrors Java `BatchVectorSearchBuilderImpl.withFilter`. Covers the whole-snapshot
   `execute_read` path (single-query and batch converge on the shared core).
   
   Output is unchanged; this reduces residual I/O and per-row work for
   partition-referencing filters and keeps the residual's inputs to data-only
   predicates (so it never depends on partition columns being materialized).
   
   ### Tests
   
   `residual_uses_only_data_conjuncts_of_the_filter`: partition-only → no residual;
   data-only → kept; `AND(partition, data)` → only the data conjunct; mixed `OR`
   → stays whole.
   
   `cargo test -p paimon`, `cargo fmt --all --check`, and
   `cargo clippy -p paimon --all-targets -- -D warnings` pass.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(vector-search): use data conjuncts only for the PK-vector residual and pre-filter guard [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #609:
URL: https://github.com/apache/paimon-rust/pull/609


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org