[PR] fix(table): keep _ROW_ID predicates out of partition-key projection [paimon-rust]

["plusplusjiajia (via GitHub)" <gi...@apache.org>]

plusplusjiajia opened a new pull request, #642:
URL: https://github.com/apache/paimon-rust/pull/642

   ### Purpose
   
     _ROW_ID is not a table column, so callers hand-build the leaf with a placeholder index (0, as test_with_filter_extracts_row_id_ranges does). ReadBuilder::with_filter ran normalize_filter — whose
     partition/bucket split matches by index — before that leaf was removed, so on a table whose first field is a partition key, WHERE _ROW_ID >= 10 became a predicate on that key and pruned partitions by comparing
     them against row ids, silently dropping rows.
   
     Java strips row-id conjuncts by name in DataEvolutionBatchScan.rowIdSafeResidualFilter; pypaimon rejects a field not in partition_keys before looking at an index. Rust was the only implementation matching by
     index with no name-level guard.
   
   ### Brief change log
   
     - strip_row_id_filter removes row-id conjuncts by name and feeds only normalize_filter.
     - Its complement retain_row_id_filter puts them back as a data predicate when extract_row_id_ranges has no range form — leaf_to_ranges covers Eq/Gt/GtEq/Lt/LtEq/In only, so BETWEEN, !=, NOT IN and mixed
     disjunctions must still filter rows.
     - Replaces remove_row_id_filter, which kept a whole Or mentioning _ROW_ID and left the leaf in place.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(predicate): resolve _ROW_ID by name, not by its placeholder leaf index [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi commented on PR #642:
URL: https://github.com/apache/paimon-rust/pull/642#issuecomment-5204839144

   `_ROW_ID = ...` predicates targeting non-Data Evolution tables are extracted and removed from the residual; they subsequently become silently ineffective, resulting in the return of all data. This behavior has been reproduced in local tests. A check for Data Evolution should be performed prior to extraction, or the predicates should be retained in the residual and rejected across the board.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(predicate): resolve _ROW_ID by name, not by its placeholder leaf index [paimon-rust]

["plusplusjiajia (via GitHub)" <gi...@apache.org>]

plusplusjiajia commented on PR #642:
URL: https://github.com/apache/paimon-rust/pull/642#issuecomment-5214806872

   > `_ROW_ID = ...` predicates targeting non-Data Evolution tables are extracted and removed from the residual; they subsequently become silently ineffective, resulting in the return of all data. This behavior has been reproduced in local tests. A check for Data Evolution should be performed prior to extraction, or the predicates should be retained in the residual and rejected across the board.
   
   @JingsongLi  Thanks for catching this — confirmed and fixed in the second commit. _ROW_ID >= 102 returned all four rows of a file at first_row_id = 100. Took your second option. One thing worth your check: I gated on row-tracking.enabled || data-evolution.enabled rather than data evolution alone, since what matters is whether data files record a first_row_id, which row tracking assigns.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(predicate): resolve _ROW_ID by name, not by its placeholder leaf index [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #642:
URL: https://github.com/apache/paimon-rust/pull/642


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org