[PR] [format] Decline ORC pushdown of empty IN/NOT IN instead of crashing the reader [paimon]

["zhuxiangyi (via GitHub)" <gi...@apache.org>]

zhuxiangyi opened a new pull request, #10015:
URL: https://github.com/apache/paimon/pull/10015

   ### Purpose
   
   Bug fix: any ORC read that carries an `IN ()` or `NOT IN ()` predicate fails while the reader is being created.
   
   `PredicateBuilder.in(idx, emptyList())` is a legitimate always-false leaf (the builder already special-cases an empty list, exactly as it does for more than 20 literals), and `notIn` inherits it through `negate()`. But `OrcPredicateFunctionVisitor.visitIn` turns that empty list into an empty `Object[]` and hands it to Hive's `SearchArgument.Builder.in(...)`, which rejects a zero-length call outright:
   
   ```
   java.lang.IllegalArgumentException: Can't create in expression with no arguments
   	at org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$BuilderImpl.in(SearchArgumentImpl.java:419)
   	at org.apache.paimon.format.orc.filter.OrcFilters$In.add(OrcFilters.java:328)
   	at org.apache.paimon.format.orc.OrcReaderFactory.createRecordReader(OrcReaderFactory.java:466)
   ```
   
   So a predicate that should simply match nothing (or everything, for `NOT IN ()`) instead takes the whole read down. Nothing nested or `Transform`-based is needed to hit it — a plain index-based `in`/`notIn` on an ORC table is enough.
   
   The fix has `visitIn` decline the pushdown (`Optional.empty()`) on an empty literal list, the same way the visitor already declines every other predicate it cannot express in a SARG. `visitNotIn` delegates to `visitIn`, so it is covered by the same branch. Correctness is unaffected: engines re-evaluate data filters after the scan (Spark keeps every data filter as a post-scan predicate; Paimon's own row-level filtering is opt-in via `TableRead.executeFilter()`), so the always-false / always-true semantics still hold — only the stripe-level pruning for that one predicate is given up.
   
   This is the ORC counterpart of the `ParquetFilters.visitIn`/`visitNotIn` guard added in #9423 (parquet-mr's `SetColumnFilterPredicate` refuses an empty set the same way). It is independent of that PR and predates it.
   
   ### Tests
   
   - `OrcFilterConverterTest.testInPredicateWithEmptyValuesIsNotPushedDown` — `in(idx, [])` and `notIn(idx, [])` both resolve to `Optional.empty()`; also pins that an empty `In` leaf inside `or(...)`/`and(...)` declines the whole compound rather than crashing (the builder only folds `AlwaysFalse.INSTANCE`, so the leaf does survive into a compound).
   - `OrcFormatReadWriteTest.testEmptyInAndNotInPredicatesDoNotCrashTheReader` — writes a real ORC file and opens a reader with each predicate through `OrcFileFormat.createReaderFactory`; reproduces the `IllegalArgumentException` before the fix, returns every row unfiltered after it.
   
   ### API and Format
   
   No changes.
   
   ### Documentation
   
   No changes.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [format] Decline ORC pushdown of empty IN/NOT IN instead of crashing the reader [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #10015:
URL: https://github.com/apache/paimon/pull/10015


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org