[I] Fix decimal scale in Parquet dictionary pruning [auron]

["Sigma-Ma (via GitHub)" <gi...@apache.org>]

Sigma-Ma opened a new issue, #2591:
URL: https://github.com/apache/auron/issues/2591

   **Describe the bug**
   
   When a decimal equality predicate reaches native Parquet pruning, the dictionary pruning extension can incorrectly discard a row group containing matching rows.
   
   For INT32- and INT64-backed decimals, dictionary extraction returns raw integer arrays without restoring the decimal type. Building the pruning statistics then uses a regular integer-to-decimal cast, which applies the scale again.
   
   For example, the unscaled value 100 represents 1.00 in DECIMAL(18,2), but becomes 100.00 during that cast. The dictionary membership check for 1.00 returns false, causing the matching row group to be skipped without a predicate evaluation error.
   
   **To Reproduce**
   
   1. Use parquet-mr 1.13.1 to write a file with:
      - Schema: `required int64 d (DECIMAL(18,2))`
      - Writer version: `PARQUET_1_0`
      - Dictionary encoding enabled
      - One row group containing unscaled values `[-100, 0, 100, 200]`, repeated 250 times.
   
   2. Confirm that the footer reports `PLAIN_DICTIONARY` for both dictionary and data pages, with column encodings `[PLAIN_DICTIONARY, BIT_PACKED]`.
   
   3. Read the file normally and confirm that `d = 1.00` has 250 matching rows.
   
   4. Supply the corresponding typed decimal equality predicate to `RowGroupAccessPlanFilter::prune_by_statistics`: `ScalarValue::Decimal128(Some(100), 18, 2)`.
   
   The resulting access plan contains no row groups, although row group 0 contains the matching rows. `predicate_evaluation_errors` remains zero.
   
   The same failure reproduces with INT32-backed DECIMAL(9,2) and with `d = -1.00`. Zero is preserved. Plain-encoded and PARQUET_2_0 dictionary files serve as controls: they do not enter this dictionary extraction path and retain the matching group.
   
   **Expected behavior**
   
   Dictionary pruning must preserve decimal precision and scale and must not remove a row group containing matching rows. If decimal dictionary values cannot be interpreted safely, skip dictionary pruning for that column.
   
   **Additional context**
   
   This is a follow-up to #1032 and the workaround in #1033.
   
   Reproduced at native component level using parquet-mr 1.13.1 and Auron-patched DataFusion 49.0.0 / Arrow-Parquet 55.2.0 components.
   
   The existing planner guards currently skip decimal pruning predicates. Keep those guards in place while fixing this native path; enabling Iceberg decimal pruning is a separate change.
   
   Relevant patches:
   - https://github.com/apache/auron/blob/7dfc2412143f09c564ecaa9d659cb6276918c7fa/dev/vendors/patches/arrow-rs/0002-auron-add-get_dictionary.patch
   - https://github.com/apache/auron/blob/7dfc2412143f09c564ecaa9d659cb6276918c7fa/dev/vendors/patches/datafusion/0005-auron-supports-parquet-dictionary-filtering.patch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org