Re: [I] Plan error when partition column is timestamp type and SQL query contains filter condition which contains partition [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on issue #15786:
URL: https://github.com/apache/hudi/issues/15786#issuecomment-4795475140

   This appears to be **fixed** on current master. The reported stack trace originates in Flink's `PushPartitionIntoTableSourceScanRule`, which only fires when the table source implements `SupportsPartitionPushDown` or the catalog returns partitions via `listPartitionsByFilter`.
   
   In current master:
   - `HoodieTableSource` does not implement `SupportsPartitionPushDown` (no references anywhere in the repo).
   - `HoodieCatalog.listPartitionsByFilter` and `HoodieHiveCatalog.listPartitionsByFilter` both return `Collections.emptyList()`.
   
   The relevant change is commit [`84a14a4`](https://github.com/apache/hudi/commit/84a14a4380f98302ab40101b65b749b2a787e6ac) ([HUDI-5880] Support partition pruning for flink streaming source in runtime, #8102), which moved partition pruning into Hudi's internal `PartitionPruners` and routed predicates through `SupportsFilterPushDown` instead. As a result, Flink's `PushPartitionIntoTableSourceScanRule.PartitionPruner.convertPartitionFieldValue` is no longer invoked for Hudi sources.
   
   **Caveat / follow-up:** Hudi's own `DynamicPartitionPruner` calls `DataTypeUtils.resolvePartition`, which uses `LocalDateTime.parse(partition)` for `TIMESTAMP_WITHOUT_TIME_ZONE`. This will still fail for non-ISO partition path formats (e.g., `yyyyMMddHH` produced by `TimestampBasedKeyGenerator`). If you can still reproduce a partition-pruning failure on a recent build using a custom timestamp partition format, please share the new stack trace and a minimal repro — that would be a distinct bug worth tracking separately. Otherwise, this issue can probably be closed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org