[jira] [Assigned] (FLINK-39421) Metadata filter push-down for table sources

["Jim Hughes (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-39421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jim Hughes reassigned FLINK-39421:
----------------------------------

    Assignee: Jim Hughes

> Metadata filter push-down for table sources
> -------------------------------------------
>
>                 Key: FLINK-39421
>                 URL: https://issues.apache.org/jira/browse/FLINK-39421
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / Planner
>            Reporter: Jim Hughes
>            Assignee: Jim Hughes
>            Priority: Major
>              Labels: pull-request-available
>
> Predicates on metadata columns (e.g., Kafka offset, timestamp, partition) are excluded from the existing SupportsFilterPushDown mechanism. FilterPushDownSpec serializes RexInputRef indices relative to the scan's produced row type, but metadata columns are appended after physical columns. When ProjectPushDownSpec narrows the row type during compiled plan restoration, metadata indices become out-of-bounds, causing ArrayIndexOutOfBoundsException.
> This ticket introduces a dedicated metadata filter push-down path:
> - New supportsMetadataFilterPushDown() and applyMetadataFilters() default methods on SupportsReadingMetadata (opt-in; existing sources unaffected)
> - New MetadataFilterPushDownSpec that snapshots the row type at optimization time, solving the index mismatch
> - Extension to PushFilterIntoTableSourceScanRule to classify predicates as physical, metadata, or mixed
> See FLIP-574 (https://cwiki.apache.org/confluence/display/FLINK/FLIP-574+Metadata+Filter+Push-Down+for+Table+Sources) for full design.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)