[jira] [Commented] (SPARK-53798) Enable operator pushdown in Data Source V2 streaming

["Dongjoon Hyun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-53798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18058526#comment-18058526 ] 

Dongjoon Hyun commented on SPARK-53798:
---------------------------------------

I'm just curious we are starting this for 4.2.0 or postpone further, [~Gengliang.Wang] . Feel free to decide.

> Enable operator pushdown in Data Source V2 streaming
> ----------------------------------------------------
>
>                 Key: SPARK-53798
>                 URL: https://issues.apache.org/jira/browse/SPARK-53798
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 4.2.0
>            Reporter: Gengliang Wang
>            Priority: Major
>
> This umbrella tracks enabling operator pushdowns in DataSource V2 streaming, bringing it closer to batch parity. Supported pushdowns (filters, projections, aggregates, etc) will be applied directly in DSv2 readers during stream analysis to reduce scanned data and compute cost.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org