You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Gengliang Wang <lt...@gmail.com> on 2026/02/25 22:44:34 UTC
[VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Hi Spark devs,
I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
Filtering.*
*Summary:*
The DataSource V2 (DSV2) framework does not currently provide full
data-skipping capabilities comparable to Spark-native sources, primarily
due to limitations in Catalyst expression evaluation.
This SPIP bridges that gap to achieve partition-skipping parity. To support
this, Spark will push new *PartitionPredicate* objects that encapsulate
Catalyst partition filter expressions and the evaluation logic, allowing
data sources to skip irrelevant partitions effectively.
*Relevant Links:*
-
- *SPIP Doc:*
https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
-
*Discuss Thread:*
https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
-
*JIRA:* SPARK-55596 <https://issues.apache.org/jira/browse/SPARK-55596>
-
*POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
*The vote will be open for at least 72 hours. *Please vote:
[ ] +1: Accept the proposal as an official SPIP
[ ] +0
[ ] -1: I don't think this is a good idea because ...
Thanks,
Gengliang Wang
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Liu Cao <tw...@gmail.com>.
+1 (non-binding) Thank you! Looking forward to this
On Wed, Feb 25, 2026 at 7:52 PM Tathagata Das <ta...@gmail.com>
wrote:
> +1 binding. very excited.
>
> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
>
>> +1 (non-binding) Appreciate it!
>>
>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
>> wrote:
>>
>>> +1 (non-binding)
>>>
>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com> wrote:
>>>
>>>> +1 non-binding
>>>>
>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Spark devs,
>>>>>>
>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>>>>>> Filtering.*
>>>>>>
>>>>>> *Summary:*
>>>>>>
>>>>>> The DataSource V2 (DSV2) framework does not currently provide full
>>>>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>>>>> due to limitations in Catalyst expression evaluation.
>>>>>>
>>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>>>>> support this, Spark will push new *PartitionPredicate* objects that
>>>>>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>>>>>> allowing data sources to skip irrelevant partitions effectively.
>>>>>>
>>>>>> *Relevant Links:*
>>>>>>
>>>>>> -
>>>>>>
>>>>>> - *SPIP Doc:*
>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>> -
>>>>>>
>>>>>> *Discuss Thread:*
>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>> -
>>>>>>
>>>>>> *JIRA:* SPARK-55596
>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>> -
>>>>>>
>>>>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>>
>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>
>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>
>>>>>> [ ] +0
>>>>>>
>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>
>>>>>> Thanks,
>>>>>> Gengliang Wang
>>>>>>
>>>>>>
>>
>> --
>> John Zhuge
>>
>
--
Liu Cao
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Gengliang Wang <lt...@gmail.com>.
Thank you all. I'm going to conclude this vote.
On Sat, Feb 28, 2026 at 4:10 AM roryqi <ro...@apache.org> wrote:
> +1 (non-binding)
>
> YE <ad...@gmail.com> 于2026年2月28日周六 19:32写道：
> >
> > +1 (non-binding)
> >
> > On Feb 28, 2026, at 1:02 PM, Chao Sun <su...@apache.org> wrote:
> >
> > +1
> >
> > On Fri, Feb 27, 2026 at 3:12 AM DB Tsai <db...@dbtsai.com> wrote:
> >>
> >> +1
> >>
> >> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
> >>
> >> On Feb 25, 2026, at 11:33 PM, Kousuke Saruta <sa...@apache.org>
> wrote:
> >>
> >> +1
> >>
> >>
> >> 2026年2月26日(木) 16:32 Mridul Muralidharan <mr...@gmail.com>:
> >>>
> >>>
> >>> +1
> >>>
> >>> Regards,
> >>> Mridul
> >>>
> >>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cl...@gmail.com>
> wrote:
> >>>>
> >>>> +1
> >>>>
> >>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
> >>>>>
> >>>>> +1
> >>>>>
> >>>>> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
> >>>>>>
> >>>>>> +1 binding. very excited.
> >>>>>>
> >>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org>
> wrote:
> >>>>>>>
> >>>>>>> +1 (non-binding) Appreciate it!
> >>>>>>>
> >>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <
> karuppayya1990@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>> +1 (non-binding)
> >>>>>>>>
> >>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com>
> wrote:
> >>>>>>>>>
> >>>>>>>>> +1 non-binding
> >>>>>>>>>
> >>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com>
> wrote:
> >>>>>>>>>>
> >>>>>>>>>> +1
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <
> ltnwgl@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi Spark devs,
> >>>>>>>>>>>
> >>>>>>>>>>> I'd like to call a vote on the SPIP: DSV2 Enhanced Partition
> Stats Filtering.
> >>>>>>>>>>>
> >>>>>>>>>>> Summary:
> >>>>>>>>>>>
> >>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide
> full data-skipping capabilities comparable to Spark-native sources,
> primarily due to limitations in Catalyst expression evaluation.
> >>>>>>>>>>>
> >>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping
> parity. To support this, Spark will push new PartitionPredicate objects
> that encapsulate Catalyst partition filter expressions and the evaluation
> logic, allowing data sources to skip irrelevant partitions effectively.
> >>>>>>>>>>>
> >>>>>>>>>>> Relevant Links:
> >>>>>>>>>>>
> >>>>>>>>>>> SPIP Doc:
> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
> >>>>>>>>>>>
> >>>>>>>>>>> Discuss Thread:
> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
> >>>>>>>>>>>
> >>>>>>>>>>> JIRA: SPARK-55596
> >>>>>>>>>>>
> >>>>>>>>>>> POC PR: PR 54459
> >>>>>>>>>>>
> >>>>>>>>>>> The vote will be open for at least 72 hours. Please vote:
> >>>>>>>>>>>
> >>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
> >>>>>>>>>>>
> >>>>>>>>>>> [ ] +0
> >>>>>>>>>>>
> >>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Gengliang Wang
> >>>>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> John Zhuge
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by roryqi <ro...@apache.org>.
+1 (non-binding)
YE <ad...@gmail.com> 于2026年2月28日周六 19:32写道：
>
> +1 (non-binding)
>
> On Feb 28, 2026, at 1:02 PM, Chao Sun <su...@apache.org> wrote:
>
> +1
>
> On Fri, Feb 27, 2026 at 3:12 AM DB Tsai <db...@dbtsai.com> wrote:
>>
>> +1
>>
>> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
>>
>> On Feb 25, 2026, at 11:33 PM, Kousuke Saruta <sa...@apache.org> wrote:
>>
>> +1
>>
>>
>> 2026年2月26日(木) 16:32 Mridul Muralidharan <mr...@gmail.com>:
>>>
>>>
>>> +1
>>>
>>> Regards,
>>> Mridul
>>>
>>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cl...@gmail.com> wrote:
>>>>
>>>> +1
>>>>
>>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
>>>>>
>>>>> +1
>>>>>
>>>>> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
>>>>>>
>>>>>> +1 binding. very excited.
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
>>>>>>>
>>>>>>> +1 (non-binding) Appreciate it!
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> +1 (non-binding)
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> +1 non-binding
>>>>>>>>>
>>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> +1
>>>>>>>>>>
>>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Spark devs,
>>>>>>>>>>>
>>>>>>>>>>> I'd like to call a vote on the SPIP: DSV2 Enhanced Partition Stats Filtering.
>>>>>>>>>>>
>>>>>>>>>>> Summary:
>>>>>>>>>>>
>>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide full data-skipping capabilities comparable to Spark-native sources, primarily due to limitations in Catalyst expression evaluation.
>>>>>>>>>>>
>>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To support this, Spark will push new PartitionPredicate objects that encapsulate Catalyst partition filter expressions and the evaluation logic, allowing data sources to skip irrelevant partitions effectively.
>>>>>>>>>>>
>>>>>>>>>>> Relevant Links:
>>>>>>>>>>>
>>>>>>>>>>> SPIP Doc: https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>>>>
>>>>>>>>>>> Discuss Thread: https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>>>>
>>>>>>>>>>> JIRA: SPARK-55596
>>>>>>>>>>>
>>>>>>>>>>> POC PR: PR 54459
>>>>>>>>>>>
>>>>>>>>>>> The vote will be open for at least 72 hours. Please vote:
>>>>>>>>>>>
>>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>>>
>>>>>>>>>>> [ ] +0
>>>>>>>>>>>
>>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Gengliang Wang
>>>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> John Zhuge
>>
>>
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by YE <ad...@gmail.com>.
+1 (non-binding)
> On Feb 28, 2026, at 1:02 PM, Chao Sun <su...@apache.org> wrote:
>
> +1
>
> On Fri, Feb 27, 2026 at 3:12 AM DB Tsai <dbtsai@dbtsai.com <ma...@dbtsai.com>> wrote:
>> +1
>>
>> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
>>
>>> On Feb 25, 2026, at 11:33 PM, Kousuke Saruta <sarutak@apache.org <ma...@apache.org>> wrote:
>>>
>>> +1
>>>
>>>
>>> 2026年2月26日(木) 16:32 Mridul Muralidharan <mridul@gmail.com <ma...@gmail.com>>:
>>>>
>>>> +1
>>>>
>>>> Regards,
>>>> Mridul
>>>>
>>>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cloud0fan@gmail.com <ma...@gmail.com>> wrote:
>>>>> +1
>>>>>
>>>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <yao@apache.org <ma...@apache.org>> wrote:
>>>>>> +1
>>>>>>
>>>>>> Tathagata Das <tathagata.das1565@gmail.com <ma...@gmail.com>> 于2026年2月26日周四 11:52写道：
>>>>>>> +1 binding. very excited.
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jzhuge@apache.org <ma...@apache.org>> wrote:
>>>>>>>> +1 (non-binding) Appreciate it!
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <karuppayya1990@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>> +1 (non-binding)
>>>>>>>>>
>>>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <qlong8@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>> +1 non-binding
>>>>>>>>>>
>>>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <viirya@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>>>> Hi Spark devs,
>>>>>>>>>>>>
>>>>>>>>>>>> I'd like to call a vote on the SPIP: DSV2 Enhanced Partition Stats Filtering.
>>>>>>>>>>>>
>>>>>>>>>>>> Summary:
>>>>>>>>>>>>
>>>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide full data-skipping capabilities comparable to Spark-native sources, primarily due to limitations in Catalyst expression evaluation.
>>>>>>>>>>>>
>>>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To support this, Spark will push new PartitionPredicate objects that encapsulate Catalyst partition filter expressions and the evaluation logic, allowing data sources to skip irrelevant partitions effectively.
>>>>>>>>>>>>
>>>>>>>>>>>> Relevant Links:
>>>>>>>>>>>>
>>>>>>>>>>>> SPIP Doc: https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>>>>> Discuss Thread: https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>>>>>
>>>>>>>>>>>> JIRA: SPARK-55596 <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>>>>>> POC PR: PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>>>>>>>> The vote will be open for at least 72 hours. Please vote:
>>>>>>>>>>>>
>>>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>>>>
>>>>>>>>>>>> [ ] +0
>>>>>>>>>>>>
>>>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Gengliang Wang
>>>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> John Zhuge
>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Chao Sun <su...@apache.org>.
+1
On Fri, Feb 27, 2026 at 3:12 AM DB Tsai <db...@dbtsai.com> wrote:
> +1
>
> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
>
> On Feb 25, 2026, at 11:33 PM, Kousuke Saruta <sa...@apache.org> wrote:
>
> +1
>
>
> 2026年2月26日(木) 16:32 Mridul Muralidharan <mr...@gmail.com>:
>
>>
>> +1
>>
>> Regards,
>> Mridul
>>
>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cl...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
>>>
>>>> +1
>>>>
>>>> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
>>>>
>>>>> +1 binding. very excited.
>>>>>
>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
>>>>>
>>>>>> +1 (non-binding) Appreciate it!
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 (non-binding)
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 non-binding
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1
>>>>>>>>>
>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Spark devs,
>>>>>>>>>>
>>>>>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition
>>>>>>>>>> Stats Filtering.*
>>>>>>>>>>
>>>>>>>>>> *Summary:*
>>>>>>>>>>
>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide
>>>>>>>>>> full data-skipping capabilities comparable to Spark-native sources,
>>>>>>>>>> primarily due to limitations in Catalyst expression evaluation.
>>>>>>>>>>
>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity.
>>>>>>>>>> To support this, Spark will push new *PartitionPredicate*
>>>>>>>>>> objects that encapsulate Catalyst partition filter expressions and the
>>>>>>>>>> evaluation logic, allowing data sources to skip irrelevant partitions
>>>>>>>>>> effectively.
>>>>>>>>>>
>>>>>>>>>> *Relevant Links:*
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> - *SPIP Doc:*
>>>>>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> *Discuss Thread:*
>>>>>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> *JIRA:* SPARK-55596
>>>>>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> *POC PR:* PR 54459
>>>>>>>>>> <https://github.com/apache/spark/pull/54459>
>>>>>>>>>>
>>>>>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>>>>>
>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>>
>>>>>>>>>> [ ] +0
>>>>>>>>>>
>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Gengliang Wang
>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> John Zhuge
>>>>>>
>>>>>
>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by DB Tsai <db...@dbtsai.com>.
+1
DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
> On Feb 25, 2026, at 11:33 PM, Kousuke Saruta <sa...@apache.org> wrote:
>
> +1
>
>
> 2026年2月26日(木) 16:32 Mridul Muralidharan <mridul@gmail.com <ma...@gmail.com>>:
>>
>> +1
>>
>> Regards,
>> Mridul
>>
>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cloud0fan@gmail.com <ma...@gmail.com>> wrote:
>>> +1
>>>
>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <yao@apache.org <ma...@apache.org>> wrote:
>>>> +1
>>>>
>>>> Tathagata Das <tathagata.das1565@gmail.com <ma...@gmail.com>> 于2026年2月26日周四 11:52写道：
>>>>> +1 binding. very excited.
>>>>>
>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jzhuge@apache.org <ma...@apache.org>> wrote:
>>>>>> +1 (non-binding) Appreciate it!
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <karuppayya1990@gmail.com <ma...@gmail.com>> wrote:
>>>>>>> +1 (non-binding)
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <qlong8@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>> +1 non-binding
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <viirya@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>> +1
>>>>>>>>>
>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>> Hi Spark devs,
>>>>>>>>>>
>>>>>>>>>> I'd like to call a vote on the SPIP: DSV2 Enhanced Partition Stats Filtering.
>>>>>>>>>>
>>>>>>>>>> Summary:
>>>>>>>>>>
>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide full data-skipping capabilities comparable to Spark-native sources, primarily due to limitations in Catalyst expression evaluation.
>>>>>>>>>>
>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To support this, Spark will push new PartitionPredicate objects that encapsulate Catalyst partition filter expressions and the evaluation logic, allowing data sources to skip irrelevant partitions effectively.
>>>>>>>>>>
>>>>>>>>>> Relevant Links:
>>>>>>>>>>
>>>>>>>>>> SPIP Doc: https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>>> Discuss Thread: https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>>>
>>>>>>>>>> JIRA: SPARK-55596 <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>>>> POC PR: PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>>>>>> The vote will be open for at least 72 hours. Please vote:
>>>>>>>>>>
>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>>
>>>>>>>>>> [ ] +0
>>>>>>>>>>
>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Gengliang Wang
>>>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> John Zhuge
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Dongjoon Hyun <do...@gmail.com>.
+1
On Thu, Feb 26, 2026 at 10:15 Szehon Ho <sz...@gmail.com> wrote:
> +1 (non binding)
> Thanks all,
> Szehon
>
> On Thu, Feb 26, 2026 at 7:17 AM Peter Toth <pe...@gmail.com> wrote:
>
>> +1 (non-binding)
>>
>> On Thu, Feb 26, 2026 at 8:42 AM Cheng Pan <pa...@gmail.com> wrote:
>>
>>> +1 (non-binding)
>>>
>>> Thanks,
>>> Cheng Pan
>>>
>>>
>>>
>>> On Feb 26, 2026, at 15:33, Kousuke Saruta <sa...@apache.org> wrote:
>>>
>>> +1
>>>
>>>
>>> 2026年2月26日(木) 16:32 Mridul Muralidharan <mr...@gmail.com>:
>>>
>>>>
>>>> +1
>>>>
>>>> Regards,
>>>> Mridul
>>>>
>>>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cl...@gmail.com>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
>>>>>>
>>>>>>> +1 binding. very excited.
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 (non-binding) Appreciate it!
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <
>>>>>>>> karuppayya1990@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> +1 (non-binding)
>>>>>>>>>
>>>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 non-binding
>>>>>>>>>>
>>>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Spark devs,
>>>>>>>>>>>>
>>>>>>>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition
>>>>>>>>>>>> Stats Filtering.*
>>>>>>>>>>>>
>>>>>>>>>>>> *Summary:*
>>>>>>>>>>>>
>>>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide
>>>>>>>>>>>> full data-skipping capabilities comparable to Spark-native sources,
>>>>>>>>>>>> primarily due to limitations in Catalyst expression evaluation.
>>>>>>>>>>>>
>>>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping
>>>>>>>>>>>> parity. To support this, Spark will push new
>>>>>>>>>>>> *PartitionPredicate* objects that encapsulate Catalyst
>>>>>>>>>>>> partition filter expressions and the evaluation logic, allowing data
>>>>>>>>>>>> sources to skip irrelevant partitions effectively.
>>>>>>>>>>>>
>>>>>>>>>>>> *Relevant Links:*
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> - *SPIP Doc:*
>>>>>>>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> *Discuss Thread:*
>>>>>>>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> *JIRA:* SPARK-55596
>>>>>>>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> *POC PR:* PR 54459
>>>>>>>>>>>> <https://github.com/apache/spark/pull/54459>
>>>>>>>>>>>>
>>>>>>>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>>>>>>>
>>>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>>>>
>>>>>>>>>>>> [ ] +0
>>>>>>>>>>>>
>>>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Gengliang Wang
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> John Zhuge
>>>>>>>>
>>>>>>>
>>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Szehon Ho <sz...@gmail.com>.
+1 (non binding)
Thanks all,
Szehon
On Thu, Feb 26, 2026 at 7:17 AM Peter Toth <pe...@gmail.com> wrote:
> +1 (non-binding)
>
> On Thu, Feb 26, 2026 at 8:42 AM Cheng Pan <pa...@gmail.com> wrote:
>
>> +1 (non-binding)
>>
>> Thanks,
>> Cheng Pan
>>
>>
>>
>> On Feb 26, 2026, at 15:33, Kousuke Saruta <sa...@apache.org> wrote:
>>
>> +1
>>
>>
>> 2026年2月26日(木) 16:32 Mridul Muralidharan <mr...@gmail.com>:
>>
>>>
>>> +1
>>>
>>> Regards,
>>> Mridul
>>>
>>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cl...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
>>>>>
>>>>>> +1 binding. very excited.
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 (non-binding) Appreciate it!
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 (non-binding)
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1 non-binding
>>>>>>>>>
>>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1
>>>>>>>>>>
>>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Spark devs,
>>>>>>>>>>>
>>>>>>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition
>>>>>>>>>>> Stats Filtering.*
>>>>>>>>>>>
>>>>>>>>>>> *Summary:*
>>>>>>>>>>>
>>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide
>>>>>>>>>>> full data-skipping capabilities comparable to Spark-native sources,
>>>>>>>>>>> primarily due to limitations in Catalyst expression evaluation.
>>>>>>>>>>>
>>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity.
>>>>>>>>>>> To support this, Spark will push new *PartitionPredicate*
>>>>>>>>>>> objects that encapsulate Catalyst partition filter expressions and the
>>>>>>>>>>> evaluation logic, allowing data sources to skip irrelevant partitions
>>>>>>>>>>> effectively.
>>>>>>>>>>>
>>>>>>>>>>> *Relevant Links:*
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> - *SPIP Doc:*
>>>>>>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> *Discuss Thread:*
>>>>>>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> *JIRA:* SPARK-55596
>>>>>>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> *POC PR:* PR 54459
>>>>>>>>>>> <https://github.com/apache/spark/pull/54459>
>>>>>>>>>>>
>>>>>>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>>>>>>
>>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>>>
>>>>>>>>>>> [ ] +0
>>>>>>>>>>>
>>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Gengliang Wang
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> John Zhuge
>>>>>>>
>>>>>>
>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Peter Toth <pe...@gmail.com>.
+1 (non-binding)
On Thu, Feb 26, 2026 at 8:42 AM Cheng Pan <pa...@gmail.com> wrote:
> +1 (non-binding)
>
> Thanks,
> Cheng Pan
>
>
>
> On Feb 26, 2026, at 15:33, Kousuke Saruta <sa...@apache.org> wrote:
>
> +1
>
>
> 2026年2月26日(木) 16:32 Mridul Muralidharan <mr...@gmail.com>:
>
>>
>> +1
>>
>> Regards,
>> Mridul
>>
>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cl...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
>>>
>>>> +1
>>>>
>>>> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
>>>>
>>>>> +1 binding. very excited.
>>>>>
>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
>>>>>
>>>>>> +1 (non-binding) Appreciate it!
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 (non-binding)
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 non-binding
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1
>>>>>>>>>
>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Spark devs,
>>>>>>>>>>
>>>>>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition
>>>>>>>>>> Stats Filtering.*
>>>>>>>>>>
>>>>>>>>>> *Summary:*
>>>>>>>>>>
>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide
>>>>>>>>>> full data-skipping capabilities comparable to Spark-native sources,
>>>>>>>>>> primarily due to limitations in Catalyst expression evaluation.
>>>>>>>>>>
>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity.
>>>>>>>>>> To support this, Spark will push new *PartitionPredicate*
>>>>>>>>>> objects that encapsulate Catalyst partition filter expressions and the
>>>>>>>>>> evaluation logic, allowing data sources to skip irrelevant partitions
>>>>>>>>>> effectively.
>>>>>>>>>>
>>>>>>>>>> *Relevant Links:*
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> - *SPIP Doc:*
>>>>>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> *Discuss Thread:*
>>>>>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> *JIRA:* SPARK-55596
>>>>>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> *POC PR:* PR 54459
>>>>>>>>>> <https://github.com/apache/spark/pull/54459>
>>>>>>>>>>
>>>>>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>>>>>
>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>>
>>>>>>>>>> [ ] +0
>>>>>>>>>>
>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Gengliang Wang
>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> John Zhuge
>>>>>>
>>>>>
>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Cheng Pan <pa...@gmail.com>.
+1 (non-binding)
Thanks,
Cheng Pan
> On Feb 26, 2026, at 15:33, Kousuke Saruta <sa...@apache.org> wrote:
>
> +1
>
>
> 2026年2月26日(木) 16:32 Mridul Muralidharan <mridul@gmail.com <ma...@gmail.com>>:
>>
>> +1
>>
>> Regards,
>> Mridul
>>
>> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cloud0fan@gmail.com <ma...@gmail.com>> wrote:
>>> +1
>>>
>>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <yao@apache.org <ma...@apache.org>> wrote:
>>>> +1
>>>>
>>>> Tathagata Das <tathagata.das1565@gmail.com <ma...@gmail.com>> 于2026年2月26日周四 11:52写道：
>>>>> +1 binding. very excited.
>>>>>
>>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jzhuge@apache.org <ma...@apache.org>> wrote:
>>>>>> +1 (non-binding) Appreciate it!
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <karuppayya1990@gmail.com <ma...@gmail.com>> wrote:
>>>>>>> +1 (non-binding)
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <qlong8@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>> +1 non-binding
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <viirya@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>> +1
>>>>>>>>>
>>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>>>> Hi Spark devs,
>>>>>>>>>>
>>>>>>>>>> I'd like to call a vote on the SPIP: DSV2 Enhanced Partition Stats Filtering.
>>>>>>>>>>
>>>>>>>>>> Summary:
>>>>>>>>>>
>>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide full data-skipping capabilities comparable to Spark-native sources, primarily due to limitations in Catalyst expression evaluation.
>>>>>>>>>>
>>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To support this, Spark will push new PartitionPredicate objects that encapsulate Catalyst partition filter expressions and the evaluation logic, allowing data sources to skip irrelevant partitions effectively.
>>>>>>>>>>
>>>>>>>>>> Relevant Links:
>>>>>>>>>>
>>>>>>>>>> SPIP Doc: https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>>> Discuss Thread: https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>>>
>>>>>>>>>> JIRA: SPARK-55596 <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>>>> POC PR: PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>>>>>> The vote will be open for at least 72 hours. Please vote:
>>>>>>>>>>
>>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>>
>>>>>>>>>> [ ] +0
>>>>>>>>>>
>>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Gengliang Wang
>>>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> John Zhuge
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Kousuke Saruta <sa...@apache.org>.
+1
2026年2月26日(木) 16:32 Mridul Muralidharan <mr...@gmail.com>:
>
> +1
>
> Regards,
> Mridul
>
> On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cl...@gmail.com> wrote:
>
>> +1
>>
>> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
>>
>>> +1
>>>
>>> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
>>>
>>>> +1 binding. very excited.
>>>>
>>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
>>>>
>>>>> +1 (non-binding) Appreciate it!
>>>>>
>>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> +1 (non-binding)
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 non-binding
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1
>>>>>>>>
>>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Spark devs,
>>>>>>>>>
>>>>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition
>>>>>>>>> Stats Filtering.*
>>>>>>>>>
>>>>>>>>> *Summary:*
>>>>>>>>>
>>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide full
>>>>>>>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>>>>>>>> due to limitations in Catalyst expression evaluation.
>>>>>>>>>
>>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity.
>>>>>>>>> To support this, Spark will push new *PartitionPredicate* objects
>>>>>>>>> that encapsulate Catalyst partition filter expressions and the evaluation
>>>>>>>>> logic, allowing data sources to skip irrelevant partitions effectively.
>>>>>>>>>
>>>>>>>>> *Relevant Links:*
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> - *SPIP Doc:*
>>>>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> *Discuss Thread:*
>>>>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> *JIRA:* SPARK-55596
>>>>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>>>>>
>>>>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>>>>
>>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>>
>>>>>>>>> [ ] +0
>>>>>>>>>
>>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Gengliang Wang
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>> --
>>>>> John Zhuge
>>>>>
>>>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Mridul Muralidharan <mr...@gmail.com>.
+1
Regards,
Mridul
On Wed, Feb 25, 2026 at 9:58 PM Wenchen Fan <cl...@gmail.com> wrote:
> +1
>
> On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
>
>> +1
>>
>> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
>>
>>> +1 binding. very excited.
>>>
>>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
>>>
>>>> +1 (non-binding) Appreciate it!
>>>>
>>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
>>>> wrote:
>>>>
>>>>> +1 (non-binding)
>>>>>
>>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com> wrote:
>>>>>
>>>>>> +1 non-binding
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Spark devs,
>>>>>>>>
>>>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition
>>>>>>>> Stats Filtering.*
>>>>>>>>
>>>>>>>> *Summary:*
>>>>>>>>
>>>>>>>> The DataSource V2 (DSV2) framework does not currently provide full
>>>>>>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>>>>>>> due to limitations in Catalyst expression evaluation.
>>>>>>>>
>>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>>>>>>> support this, Spark will push new *PartitionPredicate* objects
>>>>>>>> that encapsulate Catalyst partition filter expressions and the evaluation
>>>>>>>> logic, allowing data sources to skip irrelevant partitions effectively.
>>>>>>>>
>>>>>>>> *Relevant Links:*
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> - *SPIP Doc:*
>>>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>>> -
>>>>>>>>
>>>>>>>> *Discuss Thread:*
>>>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>>> -
>>>>>>>>
>>>>>>>> *JIRA:* SPARK-55596
>>>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>>> -
>>>>>>>>
>>>>>>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>>>>
>>>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>>>
>>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>>
>>>>>>>> [ ] +0
>>>>>>>>
>>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Gengliang Wang
>>>>>>>>
>>>>>>>>
>>>>
>>>> --
>>>> John Zhuge
>>>>
>>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Wenchen Fan <cl...@gmail.com>.
+1
On Thu, Feb 26, 2026 at 11:54 AM Kent Yao <ya...@apache.org> wrote:
> +1
>
> Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
>
>> +1 binding. very excited.
>>
>> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
>>
>>> +1 (non-binding) Appreciate it!
>>>
>>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
>>> wrote:
>>>
>>>> +1 (non-binding)
>>>>
>>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com> wrote:
>>>>
>>>>> +1 non-binding
>>>>>
>>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Spark devs,
>>>>>>>
>>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>>>>>>> Filtering.*
>>>>>>>
>>>>>>> *Summary:*
>>>>>>>
>>>>>>> The DataSource V2 (DSV2) framework does not currently provide full
>>>>>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>>>>>> due to limitations in Catalyst expression evaluation.
>>>>>>>
>>>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>>>>>> support this, Spark will push new *PartitionPredicate* objects that
>>>>>>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>>>>>>> allowing data sources to skip irrelevant partitions effectively.
>>>>>>>
>>>>>>> *Relevant Links:*
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> - *SPIP Doc:*
>>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>>> -
>>>>>>>
>>>>>>> *Discuss Thread:*
>>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>>> -
>>>>>>>
>>>>>>> *JIRA:* SPARK-55596
>>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>>> -
>>>>>>>
>>>>>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>>>
>>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>>
>>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>>
>>>>>>> [ ] +0
>>>>>>>
>>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Gengliang Wang
>>>>>>>
>>>>>>>
>>>
>>> --
>>> John Zhuge
>>>
>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Kent Yao <ya...@apache.org>.
+1
Tathagata Das <ta...@gmail.com> 于2026年2月26日周四 11:52写道：
> +1 binding. very excited.
>
> On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
>
>> +1 (non-binding) Appreciate it!
>>
>> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
>> wrote:
>>
>>> +1 (non-binding)
>>>
>>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com> wrote:
>>>
>>>> +1 non-binding
>>>>
>>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Spark devs,
>>>>>>
>>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>>>>>> Filtering.*
>>>>>>
>>>>>> *Summary:*
>>>>>>
>>>>>> The DataSource V2 (DSV2) framework does not currently provide full
>>>>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>>>>> due to limitations in Catalyst expression evaluation.
>>>>>>
>>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>>>>> support this, Spark will push new *PartitionPredicate* objects that
>>>>>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>>>>>> allowing data sources to skip irrelevant partitions effectively.
>>>>>>
>>>>>> *Relevant Links:*
>>>>>>
>>>>>> -
>>>>>>
>>>>>> - *SPIP Doc:*
>>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>>> -
>>>>>>
>>>>>> *Discuss Thread:*
>>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>>> -
>>>>>>
>>>>>> *JIRA:* SPARK-55596
>>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>>> -
>>>>>>
>>>>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>>
>>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>>
>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>>
>>>>>> [ ] +0
>>>>>>
>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>>
>>>>>> Thanks,
>>>>>> Gengliang Wang
>>>>>>
>>>>>>
>>
>> --
>> John Zhuge
>>
>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Tathagata Das <ta...@gmail.com>.
+1 binding. very excited.
On Wed, Feb 25, 2026 at 10:47 PM John Zhuge <jz...@apache.org> wrote:
> +1 (non-binding) Appreciate it!
>
> On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com>
> wrote:
>
>> +1 (non-binding)
>>
>> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com> wrote:
>>
>>> +1 non-binding
>>>
>>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Spark devs,
>>>>>
>>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>>>>> Filtering.*
>>>>>
>>>>> *Summary:*
>>>>>
>>>>> The DataSource V2 (DSV2) framework does not currently provide full
>>>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>>>> due to limitations in Catalyst expression evaluation.
>>>>>
>>>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>>>> support this, Spark will push new *PartitionPredicate* objects that
>>>>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>>>>> allowing data sources to skip irrelevant partitions effectively.
>>>>>
>>>>> *Relevant Links:*
>>>>>
>>>>> -
>>>>>
>>>>> - *SPIP Doc:*
>>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>>> -
>>>>>
>>>>> *Discuss Thread:*
>>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>>> -
>>>>>
>>>>> *JIRA:* SPARK-55596
>>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>>> -
>>>>>
>>>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>>
>>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>>
>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>
>>>>> [ ] +0
>>>>>
>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>
>>>>> Thanks,
>>>>> Gengliang Wang
>>>>>
>>>>>
>
> --
> John Zhuge
>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by John Zhuge <jz...@apache.org>.
+1 (non-binding) Appreciate it!
On Wed, Feb 25, 2026 at 4:13 PM karuppayya <ka...@gmail.com> wrote:
> +1 (non-binding)
>
> On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com> wrote:
>
>> +1 non-binding
>>
>> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com> wrote:
>>>
>>>> Hi Spark devs,
>>>>
>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>>>> Filtering.*
>>>>
>>>> *Summary:*
>>>>
>>>> The DataSource V2 (DSV2) framework does not currently provide full
>>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>>> due to limitations in Catalyst expression evaluation.
>>>>
>>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>>> support this, Spark will push new *PartitionPredicate* objects that
>>>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>>>> allowing data sources to skip irrelevant partitions effectively.
>>>>
>>>> *Relevant Links:*
>>>>
>>>> -
>>>>
>>>> - *SPIP Doc:*
>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>> -
>>>>
>>>> *Discuss Thread:*
>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>> -
>>>>
>>>> *JIRA:* SPARK-55596
>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>> -
>>>>
>>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>
>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>
>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>
>>>> [ ] +0
>>>>
>>>> [ ] -1: I don't think this is a good idea because ...
>>>>
>>>> Thanks,
>>>> Gengliang Wang
>>>>
>>>>
--
John Zhuge
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by karuppayya <ka...@gmail.com>.
+1 (non-binding)
On Wed, Feb 25, 2026 at 3:48 PM Qiegang Long <ql...@gmail.com> wrote:
> +1 non-binding
>
> On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
>
>> +1
>>
>> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com> wrote:
>>
>>> Hi Spark devs,
>>>
>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>>> Filtering.*
>>>
>>> *Summary:*
>>>
>>> The DataSource V2 (DSV2) framework does not currently provide full
>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>> due to limitations in Catalyst expression evaluation.
>>>
>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>> support this, Spark will push new *PartitionPredicate* objects that
>>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>>> allowing data sources to skip irrelevant partitions effectively.
>>>
>>> *Relevant Links:*
>>>
>>> -
>>>
>>> - *SPIP Doc:*
>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>> -
>>>
>>> *Discuss Thread:*
>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>> -
>>>
>>> *JIRA:* SPARK-55596
>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>> -
>>>
>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>
>>> *The vote will be open for at least 72 hours. *Please vote:
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>>
>>> [ ] +0
>>>
>>> [ ] -1: I don't think this is a good idea because ...
>>>
>>> Thanks,
>>> Gengliang Wang
>>>
>>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Qiegang Long <ql...@gmail.com>.
+1 non-binding
On Wed, Feb 25, 2026 at 6:42 PM L. C. Hsieh <vi...@gmail.com> wrote:
> +1
>
> On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com> wrote:
>
>> Hi Spark devs,
>>
>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>> Filtering.*
>>
>> *Summary:*
>>
>> The DataSource V2 (DSV2) framework does not currently provide full
>> data-skipping capabilities comparable to Spark-native sources, primarily
>> due to limitations in Catalyst expression evaluation.
>>
>> This SPIP bridges that gap to achieve partition-skipping parity. To
>> support this, Spark will push new *PartitionPredicate* objects that
>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>> allowing data sources to skip irrelevant partitions effectively.
>>
>> *Relevant Links:*
>>
>> -
>>
>> - *SPIP Doc:*
>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>> -
>>
>> *Discuss Thread:*
>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>> -
>>
>> *JIRA:* SPARK-55596
>> <https://issues.apache.org/jira/browse/SPARK-55596>
>> -
>>
>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>
>> *The vote will be open for at least 72 hours. *Please vote:
>>
>> [ ] +1: Accept the proposal as an official SPIP
>>
>> [ ] +0
>>
>> [ ] -1: I don't think this is a good idea because ...
>>
>> Thanks,
>> Gengliang Wang
>>
>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by "L. C. Hsieh" <vi...@gmail.com>.
+1
On Wed, Feb 25, 2026 at 2:45 PM Gengliang Wang <lt...@gmail.com> wrote:
> Hi Spark devs,
>
> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
> Filtering.*
>
> *Summary:*
>
> The DataSource V2 (DSV2) framework does not currently provide full
> data-skipping capabilities comparable to Spark-native sources, primarily
> due to limitations in Catalyst expression evaluation.
>
> This SPIP bridges that gap to achieve partition-skipping parity. To
> support this, Spark will push new *PartitionPredicate* objects that
> encapsulate Catalyst partition filter expressions and the evaluation logic,
> allowing data sources to skip irrelevant partitions effectively.
>
> *Relevant Links:*
>
> -
>
> - *SPIP Doc:*
> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
> -
>
> *Discuss Thread:*
> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
> -
>
> *JIRA:* SPARK-55596 <https://issues.apache.org/jira/browse/SPARK-55596>
> -
>
> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>
> *The vote will be open for at least 72 hours. *Please vote:
>
> [ ] +1: Accept the proposal as an official SPIP
>
> [ ] +0
>
> [ ] -1: I don't think this is a good idea because ...
>
> Thanks,
> Gengliang Wang
>
>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Anton Okolnychyi <ao...@gmail.com>.
+1 (non-binding)
ср, 25 лют. 2026 р. о 14:55 Holden Karau <ho...@gmail.com> пише:
> +1, very excited for this feature.
>
> On Wed, Feb 25, 2026 at 2:53 PM huaxin gao <hu...@gmail.com> wrote:
>
>> +1
>>
>> On Wed, Feb 25, 2026 at 2:47 PM Gengliang Wang <lt...@gmail.com> wrote:
>>
>>> Starting with my own +1.
>>>
>>> On Wed, Feb 25, 2026 at 2:44 PM Gengliang Wang <lt...@gmail.com> wrote:
>>>
>>>> Hi Spark devs,
>>>>
>>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>>>> Filtering.*
>>>>
>>>> *Summary:*
>>>>
>>>> The DataSource V2 (DSV2) framework does not currently provide full
>>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>>> due to limitations in Catalyst expression evaluation.
>>>>
>>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>>> support this, Spark will push new *PartitionPredicate* objects that
>>>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>>>> allowing data sources to skip irrelevant partitions effectively.
>>>>
>>>> *Relevant Links:*
>>>>
>>>> -
>>>>
>>>> - *SPIP Doc:*
>>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>>> -
>>>>
>>>> *Discuss Thread:*
>>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>>> -
>>>>
>>>> *JIRA:* SPARK-55596
>>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>>> -
>>>>
>>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>>
>>>> *The vote will be open for at least 72 hours. *Please vote:
>>>>
>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>
>>>> [ ] +0
>>>>
>>>> [ ] -1: I don't think this is a good idea because ...
>>>>
>>>> Thanks,
>>>> Gengliang Wang
>>>>
>>>>
>
> --
> Twitter: https://twitter.com/holdenkarau
> Fight Health Insurance: https://www.fighthealthinsurance.com/
> <https://www.fighthealthinsurance.com/?q=hk_email>
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
> Pronouns: she/her
>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Holden Karau <ho...@gmail.com>.
+1, very excited for this feature.
On Wed, Feb 25, 2026 at 2:53 PM huaxin gao <hu...@gmail.com> wrote:
> +1
>
> On Wed, Feb 25, 2026 at 2:47 PM Gengliang Wang <lt...@gmail.com> wrote:
>
>> Starting with my own +1.
>>
>> On Wed, Feb 25, 2026 at 2:44 PM Gengliang Wang <lt...@gmail.com> wrote:
>>
>>> Hi Spark devs,
>>>
>>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>>> Filtering.*
>>>
>>> *Summary:*
>>>
>>> The DataSource V2 (DSV2) framework does not currently provide full
>>> data-skipping capabilities comparable to Spark-native sources, primarily
>>> due to limitations in Catalyst expression evaluation.
>>>
>>> This SPIP bridges that gap to achieve partition-skipping parity. To
>>> support this, Spark will push new *PartitionPredicate* objects that
>>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>>> allowing data sources to skip irrelevant partitions effectively.
>>>
>>> *Relevant Links:*
>>>
>>> -
>>>
>>> - *SPIP Doc:*
>>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>>> -
>>>
>>> *Discuss Thread:*
>>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>>> -
>>>
>>> *JIRA:* SPARK-55596
>>> <https://issues.apache.org/jira/browse/SPARK-55596>
>>> -
>>>
>>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>>
>>> *The vote will be open for at least 72 hours. *Please vote:
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>>
>>> [ ] +0
>>>
>>> [ ] -1: I don't think this is a good idea because ...
>>>
>>> Thanks,
>>> Gengliang Wang
>>>
>>>
--
Twitter: https://twitter.com/holdenkarau
Fight Health Insurance: https://www.fighthealthinsurance.com/
<https://www.fighthealthinsurance.com/?q=hk_email>
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Pronouns: she/her
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by huaxin gao <hu...@gmail.com>.
+1
On Wed, Feb 25, 2026 at 2:47 PM Gengliang Wang <lt...@gmail.com> wrote:
> Starting with my own +1.
>
> On Wed, Feb 25, 2026 at 2:44 PM Gengliang Wang <lt...@gmail.com> wrote:
>
>> Hi Spark devs,
>>
>> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
>> Filtering.*
>>
>> *Summary:*
>>
>> The DataSource V2 (DSV2) framework does not currently provide full
>> data-skipping capabilities comparable to Spark-native sources, primarily
>> due to limitations in Catalyst expression evaluation.
>>
>> This SPIP bridges that gap to achieve partition-skipping parity. To
>> support this, Spark will push new *PartitionPredicate* objects that
>> encapsulate Catalyst partition filter expressions and the evaluation logic,
>> allowing data sources to skip irrelevant partitions effectively.
>>
>> *Relevant Links:*
>>
>> -
>>
>> - *SPIP Doc:*
>> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
>> -
>>
>> *Discuss Thread:*
>> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
>> -
>>
>> *JIRA:* SPARK-55596
>> <https://issues.apache.org/jira/browse/SPARK-55596>
>> -
>>
>> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>>
>> *The vote will be open for at least 72 hours. *Please vote:
>>
>> [ ] +1: Accept the proposal as an official SPIP
>>
>> [ ] +0
>>
>> [ ] -1: I don't think this is a good idea because ...
>>
>> Thanks,
>> Gengliang Wang
>>
>>
Re: [VOTE] SPIP: DSV2 Enhanced Partition Stats Filtering
Posted by Gengliang Wang <lt...@gmail.com>.
Starting with my own +1.
On Wed, Feb 25, 2026 at 2:44 PM Gengliang Wang <lt...@gmail.com> wrote:
> Hi Spark devs,
>
> I'd like to call a vote on the SPIP*: DSV2 Enhanced Partition Stats
> Filtering.*
>
> *Summary:*
>
> The DataSource V2 (DSV2) framework does not currently provide full
> data-skipping capabilities comparable to Spark-native sources, primarily
> due to limitations in Catalyst expression evaluation.
>
> This SPIP bridges that gap to achieve partition-skipping parity. To
> support this, Spark will push new *PartitionPredicate* objects that
> encapsulate Catalyst partition filter expressions and the evaluation logic,
> allowing data sources to skip irrelevant partitions effectively.
>
> *Relevant Links:*
>
> -
>
> - *SPIP Doc:*
> https://docs.google.com/document/d/17vcw411PxSRLWoK-BiLI56UiNdokLWtovF8JZUlDTOo/edit?usp=sharing
> -
>
> *Discuss Thread:*
> https://lists.apache.org/thread/p2cwngj9bmtcbmyplds833s9lwts8bwc
> -
>
> *JIRA:* SPARK-55596 <https://issues.apache.org/jira/browse/SPARK-55596>
> -
>
> *POC PR:* PR 54459 <https://github.com/apache/spark/pull/54459>
>
> *The vote will be open for at least 72 hours. *Please vote:
>
> [ ] +1: Accept the proposal as an official SPIP
>
> [ ] +0
>
> [ ] -1: I don't think this is a good idea because ...
>
> Thanks,
> Gengliang Wang
>
>