You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gluten.apache.org by WeitingChen <we...@apache.org> on 2025/02/10 09:20:37 UTC

Deprecate Spark3.2 support plan

Hi all,

This is a discussion thread about deprecating Spark3.2 in the
following version of Gluten release.

We are planning to deprecate Spark3.2 maintenance for several reasons.

1. To reduce CI/CD resource consumption in github action: currently
Gluten needs to run every version of Spark(current support
3.2/3.3/3.4/3.5) during submitting each PR. We would like to reduce
resource usage during CI/CD testing.

2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
customers are moving to Spark3.4/3.5 and in the upcoming Gluten
release, we will support Spark 4.0 as well.

3. For Spark3.2 users, we will recommend to use an older version of
Gluten such as Gluten 1.2 or 1.3 and a minor version upgrade(such as
1.2.2/1.3.1) is on demand to give the best support.

It's better to support up to four versions of Spark in a Gluten
release cycle. The proposed plan will be in below:

Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API support only

Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT support &
deprecate Spark 3.2 support

Please feel free to discuss this if you have any comments on this plan.

Thanks,
Weiting

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
For additional commands, e-mail: dev-help@gluten.apache.org

Re: Deprecate Spark3.2 support plan

Posted by Yang Zhang <ya...@apache.org>.
+1 on deprecate Spark3.2 support in main branch.

WeitingChen <we...@apache.org> 于 2025年2月10日周一 下午5:20写道：

> Hi all,
>
> This is a discussion thread about deprecating Spark3.2 in the
> following version of Gluten release.
>
> We are planning to deprecate Spark3.2 maintenance for several reasons.
>
> 1. To reduce CI/CD resource consumption in github action: currently
> Gluten needs to run every version of Spark(current support
> 3.2/3.3/3.4/3.5) during submitting each PR. We would like to reduce
> resource usage during CI/CD testing.
>
> 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
> customers are moving to Spark3.4/3.5 and in the upcoming Gluten
> release, we will support Spark 4.0 as well.
>
> 3. For Spark3.2 users, we will recommend to use an older version of
> Gluten such as Gluten 1.2 or 1.3 and a minor version upgrade(such as
> 1.2.2/1.3.1) is on demand to give the best support.
>
> It's better to support up to four versions of Spark in a Gluten
> release cycle. The proposed plan will be in below:
>
> Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API support only
>
> Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT support &
> deprecate Spark 3.2 support
>
> Please feel free to discuss this if you have any comments on this plan.
>
> Thanks,
> Weiting
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> For additional commands, e-mail: dev-help@gluten.apache.org
>
>

Re: Deprecate Spark3.2 support plan

Posted by Terry Wang <zj...@gmail.com>.
+1

On Wed, Feb 12, 2025 at 3:06 PM Ma, Yan <ya...@intel.com> wrote:

> +1
>
> -----Original Message-----
> From: Rong Ma <ro...@gmail.com>
> Sent: Wednesday, February 12, 2025 3:00 PM
> To: dev@gluten.apache.org
> Cc: dev@gluten.apache.org
> Subject: Re: Deprecate Spark3.2 support plan
>
>
> +1
>
> > On Feb 12, 2025, at 06:50, Ke Jia <ke...@gmail.com> wrote:
> >
> > +1
> >
> >> On Wed, Feb 12, 2025 at 2:27 PM Philo <ph...@apache.org> wrote:
> >>
> >> +1
> >>
> >>> On Wed, Feb 12, 2025 at 1:52 PM Hongze Zhang <ho...@apache.org>
> wrote:
> >>>
> >>> +1
> >>>
> >>> On Wed, Feb 12, 2025 at 1:44 PM Jacky Lee <qc...@gmail.com> wrote:
> >>>>
> >>>> +1
> >>>>
> >>>> Chang Chen <ch...@apache.org> 于2025年2月12日周三 13:06写道：
> >>>>>
> >>>>> +1
> >>>>>
> >>>>> On 2025/02/10 09:20:37 WeitingChen wrote:
> >>>>>> Hi all,
> >>>>>>
> >>>>>> This is a discussion thread about deprecating Spark3.2 in the
> >>>>>> following version of Gluten release.
> >>>>>>
> >>>>>> We are planning to deprecate Spark3.2 maintenance for several
> >>> reasons.
> >>>>>>
> >>>>>> 1. To reduce CI/CD resource consumption in github action:
> >>>>>> currently Gluten needs to run every version of Spark(current
> >>>>>> support
> >>>>>> 3.2/3.3/3.4/3.5) during submitting each PR. We would like to
> >>>>>> reduce resource usage during CI/CD testing.
> >>>>>>
> >>>>>> 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
> >>>>>> customers are moving to Spark3.4/3.5 and in the upcoming Gluten
> >>>>>> release, we will support Spark 4.0 as well.
> >>>>>>
> >>>>>> 3. For Spark3.2 users, we will recommend to use an older version
> >>>>>> of Gluten such as Gluten 1.2 or 1.3 and a minor version
> >>>>>> upgrade(such
> >> as
> >>>>>> 1.2.2/1.3.1) is on demand to give the best support.
> >>>>>>
> >>>>>> It's better to support up to four versions of Spark in a Gluten
> >>>>>> release cycle. The proposed plan will be in below:
> >>>>>>
> >>>>>> Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API
> >> support
> >>> only
> >>>>>>
> >>>>>> Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT
> >> support &
> >>>>>> deprecate Spark 3.2 support
> >>>>>>
> >>>>>> Please feel free to discuss this if you have any comments on this
> >>> plan.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Weiting
> >>>>>>
> >>>>>>
> >> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For
> >>>>>> additional commands, e-mail: dev-help@gluten.apache.org
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> ------------------------------------------------------------------
> >>>>> --- To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For
> >>>>> additional commands, e-mail: dev-help@gluten.apache.org
> >>>>>
> >>>>
> >>>> -------------------------------------------------------------------
> >>>> -- To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For
> >>>> additional commands, e-mail: dev-help@gluten.apache.org
> >>>>
> >>>
> >>> --------------------------------------------------------------------
> >>> - To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For
> >>> additional commands, e-mail: dev-help@gluten.apache.org
> >>>
> >>>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For additional
> commands, e-mail: dev-help@gluten.apache.org
>
>

-- 
Best Regards,
Terry Wang

RE: Deprecate Spark3.2 support plan

Posted by "Ma, Yan" <ya...@intel.com>.
+1

-----Original Message-----
From: Rong Ma <ro...@gmail.com> 
Sent: Wednesday, February 12, 2025 3:00 PM
To: dev@gluten.apache.org
Cc: dev@gluten.apache.org
Subject: Re: Deprecate Spark3.2 support plan


+1

> On Feb 12, 2025, at 06:50, Ke Jia <ke...@gmail.com> wrote:
> 
> +1
> 
>> On Wed, Feb 12, 2025 at 2:27 PM Philo <ph...@apache.org> wrote:
>> 
>> +1
>> 
>>> On Wed, Feb 12, 2025 at 1:52 PM Hongze Zhang <ho...@apache.org> wrote:
>>> 
>>> +1
>>> 
>>> On Wed, Feb 12, 2025 at 1:44 PM Jacky Lee <qc...@gmail.com> wrote:
>>>> 
>>>> +1
>>>> 
>>>> Chang Chen <ch...@apache.org> 于2025年2月12日周三 13:06写道：
>>>>> 
>>>>> +1
>>>>> 
>>>>> On 2025/02/10 09:20:37 WeitingChen wrote:
>>>>>> Hi all,
>>>>>> 
>>>>>> This is a discussion thread about deprecating Spark3.2 in the 
>>>>>> following version of Gluten release.
>>>>>> 
>>>>>> We are planning to deprecate Spark3.2 maintenance for several
>>> reasons.
>>>>>> 
>>>>>> 1. To reduce CI/CD resource consumption in github action: 
>>>>>> currently Gluten needs to run every version of Spark(current 
>>>>>> support
>>>>>> 3.2/3.3/3.4/3.5) during submitting each PR. We would like to 
>>>>>> reduce resource usage during CI/CD testing.
>>>>>> 
>>>>>> 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten 
>>>>>> customers are moving to Spark3.4/3.5 and in the upcoming Gluten 
>>>>>> release, we will support Spark 4.0 as well.
>>>>>> 
>>>>>> 3. For Spark3.2 users, we will recommend to use an older version 
>>>>>> of Gluten such as Gluten 1.2 or 1.3 and a minor version 
>>>>>> upgrade(such
>> as
>>>>>> 1.2.2/1.3.1) is on demand to give the best support.
>>>>>> 
>>>>>> It's better to support up to four versions of Spark in a Gluten 
>>>>>> release cycle. The proposed plan will be in below:
>>>>>> 
>>>>>> Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API
>> support
>>> only
>>>>>> 
>>>>>> Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT
>> support &
>>>>>> deprecate Spark 3.2 support
>>>>>> 
>>>>>> Please feel free to discuss this if you have any comments on this
>>> plan.
>>>>>> 
>>>>>> Thanks,
>>>>>> Weiting
>>>>>> 
>>>>>> 
>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For 
>>>>>> additional commands, e-mail: dev-help@gluten.apache.org
>>>>>> 
>>>>>> 
>>>>> 
>>>>> ------------------------------------------------------------------
>>>>> --- To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For 
>>>>> additional commands, e-mail: dev-help@gluten.apache.org
>>>>> 
>>>> 
>>>> -------------------------------------------------------------------
>>>> -- To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For 
>>>> additional commands, e-mail: dev-help@gluten.apache.org
>>>> 
>>> 
>>> --------------------------------------------------------------------
>>> - To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For 
>>> additional commands, e-mail: dev-help@gluten.apache.org
>>> 
>>> 
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org For additional commands, e-mail: dev-help@gluten.apache.org

Re: Deprecate Spark3.2 support plan

Posted by Rong Ma <ro...@gmail.com>.
+1

> On Feb 12, 2025, at 06:50, Ke Jia <ke...@gmail.com> wrote:
> 
> +1
> 
>> On Wed, Feb 12, 2025 at 2:27 PM Philo <ph...@apache.org> wrote:
>> 
>> +1
>> 
>>> On Wed, Feb 12, 2025 at 1:52 PM Hongze Zhang <ho...@apache.org> wrote:
>>> 
>>> +1
>>> 
>>> On Wed, Feb 12, 2025 at 1:44 PM Jacky Lee <qc...@gmail.com> wrote:
>>>> 
>>>> +1
>>>> 
>>>> Chang Chen <ch...@apache.org> 于2025年2月12日周三 13:06写道：
>>>>> 
>>>>> +1
>>>>> 
>>>>> On 2025/02/10 09:20:37 WeitingChen wrote:
>>>>>> Hi all,
>>>>>> 
>>>>>> This is a discussion thread about deprecating Spark3.2 in the
>>>>>> following version of Gluten release.
>>>>>> 
>>>>>> We are planning to deprecate Spark3.2 maintenance for several
>>> reasons.
>>>>>> 
>>>>>> 1. To reduce CI/CD resource consumption in github action: currently
>>>>>> Gluten needs to run every version of Spark(current support
>>>>>> 3.2/3.3/3.4/3.5) during submitting each PR. We would like to reduce
>>>>>> resource usage during CI/CD testing.
>>>>>> 
>>>>>> 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
>>>>>> customers are moving to Spark3.4/3.5 and in the upcoming Gluten
>>>>>> release, we will support Spark 4.0 as well.
>>>>>> 
>>>>>> 3. For Spark3.2 users, we will recommend to use an older version of
>>>>>> Gluten such as Gluten 1.2 or 1.3 and a minor version upgrade(such
>> as
>>>>>> 1.2.2/1.3.1) is on demand to give the best support.
>>>>>> 
>>>>>> It's better to support up to four versions of Spark in a Gluten
>>>>>> release cycle. The proposed plan will be in below:
>>>>>> 
>>>>>> Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API
>> support
>>> only
>>>>>> 
>>>>>> Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT
>> support &
>>>>>> deprecate Spark 3.2 support
>>>>>> 
>>>>>> Please feel free to discuss this if you have any comments on this
>>> plan.
>>>>>> 
>>>>>> Thanks,
>>>>>> Weiting
>>>>>> 
>>>>>> 
>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
>>>>>> For additional commands, e-mail: dev-help@gluten.apache.org
>>>>>> 
>>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
>>>>> For additional commands, e-mail: dev-help@gluten.apache.org
>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
>>>> For additional commands, e-mail: dev-help@gluten.apache.org
>>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
>>> For additional commands, e-mail: dev-help@gluten.apache.org
>>> 
>>> 
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
For additional commands, e-mail: dev-help@gluten.apache.org

Re: Deprecate Spark3.2 support plan

Posted by Ke Jia <ke...@gmail.com>.
+1

On Wed, Feb 12, 2025 at 2:27 PM Philo <ph...@apache.org> wrote:

> +1
>
> On Wed, Feb 12, 2025 at 1:52 PM Hongze Zhang <ho...@apache.org> wrote:
>
> > +1
> >
> > On Wed, Feb 12, 2025 at 1:44 PM Jacky Lee <qc...@gmail.com> wrote:
> > >
> > > +1
> > >
> > > Chang Chen <ch...@apache.org> 于2025年2月12日周三 13:06写道：
> > > >
> > > > +1
> > > >
> > > > On 2025/02/10 09:20:37 WeitingChen wrote:
> > > > > Hi all,
> > > > >
> > > > > This is a discussion thread about deprecating Spark3.2 in the
> > > > > following version of Gluten release.
> > > > >
> > > > > We are planning to deprecate Spark3.2 maintenance for several
> > reasons.
> > > > >
> > > > > 1. To reduce CI/CD resource consumption in github action: currently
> > > > > Gluten needs to run every version of Spark(current support
> > > > > 3.2/3.3/3.4/3.5) during submitting each PR. We would like to reduce
> > > > > resource usage during CI/CD testing.
> > > > >
> > > > > 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
> > > > > customers are moving to Spark3.4/3.5 and in the upcoming Gluten
> > > > > release, we will support Spark 4.0 as well.
> > > > >
> > > > > 3. For Spark3.2 users, we will recommend to use an older version of
> > > > > Gluten such as Gluten 1.2 or 1.3 and a minor version upgrade(such
> as
> > > > > 1.2.2/1.3.1) is on demand to give the best support.
> > > > >
> > > > > It's better to support up to four versions of Spark in a Gluten
> > > > > release cycle. The proposed plan will be in below:
> > > > >
> > > > > Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API
> support
> > only
> > > > >
> > > > > Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT
> support &
> > > > > deprecate Spark 3.2 support
> > > > >
> > > > > Please feel free to discuss this if you have any comments on this
> > plan.
> > > > >
> > > > > Thanks,
> > > > > Weiting
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > > > > For additional commands, e-mail: dev-help@gluten.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > > > For additional commands, e-mail: dev-help@gluten.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > > For additional commands, e-mail: dev-help@gluten.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > For additional commands, e-mail: dev-help@gluten.apache.org
> >
> >
>

Re: Deprecate Spark3.2 support plan

Posted by Philo <ph...@apache.org>.
+1

On Wed, Feb 12, 2025 at 1:52 PM Hongze Zhang <ho...@apache.org> wrote:

> +1
>
> On Wed, Feb 12, 2025 at 1:44 PM Jacky Lee <qc...@gmail.com> wrote:
> >
> > +1
> >
> > Chang Chen <ch...@apache.org> 于2025年2月12日周三 13:06写道：
> > >
> > > +1
> > >
> > > On 2025/02/10 09:20:37 WeitingChen wrote:
> > > > Hi all,
> > > >
> > > > This is a discussion thread about deprecating Spark3.2 in the
> > > > following version of Gluten release.
> > > >
> > > > We are planning to deprecate Spark3.2 maintenance for several
> reasons.
> > > >
> > > > 1. To reduce CI/CD resource consumption in github action: currently
> > > > Gluten needs to run every version of Spark(current support
> > > > 3.2/3.3/3.4/3.5) during submitting each PR. We would like to reduce
> > > > resource usage during CI/CD testing.
> > > >
> > > > 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
> > > > customers are moving to Spark3.4/3.5 and in the upcoming Gluten
> > > > release, we will support Spark 4.0 as well.
> > > >
> > > > 3. For Spark3.2 users, we will recommend to use an older version of
> > > > Gluten such as Gluten 1.2 or 1.3 and a minor version upgrade(such as
> > > > 1.2.2/1.3.1) is on demand to give the best support.
> > > >
> > > > It's better to support up to four versions of Spark in a Gluten
> > > > release cycle. The proposed plan will be in below:
> > > >
> > > > Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API support
> only
> > > >
> > > > Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT support &
> > > > deprecate Spark 3.2 support
> > > >
> > > > Please feel free to discuss this if you have any comments on this
> plan.
> > > >
> > > > Thanks,
> > > > Weiting
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > > > For additional commands, e-mail: dev-help@gluten.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > > For additional commands, e-mail: dev-help@gluten.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > For additional commands, e-mail: dev-help@gluten.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> For additional commands, e-mail: dev-help@gluten.apache.org
>
>

Re: Deprecate Spark3.2 support plan

Posted by Hongze Zhang <ho...@apache.org>.
+1

On Wed, Feb 12, 2025 at 1:44 PM Jacky Lee <qc...@gmail.com> wrote:
>
> +1
>
> Chang Chen <ch...@apache.org> 于2025年2月12日周三 13:06写道：
> >
> > +1
> >
> > On 2025/02/10 09:20:37 WeitingChen wrote:
> > > Hi all,
> > >
> > > This is a discussion thread about deprecating Spark3.2 in the
> > > following version of Gluten release.
> > >
> > > We are planning to deprecate Spark3.2 maintenance for several reasons.
> > >
> > > 1. To reduce CI/CD resource consumption in github action: currently
> > > Gluten needs to run every version of Spark(current support
> > > 3.2/3.3/3.4/3.5) during submitting each PR. We would like to reduce
> > > resource usage during CI/CD testing.
> > >
> > > 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
> > > customers are moving to Spark3.4/3.5 and in the upcoming Gluten
> > > release, we will support Spark 4.0 as well.
> > >
> > > 3. For Spark3.2 users, we will recommend to use an older version of
> > > Gluten such as Gluten 1.2 or 1.3 and a minor version upgrade(such as
> > > 1.2.2/1.3.1) is on demand to give the best support.
> > >
> > > It's better to support up to four versions of Spark in a Gluten
> > > release cycle. The proposed plan will be in below:
> > >
> > > Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API support only
> > >
> > > Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT support &
> > > deprecate Spark 3.2 support
> > >
> > > Please feel free to discuss this if you have any comments on this plan.
> > >
> > > Thanks,
> > > Weiting
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > > For additional commands, e-mail: dev-help@gluten.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > For additional commands, e-mail: dev-help@gluten.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> For additional commands, e-mail: dev-help@gluten.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
For additional commands, e-mail: dev-help@gluten.apache.org

Re: Deprecate Spark3.2 support plan

Posted by Jacky Lee <qc...@gmail.com>.
+1

Chang Chen <ch...@apache.org> 于2025年2月12日周三 13:06写道：
>
> +1
>
> On 2025/02/10 09:20:37 WeitingChen wrote:
> > Hi all,
> >
> > This is a discussion thread about deprecating Spark3.2 in the
> > following version of Gluten release.
> >
> > We are planning to deprecate Spark3.2 maintenance for several reasons.
> >
> > 1. To reduce CI/CD resource consumption in github action: currently
> > Gluten needs to run every version of Spark(current support
> > 3.2/3.3/3.4/3.5) during submitting each PR. We would like to reduce
> > resource usage during CI/CD testing.
> >
> > 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
> > customers are moving to Spark3.4/3.5 and in the upcoming Gluten
> > release, we will support Spark 4.0 as well.
> >
> > 3. For Spark3.2 users, we will recommend to use an older version of
> > Gluten such as Gluten 1.2 or 1.3 and a minor version upgrade(such as
> > 1.2.2/1.3.1) is on demand to give the best support.
> >
> > It's better to support up to four versions of Spark in a Gluten
> > release cycle. The proposed plan will be in below:
> >
> > Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API support only
> >
> > Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT support &
> > deprecate Spark 3.2 support
> >
> > Please feel free to discuss this if you have any comments on this plan.
> >
> > Thanks,
> > Weiting
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> > For additional commands, e-mail: dev-help@gluten.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> For additional commands, e-mail: dev-help@gluten.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
For additional commands, e-mail: dev-help@gluten.apache.org

Re: Deprecate Spark3.2 support plan

Posted by Chang Chen <ch...@apache.org>.
+1

On 2025/02/10 09:20:37 WeitingChen wrote:
> Hi all,
> 
> This is a discussion thread about deprecating Spark3.2 in the
> following version of Gluten release.
> 
> We are planning to deprecate Spark3.2 maintenance for several reasons.
> 
> 1. To reduce CI/CD resource consumption in github action: currently
> Gluten needs to run every version of Spark(current support
> 3.2/3.3/3.4/3.5) during submitting each PR. We would like to reduce
> resource usage during CI/CD testing.
> 
> 2. Most customers are moving to Spark 3.4/3.5/4.0: Many Gluten
> customers are moving to Spark3.4/3.5 and in the upcoming Gluten
> release, we will support Spark 4.0 as well.
> 
> 3. For Spark3.2 users, we will recommend to use an older version of
> Gluten such as Gluten 1.2 or 1.3 and a minor version upgrade(such as
> 1.2.2/1.3.1) is on demand to give the best support.
> 
> It's better to support up to four versions of Spark in a Gluten
> release cycle. The proposed plan will be in below:
> 
> Gluten 1.4.0: target for Q1'25 and plan to add Spark 4.0 API support only
> 
> Gluten 1.5.0: target for Q2'25 and plan to add Spark 4.0 UT support &
> deprecate Spark 3.2 support
> 
> Please feel free to discuss this if you have any comments on this plan.
> 
> Thanks,
> Weiting
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
> For additional commands, e-mail: dev-help@gluten.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@gluten.apache.org
For additional commands, e-mail: dev-help@gluten.apache.org