You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Jungtaek Lim <ka...@gmail.com> on 2026/08/03 00:25:08 UTC
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
FYI, this is the last reminder before branch cut on Apache Spark 4.3.0.
I'll cut the branch tomorrow (Aug 3 in US tz) if there is no reasonable
objection. I'll give an update tomorrow once I cut the branch, so that
folks have visibility of whether the fix is included or not.
On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <ka...@gmail.com>
wrote:
> UPDATE: The website is updated to reflect the plan.
>
> https://spark.apache.org/versioning-policy.html
>
> As a reminder, *Aug 1 is the rough date to cut the branch for Apache
> Spark 4.3.0*. Since there is no feedback on the plan so far, I'd consider
> this as no issue in proceeding with the plan.
>
>
> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org> wrote:
>
>> SGTM
>>
>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>
>>> Just a head-up, since there is no major feedback, I'll proceed to update
>>> the release plan as discussed with prior SPIP.
>>>
>>> Given the practice of release scheduling, the plan for Apache 4.3.0 will
>>> be following:
>>>
>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>> * Mid Aug 2026: QA period
>>> * Late Aug 2026: RC
>>>
>>> I'm going to update the versioning policy page with the above.
>>>
>>>
>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> Hi dev,
>>>>
>>>> I'm happy to serve the role of the release manager for Apache Spark
>>>> 4.3. Since this is the first release we followed the new policy of 3 months
>>>> cadence, there might be some confusion/concern and I need your help and
>>>> understanding. (Thanks in advance for your help and patience!)
>>>>
>>>> I'm collecting the projects the workstream has been happening on Apache
>>>> Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP, but
>>>> please give the head-up for your project which has partially landed to
>>>> Apache Spark 4.3 and you are close to the finish line.
>>>>
>>>> For the projects you are unlikely to complete before branch cut
>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
>>>> "be open to postpone the schedule for the project" is a key success factor
>>>> for faster release. Thanks for your understanding!
>>>>
>>>> I'll update the versioning policy page to reflect the expected schedule
>>>> of Apache Spark 4.3.0 release. Please post here for any questions and
>>>> voices/feedback.
>>>>
>>>> Thanks!
>>>> Jungtaek Lim (HeartSaVioR)
>>>>
>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <gu...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I would like to send a gentle reminder as heads up that we're going to
>>>>> cut the branch early Aug, 1st to prepare branch-4.3. In the SPIP about
>>>>> releasing, we decided to do the faster release ones, and this would be the
>>>>> very first one.
>>>>>
>>>>> I know that we just released Spark 4.2. That was unexpectedly
>>>>> delayed some more. I would like to avoid affecting our first release
>>>>> schedule - It has been our chronic issue in history, and I do think we
>>>>> should change this from 4.3 at the first predictable release, and push it
>>>>> forward. Let's get this done and let the releases be predictable.
>>>>>
>>>>> PS: would need to update
>>>>> https://spark.apache.org/versioning-policy.html for details about
>>>>> branchcut, RC and releases.
>>>>>
>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
>>>>> release manager for 4.3.
>>>>>
>>>>>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Jungtaek Lim <ka...@gmail.com>.
I'm generally OK with merging them in Spark 4.3.0 as long as these commits
do not open/extend the half-baked feature. (If you think that's not
something to "complete" the feature, please consider more on the
justification) - regardless of my comment, I'd leave the final decision to
committer+ who signs off those PRs.
On Tue, Aug 11, 2026 at 9:14 AM Yan Yan <yy...@apache.org> wrote:
> Hi Jungtaek,
>
> Apologies for the delay, but I wonder if it's possible to include the
> following PRs in the 4.3 release:
> - https://github.com/apache/spark/pull/57799/
> - https://github.com/apache/spark/pull/57585
> - https://github.com/apache/spark/pull/57865
>
> They are follow up changes to https://github.com/apache/spark/pull/57582
> which got merged around the branch cut time, and they together complete the
> support for passing options to the connector during DSv2 read/write
> operations while ensuring table version remains consistent throughout a
> single query. Some are very close to be merged.
>
> Thank you!
> Yan
>
> On 2026/08/05 04:52:39 Jungtaek Lim wrote:
> > Thanks for the information! I'm OK with including it in Apache Spark
> 4.3.0
> > as long as it is self-contained and reviewers also agree with your
> > statement. I'll leave the decision to the committer who will perform the
> > final sign-off.
> >
> > On Wed, Aug 5, 2026 at 12:10 PM Anurag Mantripragada <
> > anuragmantri93@gmail.com> wrote:
> >
> > > Hi Jungtaek,
> > >
> > > To add some context, this PR is part of the SPIP to introduce new DSv2
> > > capabilities for connectors supporting column-level updates. It is
> fully
> > > self-contained and enables UPDATE statements, allowing connectors to
> opt in
> > > and receive only the updated data. This will help connectors like
> Iceberg
> > > and Delta fully implement column-level UPDATE statements, which are the
> > > primary use cases for this feature. I expect just one more JIRA next to
> > > support MERGE INTO cases, which can target 4.4.
> > >
> > > I wanted to bring this to your attention as the release manager since
> the
> > > PR might be merged soon. I will defer to the decision made by the
> reviewer,
> > > committer, and you (release manager).
> > >
> > > Thanks,
> > > Anurag
> > >
> > >
> > > On Tue, Aug 4, 2026 at 5:38 PM Jungtaek Lim <
> kabhwan.opensource@gmail.com>
> > > wrote:
> > >
> > >> Hi Anurag, could you please provide the impact of merging this in
> Apache
> > >> Spark 4.3.0?
> > >>
> > >> The SPIP only has one JIRA ticket, which doesn't provide the proper
> > >> visibility of where we are. I'd not encourage people to leave the
> project
> > >> to be released as half-baked and trigger "unknown unknown"; does the
> PR
> > >> bring up self-contained value? I assume the answer is yes, but let's
> punt
> > >> to 4.4.0 if the answer is no.
> > >>
> > >> On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada <
> > >> anuragmantripragada@apache.org> wrote:
> > >>
> > >>> Hi Jungteak,
> > >>>
> > >>> Would it be possible to include SPARK-58111
> > >>> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part
> of
> > >>> the SPIP SPARK-56599 <
> https://issues.apache.org/jira/browse/SPARK-56599>)
> > >>> in the 4.3 release?
> > >>>
> > >>> The PR has gone through several review rounds. Including this
> > >>> foundational PR with UPDATE support in 4.3 would allow projects like
> Apache
> > >>> Iceberg to start integrating sooner.
> > >>>
> > >>> Please let me know your thoughts.
> > >>>
> > >>> ~ Anurag
> > >>>
> > >>> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim <
> > >>> kabhwan.opensource@gmail.com> wrote:
> > >>>
> > >>>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with
> > >>>> post branch cut tasks.
> > >>>>
> > >>>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0
> (PR
> > >>>> <https://github.com/apache/spark/pull/57737>)
> > >>>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to
> > >>>> master (PR <https://github.com/apache/spark/pull/57738>)
> > >>>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release
> > >>>> integration (PR <https://github.com/apache/spark/pull/57741>)
> > >>>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and
> default
> > >>>> branch (PR <https://github.com/apache/spark/pull/57740>)
> > >>>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to
> branch-4.x
> > >>>> on branch-4.x (PR <https://github.com/apache/spark/pull/57739>)
> > >>>>
> > >>>> Folks, please help reviewing the changes; thanks for your help!
> > >>>>
> > >>>>
> > >>>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <pa...@gmail.com>
> wrote:
> > >>>>
> > >>>>> I just merged https://github.com/apache/spark/pull/57718, and I
> > >>>>> suppose it fixes the compilation for branch-4.x and branch-4.3
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Cheng Pan
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Aug 4, 2026, at 10:58, Jungtaek Lim <
> kabhwan.opensource@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>> UPDATE: It takes more than I expect to do post-branch cut, because
> > >>>>> branch-4.x doesn't run CI and seems broken with maven now (haven't
> run sbt
> > >>>>> yet).
> > >>>>> Maybe I'll need to fix the issue altogether in branch-4.x and
> > >>>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT.
> > >>>>>
> > >>>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <
> > >>>>> kabhwan.opensource@gmail.com> wrote:
> > >>>>>
> > >>>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC
> > >>>>>> members who merge the code change to Apache Spark 4.3.0 would
> need to
> > >>>>>> include branch-4.3 into the merge branches to account for it. I'm
> still
> > >>>>>> making the change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll
> give an
> > >>>>>> update once that is done.
> > >>>>>>
> > >>>>>> This is a soft cut and we tend to broadly accept code changes to
> > >>>>>> branch-4.3 till RC is started, but please refrain from merging
> the code
> > >>>>>> change which could make a release at risk of releasing in a month
> or so.
> > >>>>>> Dragging a release process to September 4th week would risk the
> > >>>>>> unavailability of PMC members in East Asia for a couple weeks, so
> ideally
> > >>>>>> it would be nice to finalize the RC vote till Sep 3rd week.
> > >>>>>>
> > >>>>>> Thanks!
> > >>>>>>
> > >>>>>>
> > >>>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <
> > >>>>>> kabhwan.opensource@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> FYI, this is the last reminder before branch cut on Apache Spark
> > >>>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is
> no
> > >>>>>>> reasonable objection. I'll give an update tomorrow once I cut
> the branch,
> > >>>>>>> so that folks have visibility of whether the fix is included or
> not.
> > >>>>>>>
> > >>>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <
> > >>>>>>> kabhwan.opensource@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> UPDATE: The website is updated to reflect the plan.
> > >>>>>>>>
> > >>>>>>>> https://spark.apache.org/versioning-policy.html
> > >>>>>>>>
> > >>>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for
> > >>>>>>>> Apache Spark 4.3.0*. Since there is no feedback on the plan so
> > >>>>>>>> far, I'd consider this as no issue in proceeding with the plan.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <
> gurwls223@apache.org>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> SGTM
> > >>>>>>>>>
> > >>>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <
> > >>>>>>>>> kabhwan.opensource@gmail.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Just a head-up, since there is no major feedback, I'll
> proceed to
> > >>>>>>>>>> update the release plan as discussed with prior SPIP.
> > >>>>>>>>>>
> > >>>>>>>>>> Given the practice of release scheduling, the plan for Apache
> > >>>>>>>>>> 4.3.0 will be following:
> > >>>>>>>>>>
> > >>>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
> > >>>>>>>>>> * Mid Aug 2026: QA period
> > >>>>>>>>>> * Late Aug 2026: RC
> > >>>>>>>>>>
> > >>>>>>>>>> I'm going to update the versioning policy page with the above.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
> > >>>>>>>>>> kabhwan.opensource@gmail.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Hi dev,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I'm happy to serve the role of the release manager for Apache
> > >>>>>>>>>>> Spark 4.3. Since this is the first release we followed the
> new policy of 3
> > >>>>>>>>>>> months cadence, there might be some confusion/concern and I
> need your help
> > >>>>>>>>>>> and understanding. (Thanks in advance for your help and
> patience!)
> > >>>>>>>>>>>
> > >>>>>>>>>>> I'm collecting the projects the workstream has been
> happening on
> > >>>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be
> focusing on SPIP,
> > >>>>>>>>>>> but please give the head-up for your project which has
> partially landed to
> > >>>>>>>>>>> Apache Spark 4.3 and you are close to the finish line.
> > >>>>>>>>>>>
> > >>>>>>>>>>> For the projects you are unlikely to complete before branch
> cut
> > >>>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are
> encouraged to have a
> > >>>>>>>>>>> flag to disable the partial change to take effect in Apache
> Spark 4.3, and
> > >>>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of
> missing Apache
> > >>>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months
> release cadence, and
> > >>>>>>>>>>> "be open to postpone the schedule for the project" is a key
> success factor
> > >>>>>>>>>>> for faster release. Thanks for your understanding!
> > >>>>>>>>>>>
> > >>>>>>>>>>> I'll update the versioning policy page to reflect the
> expected
> > >>>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for
> any questions
> > >>>>>>>>>>> and voices/feedback.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks!
> > >>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <
> > >>>>>>>>>>> gurwls223@apache.org> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Hi all,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I would like to send a gentle reminder as heads up that
> > >>>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare
> branch-4.3. In the
> > >>>>>>>>>>>> SPIP about releasing, we decided to do the faster release
> ones, and this
> > >>>>>>>>>>>> would be the very first one.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I know that we just released Spark 4.2. That was
> unexpectedly
> > >>>>>>>>>>>> delayed some more. I would like to avoid affecting our
> first release
> > >>>>>>>>>>>> schedule - It has been our chronic issue in history, and I
> do think we
> > >>>>>>>>>>>> should change this from 4.3 at the first predictable
> release, and push it
> > >>>>>>>>>>>> forward. Let's get this done and let the releases be
> predictable.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> PS: would need to update
> > >>>>>>>>>>>> https://spark.apache.org/versioning-policy.html for
> > >>>>>>>>>>>> details about branchcut, RC and releases.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be
> the
> > >>>>>>>>>>>> release manager for 4.3.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Yan Yan <yy...@apache.org>.
Hi Jungtaek,
Apologies for the delay, but I wonder if it's possible to include the following PRs in the 4.3 release:
- https://github.com/apache/spark/pull/57799/
- https://github.com/apache/spark/pull/57585
- https://github.com/apache/spark/pull/57865
They are follow up changes to https://github.com/apache/spark/pull/57582 which got merged around the branch cut time, and they together complete the support for passing options to the connector during DSv2 read/write operations while ensuring table version remains consistent throughout a single query. Some are very close to be merged.
Thank you!
Yan
On 2026/08/05 04:52:39 Jungtaek Lim wrote:
> Thanks for the information! I'm OK with including it in Apache Spark 4.3.0
> as long as it is self-contained and reviewers also agree with your
> statement. I'll leave the decision to the committer who will perform the
> final sign-off.
>
> On Wed, Aug 5, 2026 at 12:10 PM Anurag Mantripragada <
> anuragmantri93@gmail.com> wrote:
>
> > Hi Jungtaek,
> >
> > To add some context, this PR is part of the SPIP to introduce new DSv2
> > capabilities for connectors supporting column-level updates. It is fully
> > self-contained and enables UPDATE statements, allowing connectors to opt in
> > and receive only the updated data. This will help connectors like Iceberg
> > and Delta fully implement column-level UPDATE statements, which are the
> > primary use cases for this feature. I expect just one more JIRA next to
> > support MERGE INTO cases, which can target 4.4.
> >
> > I wanted to bring this to your attention as the release manager since the
> > PR might be merged soon. I will defer to the decision made by the reviewer,
> > committer, and you (release manager).
> >
> > Thanks,
> > Anurag
> >
> >
> > On Tue, Aug 4, 2026 at 5:38 PM Jungtaek Lim <ka...@gmail.com>
> > wrote:
> >
> >> Hi Anurag, could you please provide the impact of merging this in Apache
> >> Spark 4.3.0?
> >>
> >> The SPIP only has one JIRA ticket, which doesn't provide the proper
> >> visibility of where we are. I'd not encourage people to leave the project
> >> to be released as half-baked and trigger "unknown unknown"; does the PR
> >> bring up self-contained value? I assume the answer is yes, but let's punt
> >> to 4.4.0 if the answer is no.
> >>
> >> On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada <
> >> anuragmantripragada@apache.org> wrote:
> >>
> >>> Hi Jungteak,
> >>>
> >>> Would it be possible to include SPARK-58111
> >>> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part of
> >>> the SPIP SPARK-56599 <https://issues.apache.org/jira/browse/SPARK-56599>)
> >>> in the 4.3 release?
> >>>
> >>> The PR has gone through several review rounds. Including this
> >>> foundational PR with UPDATE support in 4.3 would allow projects like Apache
> >>> Iceberg to start integrating sooner.
> >>>
> >>> Please let me know your thoughts.
> >>>
> >>> ~ Anurag
> >>>
> >>> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim <
> >>> kabhwan.opensource@gmail.com> wrote:
> >>>
> >>>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with
> >>>> post branch cut tasks.
> >>>>
> >>>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR
> >>>> <https://github.com/apache/spark/pull/57737>)
> >>>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to
> >>>> master (PR <https://github.com/apache/spark/pull/57738>)
> >>>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release
> >>>> integration (PR <https://github.com/apache/spark/pull/57741>)
> >>>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default
> >>>> branch (PR <https://github.com/apache/spark/pull/57740>)
> >>>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x
> >>>> on branch-4.x (PR <https://github.com/apache/spark/pull/57739>)
> >>>>
> >>>> Folks, please help reviewing the changes; thanks for your help!
> >>>>
> >>>>
> >>>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <pa...@gmail.com> wrote:
> >>>>
> >>>>> I just merged https://github.com/apache/spark/pull/57718, and I
> >>>>> suppose it fixes the compilation for branch-4.x and branch-4.3
> >>>>>
> >>>>> Thanks,
> >>>>> Cheng Pan
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Aug 4, 2026, at 10:58, Jungtaek Lim <ka...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>> UPDATE: It takes more than I expect to do post-branch cut, because
> >>>>> branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt
> >>>>> yet).
> >>>>> Maybe I'll need to fix the issue altogether in branch-4.x and
> >>>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT.
> >>>>>
> >>>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <
> >>>>> kabhwan.opensource@gmail.com> wrote:
> >>>>>
> >>>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC
> >>>>>> members who merge the code change to Apache Spark 4.3.0 would need to
> >>>>>> include branch-4.3 into the merge branches to account for it. I'm still
> >>>>>> making the change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an
> >>>>>> update once that is done.
> >>>>>>
> >>>>>> This is a soft cut and we tend to broadly accept code changes to
> >>>>>> branch-4.3 till RC is started, but please refrain from merging the code
> >>>>>> change which could make a release at risk of releasing in a month or so.
> >>>>>> Dragging a release process to September 4th week would risk the
> >>>>>> unavailability of PMC members in East Asia for a couple weeks, so ideally
> >>>>>> it would be nice to finalize the RC vote till Sep 3rd week.
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <
> >>>>>> kabhwan.opensource@gmail.com> wrote:
> >>>>>>
> >>>>>>> FYI, this is the last reminder before branch cut on Apache Spark
> >>>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is no
> >>>>>>> reasonable objection. I'll give an update tomorrow once I cut the branch,
> >>>>>>> so that folks have visibility of whether the fix is included or not.
> >>>>>>>
> >>>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <
> >>>>>>> kabhwan.opensource@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> UPDATE: The website is updated to reflect the plan.
> >>>>>>>>
> >>>>>>>> https://spark.apache.org/versioning-policy.html
> >>>>>>>>
> >>>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for
> >>>>>>>> Apache Spark 4.3.0*. Since there is no feedback on the plan so
> >>>>>>>> far, I'd consider this as no issue in proceeding with the plan.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> SGTM
> >>>>>>>>>
> >>>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <
> >>>>>>>>> kabhwan.opensource@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> Just a head-up, since there is no major feedback, I'll proceed to
> >>>>>>>>>> update the release plan as discussed with prior SPIP.
> >>>>>>>>>>
> >>>>>>>>>> Given the practice of release scheduling, the plan for Apache
> >>>>>>>>>> 4.3.0 will be following:
> >>>>>>>>>>
> >>>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
> >>>>>>>>>> * Mid Aug 2026: QA period
> >>>>>>>>>> * Late Aug 2026: RC
> >>>>>>>>>>
> >>>>>>>>>> I'm going to update the versioning policy page with the above.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
> >>>>>>>>>> kabhwan.opensource@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi dev,
> >>>>>>>>>>>
> >>>>>>>>>>> I'm happy to serve the role of the release manager for Apache
> >>>>>>>>>>> Spark 4.3. Since this is the first release we followed the new policy of 3
> >>>>>>>>>>> months cadence, there might be some confusion/concern and I need your help
> >>>>>>>>>>> and understanding. (Thanks in advance for your help and patience!)
> >>>>>>>>>>>
> >>>>>>>>>>> I'm collecting the projects the workstream has been happening on
> >>>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP,
> >>>>>>>>>>> but please give the head-up for your project which has partially landed to
> >>>>>>>>>>> Apache Spark 4.3 and you are close to the finish line.
> >>>>>>>>>>>
> >>>>>>>>>>> For the projects you are unlikely to complete before branch cut
> >>>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
> >>>>>>>>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
> >>>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
> >>>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
> >>>>>>>>>>> "be open to postpone the schedule for the project" is a key success factor
> >>>>>>>>>>> for faster release. Thanks for your understanding!
> >>>>>>>>>>>
> >>>>>>>>>>> I'll update the versioning policy page to reflect the expected
> >>>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any questions
> >>>>>>>>>>> and voices/feedback.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <
> >>>>>>>>>>> gurwls223@apache.org> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I would like to send a gentle reminder as heads up that
> >>>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare branch-4.3. In the
> >>>>>>>>>>>> SPIP about releasing, we decided to do the faster release ones, and this
> >>>>>>>>>>>> would be the very first one.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly
> >>>>>>>>>>>> delayed some more. I would like to avoid affecting our first release
> >>>>>>>>>>>> schedule - It has been our chronic issue in history, and I do think we
> >>>>>>>>>>>> should change this from 4.3 at the first predictable release, and push it
> >>>>>>>>>>>> forward. Let's get this done and let the releases be predictable.
> >>>>>>>>>>>>
> >>>>>>>>>>>> PS: would need to update
> >>>>>>>>>>>> https://spark.apache.org/versioning-policy.html for
> >>>>>>>>>>>> details about branchcut, RC and releases.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
> >>>>>>>>>>>> release manager for 4.3.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Jungtaek Lim <ka...@gmail.com>.
Thanks for the information! I'm OK with including it in Apache Spark 4.3.0
as long as it is self-contained and reviewers also agree with your
statement. I'll leave the decision to the committer who will perform the
final sign-off.
On Wed, Aug 5, 2026 at 12:10 PM Anurag Mantripragada <
anuragmantri93@gmail.com> wrote:
> Hi Jungtaek,
>
> To add some context, this PR is part of the SPIP to introduce new DSv2
> capabilities for connectors supporting column-level updates. It is fully
> self-contained and enables UPDATE statements, allowing connectors to opt in
> and receive only the updated data. This will help connectors like Iceberg
> and Delta fully implement column-level UPDATE statements, which are the
> primary use cases for this feature. I expect just one more JIRA next to
> support MERGE INTO cases, which can target 4.4.
>
> I wanted to bring this to your attention as the release manager since the
> PR might be merged soon. I will defer to the decision made by the reviewer,
> committer, and you (release manager).
>
> Thanks,
> Anurag
>
>
> On Tue, Aug 4, 2026 at 5:38 PM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> Hi Anurag, could you please provide the impact of merging this in Apache
>> Spark 4.3.0?
>>
>> The SPIP only has one JIRA ticket, which doesn't provide the proper
>> visibility of where we are. I'd not encourage people to leave the project
>> to be released as half-baked and trigger "unknown unknown"; does the PR
>> bring up self-contained value? I assume the answer is yes, but let's punt
>> to 4.4.0 if the answer is no.
>>
>> On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada <
>> anuragmantripragada@apache.org> wrote:
>>
>>> Hi Jungteak,
>>>
>>> Would it be possible to include SPARK-58111
>>> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part of
>>> the SPIP SPARK-56599 <https://issues.apache.org/jira/browse/SPARK-56599>)
>>> in the 4.3 release?
>>>
>>> The PR has gone through several review rounds. Including this
>>> foundational PR with UPDATE support in 4.3 would allow projects like Apache
>>> Iceberg to start integrating sooner.
>>>
>>> Please let me know your thoughts.
>>>
>>> ~ Anurag
>>>
>>> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with
>>>> post branch cut tasks.
>>>>
>>>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR
>>>> <https://github.com/apache/spark/pull/57737>)
>>>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to
>>>> master (PR <https://github.com/apache/spark/pull/57738>)
>>>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release
>>>> integration (PR <https://github.com/apache/spark/pull/57741>)
>>>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default
>>>> branch (PR <https://github.com/apache/spark/pull/57740>)
>>>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x
>>>> on branch-4.x (PR <https://github.com/apache/spark/pull/57739>)
>>>>
>>>> Folks, please help reviewing the changes; thanks for your help!
>>>>
>>>>
>>>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <pa...@gmail.com> wrote:
>>>>
>>>>> I just merged https://github.com/apache/spark/pull/57718, and I
>>>>> suppose it fixes the compilation for branch-4.x and branch-4.3
>>>>>
>>>>> Thanks,
>>>>> Cheng Pan
>>>>>
>>>>>
>>>>>
>>>>> On Aug 4, 2026, at 10:58, Jungtaek Lim <ka...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> UPDATE: It takes more than I expect to do post-branch cut, because
>>>>> branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt
>>>>> yet).
>>>>> Maybe I'll need to fix the issue altogether in branch-4.x and
>>>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT.
>>>>>
>>>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC
>>>>>> members who merge the code change to Apache Spark 4.3.0 would need to
>>>>>> include branch-4.3 into the merge branches to account for it. I'm still
>>>>>> making the change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an
>>>>>> update once that is done.
>>>>>>
>>>>>> This is a soft cut and we tend to broadly accept code changes to
>>>>>> branch-4.3 till RC is started, but please refrain from merging the code
>>>>>> change which could make a release at risk of releasing in a month or so.
>>>>>> Dragging a release process to September 4th week would risk the
>>>>>> unavailability of PMC members in East Asia for a couple weeks, so ideally
>>>>>> it would be nice to finalize the RC vote till Sep 3rd week.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> FYI, this is the last reminder before branch cut on Apache Spark
>>>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is no
>>>>>>> reasonable objection. I'll give an update tomorrow once I cut the branch,
>>>>>>> so that folks have visibility of whether the fix is included or not.
>>>>>>>
>>>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <
>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>
>>>>>>>> UPDATE: The website is updated to reflect the plan.
>>>>>>>>
>>>>>>>> https://spark.apache.org/versioning-policy.html
>>>>>>>>
>>>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for
>>>>>>>> Apache Spark 4.3.0*. Since there is no feedback on the plan so
>>>>>>>> far, I'd consider this as no issue in proceeding with the plan.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> SGTM
>>>>>>>>>
>>>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <
>>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Just a head-up, since there is no major feedback, I'll proceed to
>>>>>>>>>> update the release plan as discussed with prior SPIP.
>>>>>>>>>>
>>>>>>>>>> Given the practice of release scheduling, the plan for Apache
>>>>>>>>>> 4.3.0 will be following:
>>>>>>>>>>
>>>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>>>>>>>>> * Mid Aug 2026: QA period
>>>>>>>>>> * Late Aug 2026: RC
>>>>>>>>>>
>>>>>>>>>> I'm going to update the versioning policy page with the above.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
>>>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi dev,
>>>>>>>>>>>
>>>>>>>>>>> I'm happy to serve the role of the release manager for Apache
>>>>>>>>>>> Spark 4.3. Since this is the first release we followed the new policy of 3
>>>>>>>>>>> months cadence, there might be some confusion/concern and I need your help
>>>>>>>>>>> and understanding. (Thanks in advance for your help and patience!)
>>>>>>>>>>>
>>>>>>>>>>> I'm collecting the projects the workstream has been happening on
>>>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP,
>>>>>>>>>>> but please give the head-up for your project which has partially landed to
>>>>>>>>>>> Apache Spark 4.3 and you are close to the finish line.
>>>>>>>>>>>
>>>>>>>>>>> For the projects you are unlikely to complete before branch cut
>>>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
>>>>>>>>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
>>>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
>>>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
>>>>>>>>>>> "be open to postpone the schedule for the project" is a key success factor
>>>>>>>>>>> for faster release. Thanks for your understanding!
>>>>>>>>>>>
>>>>>>>>>>> I'll update the versioning policy page to reflect the expected
>>>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any questions
>>>>>>>>>>> and voices/feedback.
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <
>>>>>>>>>>> gurwls223@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> I would like to send a gentle reminder as heads up that
>>>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare branch-4.3. In the
>>>>>>>>>>>> SPIP about releasing, we decided to do the faster release ones, and this
>>>>>>>>>>>> would be the very first one.
>>>>>>>>>>>>
>>>>>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly
>>>>>>>>>>>> delayed some more. I would like to avoid affecting our first release
>>>>>>>>>>>> schedule - It has been our chronic issue in history, and I do think we
>>>>>>>>>>>> should change this from 4.3 at the first predictable release, and push it
>>>>>>>>>>>> forward. Let's get this done and let the releases be predictable.
>>>>>>>>>>>>
>>>>>>>>>>>> PS: would need to update
>>>>>>>>>>>> https://spark.apache.org/versioning-policy.html for
>>>>>>>>>>>> details about branchcut, RC and releases.
>>>>>>>>>>>>
>>>>>>>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
>>>>>>>>>>>> release manager for 4.3.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Anurag Mantripragada <an...@gmail.com>.
Hi Jungtaek,
To add some context, this PR is part of the SPIP to introduce new DSv2
capabilities for connectors supporting column-level updates. It is fully
self-contained and enables UPDATE statements, allowing connectors to opt in
and receive only the updated data. This will help connectors like Iceberg
and Delta fully implement column-level UPDATE statements, which are the
primary use cases for this feature. I expect just one more JIRA next to
support MERGE INTO cases, which can target 4.4.
I wanted to bring this to your attention as the release manager since the
PR might be merged soon. I will defer to the decision made by the reviewer,
committer, and you (release manager).
Thanks,
Anurag
On Tue, Aug 4, 2026 at 5:38 PM Jungtaek Lim <ka...@gmail.com>
wrote:
> Hi Anurag, could you please provide the impact of merging this in Apache
> Spark 4.3.0?
>
> The SPIP only has one JIRA ticket, which doesn't provide the proper
> visibility of where we are. I'd not encourage people to leave the project
> to be released as half-baked and trigger "unknown unknown"; does the PR
> bring up self-contained value? I assume the answer is yes, but let's punt
> to 4.4.0 if the answer is no.
>
> On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada <
> anuragmantripragada@apache.org> wrote:
>
>> Hi Jungteak,
>>
>> Would it be possible to include SPARK-58111
>> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part of
>> the SPIP SPARK-56599 <https://issues.apache.org/jira/browse/SPARK-56599>)
>> in the 4.3 release?
>>
>> The PR has gone through several review rounds. Including this
>> foundational PR with UPDATE support in 4.3 would allow projects like Apache
>> Iceberg to start integrating sooner.
>>
>> Please let me know your thoughts.
>>
>> ~ Anurag
>>
>> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>
>>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with
>>> post branch cut tasks.
>>>
>>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR
>>> <https://github.com/apache/spark/pull/57737>)
>>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to master
>>> (PR <https://github.com/apache/spark/pull/57738>)
>>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release
>>> integration (PR <https://github.com/apache/spark/pull/57741>)
>>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default
>>> branch (PR <https://github.com/apache/spark/pull/57740>)
>>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x
>>> on branch-4.x (PR <https://github.com/apache/spark/pull/57739>)
>>>
>>> Folks, please help reviewing the changes; thanks for your help!
>>>
>>>
>>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <pa...@gmail.com> wrote:
>>>
>>>> I just merged https://github.com/apache/spark/pull/57718, and I
>>>> suppose it fixes the compilation for branch-4.x and branch-4.3
>>>>
>>>> Thanks,
>>>> Cheng Pan
>>>>
>>>>
>>>>
>>>> On Aug 4, 2026, at 10:58, Jungtaek Lim <ka...@gmail.com>
>>>> wrote:
>>>>
>>>> UPDATE: It takes more than I expect to do post-branch cut, because
>>>> branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt
>>>> yet).
>>>> Maybe I'll need to fix the issue altogether in branch-4.x and
>>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT.
>>>>
>>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC members
>>>>> who merge the code change to Apache Spark 4.3.0 would need to include
>>>>> branch-4.3 into the merge branches to account for it. I'm still making the
>>>>> change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an update once
>>>>> that is done.
>>>>>
>>>>> This is a soft cut and we tend to broadly accept code changes to
>>>>> branch-4.3 till RC is started, but please refrain from merging the code
>>>>> change which could make a release at risk of releasing in a month or so.
>>>>> Dragging a release process to September 4th week would risk the
>>>>> unavailability of PMC members in East Asia for a couple weeks, so ideally
>>>>> it would be nice to finalize the RC vote till Sep 3rd week.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> FYI, this is the last reminder before branch cut on Apache Spark
>>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is no
>>>>>> reasonable objection. I'll give an update tomorrow once I cut the branch,
>>>>>> so that folks have visibility of whether the fix is included or not.
>>>>>>
>>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> UPDATE: The website is updated to reflect the plan.
>>>>>>>
>>>>>>> https://spark.apache.org/versioning-policy.html
>>>>>>>
>>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for
>>>>>>> Apache Spark 4.3.0*. Since there is no feedback on the plan so far,
>>>>>>> I'd consider this as no issue in proceeding with the plan.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> SGTM
>>>>>>>>
>>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <
>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Just a head-up, since there is no major feedback, I'll proceed to
>>>>>>>>> update the release plan as discussed with prior SPIP.
>>>>>>>>>
>>>>>>>>> Given the practice of release scheduling, the plan for Apache
>>>>>>>>> 4.3.0 will be following:
>>>>>>>>>
>>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>>>>>>>> * Mid Aug 2026: QA period
>>>>>>>>> * Late Aug 2026: RC
>>>>>>>>>
>>>>>>>>> I'm going to update the versioning policy page with the above.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
>>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi dev,
>>>>>>>>>>
>>>>>>>>>> I'm happy to serve the role of the release manager for Apache
>>>>>>>>>> Spark 4.3. Since this is the first release we followed the new policy of 3
>>>>>>>>>> months cadence, there might be some confusion/concern and I need your help
>>>>>>>>>> and understanding. (Thanks in advance for your help and patience!)
>>>>>>>>>>
>>>>>>>>>> I'm collecting the projects the workstream has been happening on
>>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP,
>>>>>>>>>> but please give the head-up for your project which has partially landed to
>>>>>>>>>> Apache Spark 4.3 and you are close to the finish line.
>>>>>>>>>>
>>>>>>>>>> For the projects you are unlikely to complete before branch cut
>>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
>>>>>>>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
>>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
>>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
>>>>>>>>>> "be open to postpone the schedule for the project" is a key success factor
>>>>>>>>>> for faster release. Thanks for your understanding!
>>>>>>>>>>
>>>>>>>>>> I'll update the versioning policy page to reflect the expected
>>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any questions
>>>>>>>>>> and voices/feedback.
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <
>>>>>>>>>> gurwls223@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I would like to send a gentle reminder as heads up that
>>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare branch-4.3. In the
>>>>>>>>>>> SPIP about releasing, we decided to do the faster release ones, and this
>>>>>>>>>>> would be the very first one.
>>>>>>>>>>>
>>>>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly
>>>>>>>>>>> delayed some more. I would like to avoid affecting our first release
>>>>>>>>>>> schedule - It has been our chronic issue in history, and I do think we
>>>>>>>>>>> should change this from 4.3 at the first predictable release, and push it
>>>>>>>>>>> forward. Let's get this done and let the releases be predictable.
>>>>>>>>>>>
>>>>>>>>>>> PS: would need to update
>>>>>>>>>>> https://spark.apache.org/versioning-policy.html for
>>>>>>>>>>> details about branchcut, RC and releases.
>>>>>>>>>>>
>>>>>>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
>>>>>>>>>>> release manager for 4.3.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Jungtaek Lim <ka...@gmail.com>.
Hi Anurag, could you please provide the impact of merging this in Apache
Spark 4.3.0?
The SPIP only has one JIRA ticket, which doesn't provide the proper
visibility of where we are. I'd not encourage people to leave the project
to be released as half-baked and trigger "unknown unknown"; does the PR
bring up self-contained value? I assume the answer is yes, but let's punt
to 4.4.0 if the answer is no.
On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada <
anuragmantripragada@apache.org> wrote:
> Hi Jungteak,
>
> Would it be possible to include SPARK-58111
> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part of the
> SPIP SPARK-56599 <https://issues.apache.org/jira/browse/SPARK-56599>) in
> the 4.3 release?
>
> The PR has gone through several review rounds. Including this foundational
> PR with UPDATE support in 4.3 would allow projects like Apache Iceberg to
> start integrating sooner.
>
> Please let me know your thoughts.
>
> ~ Anurag
>
> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with
>> post branch cut tasks.
>>
>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR
>> <https://github.com/apache/spark/pull/57737>)
>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to master (
>> PR <https://github.com/apache/spark/pull/57738>)
>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release
>> integration (PR <https://github.com/apache/spark/pull/57741>)
>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default
>> branch (PR <https://github.com/apache/spark/pull/57740>)
>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x on
>> branch-4.x (PR <https://github.com/apache/spark/pull/57739>)
>>
>> Folks, please help reviewing the changes; thanks for your help!
>>
>>
>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <pa...@gmail.com> wrote:
>>
>>> I just merged https://github.com/apache/spark/pull/57718, and I suppose
>>> it fixes the compilation for branch-4.x and branch-4.3
>>>
>>> Thanks,
>>> Cheng Pan
>>>
>>>
>>>
>>> On Aug 4, 2026, at 10:58, Jungtaek Lim <ka...@gmail.com>
>>> wrote:
>>>
>>> UPDATE: It takes more than I expect to do post-branch cut, because
>>> branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt
>>> yet).
>>> Maybe I'll need to fix the issue altogether in branch-4.x and
>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT.
>>>
>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC members
>>>> who merge the code change to Apache Spark 4.3.0 would need to include
>>>> branch-4.3 into the merge branches to account for it. I'm still making the
>>>> change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an update once
>>>> that is done.
>>>>
>>>> This is a soft cut and we tend to broadly accept code changes to
>>>> branch-4.3 till RC is started, but please refrain from merging the code
>>>> change which could make a release at risk of releasing in a month or so.
>>>> Dragging a release process to September 4th week would risk the
>>>> unavailability of PMC members in East Asia for a couple weeks, so ideally
>>>> it would be nice to finalize the RC vote till Sep 3rd week.
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> FYI, this is the last reminder before branch cut on Apache Spark
>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is no
>>>>> reasonable objection. I'll give an update tomorrow once I cut the branch,
>>>>> so that folks have visibility of whether the fix is included or not.
>>>>>
>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> UPDATE: The website is updated to reflect the plan.
>>>>>>
>>>>>> https://spark.apache.org/versioning-policy.html
>>>>>>
>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for Apache
>>>>>> Spark 4.3.0*. Since there is no feedback on the plan so far, I'd
>>>>>> consider this as no issue in proceeding with the plan.
>>>>>>
>>>>>>
>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> SGTM
>>>>>>>
>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <
>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>
>>>>>>>> Just a head-up, since there is no major feedback, I'll proceed to
>>>>>>>> update the release plan as discussed with prior SPIP.
>>>>>>>>
>>>>>>>> Given the practice of release scheduling, the plan for Apache 4.3.0
>>>>>>>> will be following:
>>>>>>>>
>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>>>>>>> * Mid Aug 2026: QA period
>>>>>>>> * Late Aug 2026: RC
>>>>>>>>
>>>>>>>> I'm going to update the versioning policy page with the above.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
>>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi dev,
>>>>>>>>>
>>>>>>>>> I'm happy to serve the role of the release manager for Apache
>>>>>>>>> Spark 4.3. Since this is the first release we followed the new policy of 3
>>>>>>>>> months cadence, there might be some confusion/concern and I need your help
>>>>>>>>> and understanding. (Thanks in advance for your help and patience!)
>>>>>>>>>
>>>>>>>>> I'm collecting the projects the workstream has been happening on
>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP,
>>>>>>>>> but please give the head-up for your project which has partially landed to
>>>>>>>>> Apache Spark 4.3 and you are close to the finish line.
>>>>>>>>>
>>>>>>>>> For the projects you are unlikely to complete before branch cut
>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
>>>>>>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
>>>>>>>>> "be open to postpone the schedule for the project" is a key success factor
>>>>>>>>> for faster release. Thanks for your understanding!
>>>>>>>>>
>>>>>>>>> I'll update the versioning policy page to reflect the expected
>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any questions
>>>>>>>>> and voices/feedback.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>>
>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <
>>>>>>>>> gurwls223@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I would like to send a gentle reminder as heads up that
>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare branch-4.3. In the
>>>>>>>>>> SPIP about releasing, we decided to do the faster release ones, and this
>>>>>>>>>> would be the very first one.
>>>>>>>>>>
>>>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly
>>>>>>>>>> delayed some more. I would like to avoid affecting our first release
>>>>>>>>>> schedule - It has been our chronic issue in history, and I do think we
>>>>>>>>>> should change this from 4.3 at the first predictable release, and push it
>>>>>>>>>> forward. Let's get this done and let the releases be predictable.
>>>>>>>>>>
>>>>>>>>>> PS: would need to update
>>>>>>>>>> https://spark.apache.org/versioning-policy.html for
>>>>>>>>>> details about branchcut, RC and releases.
>>>>>>>>>>
>>>>>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
>>>>>>>>>> release manager for 4.3.
>>>>>>>>>>
>>>>>>>>>>
>>>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Anurag Mantripragada <an...@apache.org>.
Hi Jungteak,
Would it be possible to include SPARK-58111
<https://issues.apache.org/jira/browse/SPARK-58111> (Which is part of the
SPIP SPARK-56599 <https://issues.apache.org/jira/browse/SPARK-56599>) in
the 4.3 release?
The PR has gone through several review rounds. Including this foundational
PR with UPDATE support in 4.3 would allow projects like Apache Iceberg to
start integrating sooner.
Please let me know your thoughts.
~ Anurag
On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim <ka...@gmail.com>
wrote:
> Thanks Cheng! I'm unblocked now and have several PRs to proceed with
> post branch cut tasks.
>
> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR
> <https://github.com/apache/spark/pull/57737>)
> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to master (
> PR <https://github.com/apache/spark/pull/57738>)
> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release integration
> (PR <https://github.com/apache/spark/pull/57741>)
> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default
> branch (PR <https://github.com/apache/spark/pull/57740>)
> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x on
> branch-4.x (PR <https://github.com/apache/spark/pull/57739>)
>
> Folks, please help reviewing the changes; thanks for your help!
>
>
> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <pa...@gmail.com> wrote:
>
>> I just merged https://github.com/apache/spark/pull/57718, and I suppose
>> it fixes the compilation for branch-4.x and branch-4.3
>>
>> Thanks,
>> Cheng Pan
>>
>>
>>
>> On Aug 4, 2026, at 10:58, Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>
>> UPDATE: It takes more than I expect to do post-branch cut, because
>> branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt
>> yet).
>> Maybe I'll need to fix the issue altogether in branch-4.x and branch-4.3,
>> and then bump branch-4.x to 4.4.0-SNAPSHOT.
>>
>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <
>> kabhwan.opensource@gmail.com> wrote:
>>
>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC members
>>> who merge the code change to Apache Spark 4.3.0 would need to include
>>> branch-4.3 into the merge branches to account for it. I'm still making the
>>> change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an update once
>>> that is done.
>>>
>>> This is a soft cut and we tend to broadly accept code changes to
>>> branch-4.3 till RC is started, but please refrain from merging the code
>>> change which could make a release at risk of releasing in a month or so.
>>> Dragging a release process to September 4th week would risk the
>>> unavailability of PMC members in East Asia for a couple weeks, so ideally
>>> it would be nice to finalize the RC vote till Sep 3rd week.
>>>
>>> Thanks!
>>>
>>>
>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> FYI, this is the last reminder before branch cut on Apache Spark 4.3.0.
>>>> I'll cut the branch tomorrow (Aug 3 in US tz) if there is no reasonable
>>>> objection. I'll give an update tomorrow once I cut the branch, so that
>>>> folks have visibility of whether the fix is included or not.
>>>>
>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> UPDATE: The website is updated to reflect the plan.
>>>>>
>>>>> https://spark.apache.org/versioning-policy.html
>>>>>
>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for Apache
>>>>> Spark 4.3.0*. Since there is no feedback on the plan so far, I'd
>>>>> consider this as no issue in proceeding with the plan.
>>>>>
>>>>>
>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> SGTM
>>>>>>
>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> Just a head-up, since there is no major feedback, I'll proceed to
>>>>>>> update the release plan as discussed with prior SPIP.
>>>>>>>
>>>>>>> Given the practice of release scheduling, the plan for Apache 4.3.0
>>>>>>> will be following:
>>>>>>>
>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>>>>>> * Mid Aug 2026: QA period
>>>>>>> * Late Aug 2026: RC
>>>>>>>
>>>>>>> I'm going to update the versioning policy page with the above.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
>>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi dev,
>>>>>>>>
>>>>>>>> I'm happy to serve the role of the release manager for Apache Spark
>>>>>>>> 4.3. Since this is the first release we followed the new policy of 3 months
>>>>>>>> cadence, there might be some confusion/concern and I need your help and
>>>>>>>> understanding. (Thanks in advance for your help and patience!)
>>>>>>>>
>>>>>>>> I'm collecting the projects the workstream has been happening on
>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP,
>>>>>>>> but please give the head-up for your project which has partially landed to
>>>>>>>> Apache Spark 4.3 and you are close to the finish line.
>>>>>>>>
>>>>>>>> For the projects you are unlikely to complete before branch cut
>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
>>>>>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
>>>>>>>> "be open to postpone the schedule for the project" is a key success factor
>>>>>>>> for faster release. Thanks for your understanding!
>>>>>>>>
>>>>>>>> I'll update the versioning policy page to reflect the expected
>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any questions
>>>>>>>> and voices/feedback.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>>
>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <gu...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I would like to send a gentle reminder as heads up that
>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare branch-4.3. In the
>>>>>>>>> SPIP about releasing, we decided to do the faster release ones, and this
>>>>>>>>> would be the very first one.
>>>>>>>>>
>>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly
>>>>>>>>> delayed some more. I would like to avoid affecting our first release
>>>>>>>>> schedule - It has been our chronic issue in history, and I do think we
>>>>>>>>> should change this from 4.3 at the first predictable release, and push it
>>>>>>>>> forward. Let's get this done and let the releases be predictable.
>>>>>>>>>
>>>>>>>>> PS: would need to update
>>>>>>>>> https://spark.apache.org/versioning-policy.html for details about
>>>>>>>>> branchcut, RC and releases.
>>>>>>>>>
>>>>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
>>>>>>>>> release manager for 4.3.
>>>>>>>>>
>>>>>>>>>
>>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Jungtaek Lim <ka...@gmail.com>.
Thanks Cheng! I'm unblocked now and have several PRs to proceed with
post branch cut tasks.
* [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR
<https://github.com/apache/spark/pull/57737>)
* [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to master (PR
<https://github.com/apache/spark/pull/57738>)
* [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release integration (
PR <https://github.com/apache/spark/pull/57741>)
* [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default
branch (PR <https://github.com/apache/spark/pull/57740>)
* [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x on
branch-4.x (PR <https://github.com/apache/spark/pull/57739>)
Folks, please help reviewing the changes; thanks for your help!
On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <pa...@gmail.com> wrote:
> I just merged https://github.com/apache/spark/pull/57718, and I suppose
> it fixes the compilation for branch-4.x and branch-4.3
>
> Thanks,
> Cheng Pan
>
>
>
> On Aug 4, 2026, at 10:58, Jungtaek Lim <ka...@gmail.com>
> wrote:
>
> UPDATE: It takes more than I expect to do post-branch cut, because
> branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt
> yet).
> Maybe I'll need to fix the issue altogether in branch-4.x and branch-4.3,
> and then bump branch-4.x to 4.4.0-SNAPSHOT.
>
> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC members
>> who merge the code change to Apache Spark 4.3.0 would need to include
>> branch-4.3 into the merge branches to account for it. I'm still making the
>> change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an update once
>> that is done.
>>
>> This is a soft cut and we tend to broadly accept code changes to
>> branch-4.3 till RC is started, but please refrain from merging the code
>> change which could make a release at risk of releasing in a month or so.
>> Dragging a release process to September 4th week would risk the
>> unavailability of PMC members in East Asia for a couple weeks, so ideally
>> it would be nice to finalize the RC vote till Sep 3rd week.
>>
>> Thanks!
>>
>>
>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <ka...@gmail.com>
>> wrote:
>>
>>> FYI, this is the last reminder before branch cut on Apache Spark 4.3.0.
>>> I'll cut the branch tomorrow (Aug 3 in US tz) if there is no reasonable
>>> objection. I'll give an update tomorrow once I cut the branch, so that
>>> folks have visibility of whether the fix is included or not.
>>>
>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <
>>> kabhwan.opensource@gmail.com> wrote:
>>>
>>>> UPDATE: The website is updated to reflect the plan.
>>>>
>>>> https://spark.apache.org/versioning-policy.html
>>>>
>>>> As a reminder, *Aug 1 is the rough date to cut the branch for Apache
>>>> Spark 4.3.0*. Since there is no feedback on the plan so far, I'd
>>>> consider this as no issue in proceeding with the plan.
>>>>
>>>>
>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org>
>>>> wrote:
>>>>
>>>>> SGTM
>>>>>
>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> Just a head-up, since there is no major feedback, I'll proceed to
>>>>>> update the release plan as discussed with prior SPIP.
>>>>>>
>>>>>> Given the practice of release scheduling, the plan for Apache 4.3.0
>>>>>> will be following:
>>>>>>
>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>>>>> * Mid Aug 2026: QA period
>>>>>> * Late Aug 2026: RC
>>>>>>
>>>>>> I'm going to update the versioning policy page with the above.
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
>>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>>
>>>>>>> Hi dev,
>>>>>>>
>>>>>>> I'm happy to serve the role of the release manager for Apache Spark
>>>>>>> 4.3. Since this is the first release we followed the new policy of 3 months
>>>>>>> cadence, there might be some confusion/concern and I need your help and
>>>>>>> understanding. (Thanks in advance for your help and patience!)
>>>>>>>
>>>>>>> I'm collecting the projects the workstream has been happening on
>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP,
>>>>>>> but please give the head-up for your project which has partially landed to
>>>>>>> Apache Spark 4.3 and you are close to the finish line.
>>>>>>>
>>>>>>> For the projects you are unlikely to complete before branch cut
>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
>>>>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
>>>>>>> "be open to postpone the schedule for the project" is a key success factor
>>>>>>> for faster release. Thanks for your understanding!
>>>>>>>
>>>>>>> I'll update the versioning policy page to reflect the expected
>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any questions
>>>>>>> and voices/feedback.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>
>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <gu...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I would like to send a gentle reminder as heads up that we're going
>>>>>>>> to cut the branch early Aug, 1st to prepare branch-4.3. In the SPIP about
>>>>>>>> releasing, we decided to do the faster release ones, and this would be the
>>>>>>>> very first one.
>>>>>>>>
>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly
>>>>>>>> delayed some more. I would like to avoid affecting our first release
>>>>>>>> schedule - It has been our chronic issue in history, and I do think we
>>>>>>>> should change this from 4.3 at the first predictable release, and push it
>>>>>>>> forward. Let's get this done and let the releases be predictable.
>>>>>>>>
>>>>>>>> PS: would need to update
>>>>>>>> https://spark.apache.org/versioning-policy.html for details about
>>>>>>>> branchcut, RC and releases.
>>>>>>>>
>>>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
>>>>>>>> release manager for 4.3.
>>>>>>>>
>>>>>>>>
>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Cheng Pan <pa...@gmail.com>.
I just merged https://github.com/apache/spark/pull/57718, and I suppose it fixes the compilation for branch-4.x and branch-4.3
Thanks,
Cheng Pan
> On Aug 4, 2026, at 10:58, Jungtaek Lim <ka...@gmail.com> wrote:
>
> UPDATE: It takes more than I expect to do post-branch cut, because branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt yet).
> Maybe I'll need to fix the issue altogether in branch-4.x and branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT.
>
> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <kabhwan.opensource@gmail.com <ma...@gmail.com>> wrote:
>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC members who merge the code change to Apache Spark 4.3.0 would need to include branch-4.3 into the merge branches to account for it. I'm still making the change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an update once that is done.
>>
>> This is a soft cut and we tend to broadly accept code changes to branch-4.3 till RC is started, but please refrain from merging the code change which could make a release at risk of releasing in a month or so. Dragging a release process to September 4th week would risk the unavailability of PMC members in East Asia for a couple weeks, so ideally it would be nice to finalize the RC vote till Sep 3rd week.
>>
>> Thanks!
>>
>>
>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <kabhwan.opensource@gmail.com <ma...@gmail.com>> wrote:
>>> FYI, this is the last reminder before branch cut on Apache Spark 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is no reasonable objection. I'll give an update tomorrow once I cut the branch, so that folks have visibility of whether the fix is included or not.
>>>
>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <kabhwan.opensource@gmail.com <ma...@gmail.com>> wrote:
>>>> UPDATE: The website is updated to reflect the plan.
>>>>
>>>> https://spark.apache.org/versioning-policy.html
>>>>
>>>> As a reminder, Aug 1 is the rough date to cut the branch for Apache Spark 4.3.0. Since there is no feedback on the plan so far, I'd consider this as no issue in proceeding with the plan.
>>>>
>>>>
>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gurwls223@apache.org <ma...@apache.org>> wrote:
>>>>> SGTM
>>>>>
>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <kabhwan.opensource@gmail.com <ma...@gmail.com>> wrote:
>>>>>> Just a head-up, since there is no major feedback, I'll proceed to update the release plan as discussed with prior SPIP.
>>>>>>
>>>>>> Given the practice of release scheduling, the plan for Apache 4.3.0 will be following:
>>>>>>
>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>>>>> * Mid Aug 2026: QA period
>>>>>> * Late Aug 2026: RC
>>>>>>
>>>>>> I'm going to update the versioning policy page with the above.
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <kabhwan.opensource@gmail.com <ma...@gmail.com>> wrote:
>>>>>>> Hi dev,
>>>>>>>
>>>>>>> I'm happy to serve the role of the release manager for Apache Spark 4.3. Since this is the first release we followed the new policy of 3 months cadence, there might be some confusion/concern and I need your help and understanding. (Thanks in advance for your help and patience!)
>>>>>>>
>>>>>>> I'm collecting the projects the workstream has been happening on Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP, but please give the head-up for your project which has partially landed to Apache Spark 4.3 and you are close to the finish line.
>>>>>>>
>>>>>>> For the projects you are unlikely to complete before branch cut (loosely plus 1-2 weeks after branch cut), you are encouraged to have a flag to disable the partial change to take effect in Apache Spark 4.3, and postpone the finish line to Apache Spark 4.4. The delay of missing Apache Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and "be open to postpone the schedule for the project" is a key success factor for faster release. Thanks for your understanding!
>>>>>>>
>>>>>>> I'll update the versioning policy page to reflect the expected schedule of Apache Spark 4.3.0 release. Please post here for any questions and voices/feedback.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>>
>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <gurwls223@apache.org <ma...@apache.org>> wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I would like to send a gentle reminder as heads up that we're going to cut the branch early Aug, 1st to prepare branch-4.3. In the SPIP about releasing, we decided to do the faster release ones, and this would be the very first one.
>>>>>>>>
>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly delayed some more. I would like to avoid affecting our first release schedule - It has been our chronic issue in history, and I do think we should change this from 4.3 at the first predictable release, and push it forward. Let's get this done and let the releases be predictable.
>>>>>>>>
>>>>>>>> PS: would need to update https://spark.apache.org/versioning-policy.html for details about branchcut, RC and releases.
>>>>>>>>
>>>>>>>> Cc'ing @Jungtaek Lim <ma...@gmail.com> who's going to be the release manager for 4.3.
>>>>>>>>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Jungtaek Lim <ka...@gmail.com>.
UPDATE: It takes more than I expect to do post-branch cut, because
branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt
yet).
Maybe I'll need to fix the issue altogether in branch-4.x and branch-4.3,
and then bump branch-4.x to 4.4.0-SNAPSHOT.
On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim <ka...@gmail.com>
wrote:
> UPDATE: the new branch "branch-4.3" is created. Committers/PMC members who
> merge the code change to Apache Spark 4.3.0 would need to include
> branch-4.3 into the merge branches to account for it. I'm still making the
> change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an update once
> that is done.
>
> This is a soft cut and we tend to broadly accept code changes to
> branch-4.3 till RC is started, but please refrain from merging the code
> change which could make a release at risk of releasing in a month or so.
> Dragging a release process to September 4th week would risk the
> unavailability of PMC members in East Asia for a couple weeks, so ideally
> it would be nice to finalize the RC vote till Sep 3rd week.
>
> Thanks!
>
>
> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> FYI, this is the last reminder before branch cut on Apache Spark 4.3.0.
>> I'll cut the branch tomorrow (Aug 3 in US tz) if there is no reasonable
>> objection. I'll give an update tomorrow once I cut the branch, so that
>> folks have visibility of whether the fix is included or not.
>>
>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <
>> kabhwan.opensource@gmail.com> wrote:
>>
>>> UPDATE: The website is updated to reflect the plan.
>>>
>>> https://spark.apache.org/versioning-policy.html
>>>
>>> As a reminder, *Aug 1 is the rough date to cut the branch for Apache
>>> Spark 4.3.0*. Since there is no feedback on the plan so far, I'd
>>> consider this as no issue in proceeding with the plan.
>>>
>>>
>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org>
>>> wrote:
>>>
>>>> SGTM
>>>>
>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> Just a head-up, since there is no major feedback, I'll proceed to
>>>>> update the release plan as discussed with prior SPIP.
>>>>>
>>>>> Given the practice of release scheduling, the plan for Apache 4.3.0
>>>>> will be following:
>>>>>
>>>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>>>> * Mid Aug 2026: QA period
>>>>> * Late Aug 2026: RC
>>>>>
>>>>> I'm going to update the versioning policy page with the above.
>>>>>
>>>>>
>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
>>>>> kabhwan.opensource@gmail.com> wrote:
>>>>>
>>>>>> Hi dev,
>>>>>>
>>>>>> I'm happy to serve the role of the release manager for Apache Spark
>>>>>> 4.3. Since this is the first release we followed the new policy of 3 months
>>>>>> cadence, there might be some confusion/concern and I need your help and
>>>>>> understanding. (Thanks in advance for your help and patience!)
>>>>>>
>>>>>> I'm collecting the projects the workstream has been happening on
>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP,
>>>>>> but please give the head-up for your project which has partially landed to
>>>>>> Apache Spark 4.3 and you are close to the finish line.
>>>>>>
>>>>>> For the projects you are unlikely to complete before branch cut
>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
>>>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
>>>>>> "be open to postpone the schedule for the project" is a key success factor
>>>>>> for faster release. Thanks for your understanding!
>>>>>>
>>>>>> I'll update the versioning policy page to reflect the expected
>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any questions
>>>>>> and voices/feedback.
>>>>>>
>>>>>> Thanks!
>>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>>
>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <gu...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I would like to send a gentle reminder as heads up that we're going
>>>>>>> to cut the branch early Aug, 1st to prepare branch-4.3. In the SPIP about
>>>>>>> releasing, we decided to do the faster release ones, and this would be the
>>>>>>> very first one.
>>>>>>>
>>>>>>> I know that we just released Spark 4.2. That was unexpectedly
>>>>>>> delayed some more. I would like to avoid affecting our first release
>>>>>>> schedule - It has been our chronic issue in history, and I do think we
>>>>>>> should change this from 4.3 at the first predictable release, and push it
>>>>>>> forward. Let's get this done and let the releases be predictable.
>>>>>>>
>>>>>>> PS: would need to update
>>>>>>> https://spark.apache.org/versioning-policy.html for details about
>>>>>>> branchcut, RC and releases.
>>>>>>>
>>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
>>>>>>> release manager for 4.3.
>>>>>>>
>>>>>>>
Re: [PSA] Preparing 4.3 branch cut 1st Aug.
Posted by Jungtaek Lim <ka...@gmail.com>.
UPDATE: the new branch "branch-4.3" is created. Committers/PMC members who
merge the code change to Apache Spark 4.3.0 would need to include
branch-4.3 into the merge branches to account for it. I'm still making the
change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an update once
that is done.
This is a soft cut and we tend to broadly accept code changes to branch-4.3
till RC is started, but please refrain from merging the code change which
could make a release at risk of releasing in a month or so. Dragging a
release process to September 4th week would risk the unavailability of PMC
members in East Asia for a couple weeks, so ideally it would be nice to
finalize the RC vote till Sep 3rd week.
Thanks!
On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim <ka...@gmail.com>
wrote:
> FYI, this is the last reminder before branch cut on Apache Spark 4.3.0.
> I'll cut the branch tomorrow (Aug 3 in US tz) if there is no reasonable
> objection. I'll give an update tomorrow once I cut the branch, so that
> folks have visibility of whether the fix is included or not.
>
> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> UPDATE: The website is updated to reflect the plan.
>>
>> https://spark.apache.org/versioning-policy.html
>>
>> As a reminder, *Aug 1 is the rough date to cut the branch for Apache
>> Spark 4.3.0*. Since there is no feedback on the plan so far, I'd
>> consider this as no issue in proceeding with the plan.
>>
>>
>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <gu...@apache.org>
>> wrote:
>>
>>> SGTM
>>>
>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim <ka...@gmail.com>
>>> wrote:
>>>
>>>> Just a head-up, since there is no major feedback, I'll proceed to
>>>> update the release plan as discussed with prior SPIP.
>>>>
>>>> Given the practice of release scheduling, the plan for Apache 4.3.0
>>>> will be following:
>>>>
>>>> * Aug 1st 2026: Code Freeze. Branch Cut
>>>> * Mid Aug 2026: QA period
>>>> * Late Aug 2026: RC
>>>>
>>>> I'm going to update the versioning policy page with the above.
>>>>
>>>>
>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>>
>>>>> Hi dev,
>>>>>
>>>>> I'm happy to serve the role of the release manager for Apache Spark
>>>>> 4.3. Since this is the first release we followed the new policy of 3 months
>>>>> cadence, there might be some confusion/concern and I need your help and
>>>>> understanding. (Thanks in advance for your help and patience!)
>>>>>
>>>>> I'm collecting the projects the workstream has been happening on
>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on SPIP,
>>>>> but please give the head-up for your project which has partially landed to
>>>>> Apache Spark 4.3 and you are close to the finish line.
>>>>>
>>>>> For the projects you are unlikely to complete before branch cut
>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to have a
>>>>> flag to disable the partial change to take effect in Apache Spark 4.3, and
>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing Apache
>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release cadence, and
>>>>> "be open to postpone the schedule for the project" is a key success factor
>>>>> for faster release. Thanks for your understanding!
>>>>>
>>>>> I'll update the versioning policy page to reflect the expected
>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any questions
>>>>> and voices/feedback.
>>>>>
>>>>> Thanks!
>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>
>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon <gu...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I would like to send a gentle reminder as heads up that we're going
>>>>>> to cut the branch early Aug, 1st to prepare branch-4.3. In the SPIP about
>>>>>> releasing, we decided to do the faster release ones, and this would be the
>>>>>> very first one.
>>>>>>
>>>>>> I know that we just released Spark 4.2. That was unexpectedly
>>>>>> delayed some more. I would like to avoid affecting our first release
>>>>>> schedule - It has been our chronic issue in history, and I do think we
>>>>>> should change this from 4.3 at the first predictable release, and push it
>>>>>> forward. Let's get this done and let the releases be predictable.
>>>>>>
>>>>>> PS: would need to update
>>>>>> https://spark.apache.org/versioning-policy.html for details about
>>>>>> branchcut, RC and releases.
>>>>>>
>>>>>> Cc'ing @Jungtaek Lim <ka...@gmail.com> who's going to be the
>>>>>> release manager for 4.3.
>>>>>>
>>>>>>