You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Holden Karau <ho...@pigscanfly.ca> on 2022/02/02 01:11:52 UTC
[VOTE] Spark 3.1.3 RC3
Please vote on releasing the following candidate as Apache Spark version
3.1.3.
The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and passes
if a majority
+1 PMC votes are cast, with a minimum of 3 + 1 votes.
[ ] +1 Release this package as Apache Spark 3.1.3
[ ] -1 Do not release this package because ...
To learn more about Apache Spark, please see http://spark.apache.org/
There are currently no open issues targeting 3.1.3 in Spark's JIRA
https://issues.apache.org/jira/browse
(try project = SPARK AND "Target Version/s" = "3.1.3" AND status in (Open,
Reopened, "In Progress"))
at https://s.apache.org/n79dw
The tag to be voted on is v3.2.1-rc1 (commit
b8c0799a8cef22c56132d94033759c9f82b0cc86):
https://github.com/apache/spark/tree/v3.1.3-rc3
The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS
The staging repository for this release can be found at
:https://repository.apache.org/content/repositories/orgapachespark-1400/
The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
The list of bug fixes going into 3.1.3 can be found at the following URL:
https://s.apache.org/x0q9b
This release is using the release script in master as
of ddc77fb906cb3ce1567d277c2d0850104c89ac25
The release docker container was rebuilt since the previous version didn't
have the necessary components to build the R documentation.
FAQ
=========================
How can I help test this release?
=========================
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.
If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with an out of date RC going forward).
===========================================
What should happen to JIRA tickets still targeting 3.1.3?
===========================================
The current list of open tickets targeted at 3.2.1 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target
Version/s" = 3.1.3
Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.
==================
But my bug isn't fixed?
==================
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something that is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.
==================
What happened to RC1 & RC2?
==================
When I first went to build RC1 the build process failed due to the
lack of the R markdown package in my local rm container. By the time
I had time to debug and rebuild there was already another bug fix commit in
branch-3.1 so I decided to skip ahead to RC2 and pick it up directly.
When I went to go send the RC2 vote e-mail I noticed a correctness issue had
been fixed in branch-3.1 so I rolled RC3 to contain the correctness fix.
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [VOTE] Spark 3.1.3 RC3
Posted by Sean Owen <sr...@gmail.com>.
+1 from me, same result as the last release on my end.
I think releasing 3.1.3 is fine, it's 7 months since 3.1.2.
On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 3.1.3.
>
> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and passes
> if a majority
> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.1.3
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> There are currently no open issues targeting 3.1.3 in Spark's JIRA
> https://issues.apache.org/jira/browse
> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in (Open,
> Reopened, "In Progress"))
> at https://s.apache.org/n79dw
>
>
>
> The tag to be voted on is v3.2.1-rc1 (commit
> b8c0799a8cef22c56132d94033759c9f82b0cc86):
> https://github.com/apache/spark/tree/v3.1.3-rc3
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at
> :https://repository.apache.org/content/repositories/orgapachespark-1400/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
>
> The list of bug fixes going into 3.1.3 can be found at the following URL:
> https://s.apache.org/x0q9b
>
> This release is using the release script in master as
> of ddc77fb906cb3ce1567d277c2d0850104c89ac25
> The release docker container was rebuilt since the previous version didn't
> have the necessary components to build the R documentation.
>
> FAQ
>
>
> =========================
> How can I help test this release?
> =========================
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===========================================
> What should happen to JIRA tickets still targeting 3.1.3?
> ===========================================
>
> The current list of open tickets targeted at 3.2.1 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.1.3
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==================
> But my bug isn't fixed?
> ==================
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something that is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> ==================
> What happened to RC1 & RC2?
> ==================
>
> When I first went to build RC1 the build process failed due to the
> lack of the R markdown package in my local rm container. By the time
> I had time to debug and rebuild there was already another bug fix commit in
> branch-3.1 so I decided to skip ahead to RC2 and pick it up directly.
> When I went to go send the RC2 vote e-mail I noticed a correctness issue
> had
> been fixed in branch-3.1 so I rolled RC3 to contain the correctness fix.
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
Re: [VOTE] Spark 3.1.3 RC3
Posted by Holden Karau <ho...@pigscanfly.ca>.
Yup, I’ve run into some weirdness with docs again I want to verify before I
send the vote email though.
On Mon, Feb 7, 2022 at 10:06 PM Wenchen Fan <cl...@gmail.com> wrote:
> Shall we use the release scripts of branch 3.1 to release 3.1?
>
> On Fri, Feb 4, 2022 at 4:57 AM Holden Karau <ho...@pigscanfly.ca> wrote:
>
>> Good catch Dongjoon :)
>>
>> This release candidate fails, but feel free to keep testing for any other
>> potential blockers.
>>
>> I’ll roll RC4 next week with the older release scripts (but the more
>> modern image since the legacy image didn’t have a good time with the R doc
>> packaging).
>>
>> On Thu, Feb 3, 2022 at 3:53 PM Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> Unfortunately, -1 for 3.1.3 RC3 due to the packaging issue.
>>>
>>> It seems that the master branch release script didn't work properly for
>>> Hadoop 2 binary distribution, Holden.
>>>
>>> $ curl -s
>>> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/spark-3.1.3-bin-hadoop2.tgz
>>> | tar tz | grep hadoop-common
>>> spark-3.1.3-bin-hadoop2/jars/hadoop-common-3.2.0.jar
>>>
>>> Apache Spark didn't drop Apache Hadoop 2 based binary distribution yet.
>>>
>>> Dongjoon
>>>
>>>
>>> On Wed, Feb 2, 2022 at 3:38 PM Mridul Muralidharan <mr...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Minor nit: the tag mentioned under [1] looks like a typo - I used
>>>> "v3.1.3-rc3" for my vote (3.2.1 is mentioned in a couple of places, treat
>>>> them as 3.1.3 instead)
>>>>
>>>> +1
>>>> Signatures, digests, etc check out fine.
>>>> Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes
>>>>
>>>> Regards,
>>>> Mridul
>>>>
>>>> [1] "The tag to be voted on is v3.2.1-rc1" - the commit hash and git
>>>> url are correct.
>>>>
>>>>
>>>> On Wed, Feb 2, 2022 at 9:30 AM Mridul Muralidharan <mr...@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> Thanks Tom !
>>>>> I missed [1] (or probably forgot) the 3.1 part of the discussion given
>>>>> it centered around 3.2 ...
>>>>>
>>>>>
>>>>> Regards,
>>>>> Mridul
>>>>>
>>>>> [1] https://www.mail-archive.com/dev@spark.apache.org/msg28484.html
>>>>>
>>>>> On Wed, Feb 2, 2022 at 8:55 AM Thomas Graves <tg...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> It was discussed doing all the maintenance lines back at beginning of
>>>>>> December (Dec 6) when we were talking about release 3.2.1.
>>>>>>
>>>>>> Tom
>>>>>>
>>>>>> On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mr...@gmail.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > Hi Holden,
>>>>>> >
>>>>>> > Not that I am against releasing 3.1.3 (given the fixes that have
>>>>>> already gone in), but did we discuss releasing it ? I might have missed the
>>>>>> thread ...
>>>>>> >
>>>>>> > Regards,
>>>>>> > Mridul
>>>>>> >
>>>>>> > On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> Please vote on releasing the following candidate as Apache Spark
>>>>>> version 3.1.3.
>>>>>> >>
>>>>>> >> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and
>>>>>> passes if a majority
>>>>>> >> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
>>>>>> >>
>>>>>> >> [ ] +1 Release this package as Apache Spark 3.1.3
>>>>>> >> [ ] -1 Do not release this package because ...
>>>>>> >>
>>>>>> >> To learn more about Apache Spark, please see
>>>>>> http://spark.apache.org/
>>>>>> >>
>>>>>> >> There are currently no open issues targeting 3.1.3 in Spark's JIRA
>>>>>> https://issues.apache.org/jira/browse
>>>>>> >> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status
>>>>>> in (Open, Reopened, "In Progress"))
>>>>>> >> at https://s.apache.org/n79dw
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> The tag to be voted on is v3.2.1-rc1 (commit
>>>>>> >> b8c0799a8cef22c56132d94033759c9f82b0cc86):
>>>>>> >> https://github.com/apache/spark/tree/v3.1.3-rc3
>>>>>> >>
>>>>>> >> The release files, including signatures, digests, etc. can be
>>>>>> found at:
>>>>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
>>>>>> >>
>>>>>> >> Signatures used for Spark RCs can be found in this file:
>>>>>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>> >>
>>>>>> >> The staging repository for this release can be found at
>>>>>> >> :
>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1400/
>>>>>> >>
>>>>>> >> The documentation corresponding to this release can be found at:
>>>>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
>>>>>> >>
>>>>>> >> The list of bug fixes going into 3.1.3 can be found at the
>>>>>> following URL:
>>>>>> >> https://s.apache.org/x0q9b
>>>>>> >>
>>>>>> >> This release is using the release script in master as of
>>>>>> ddc77fb906cb3ce1567d277c2d0850104c89ac25
>>>>>> >> The release docker container was rebuilt since the previous
>>>>>> version didn't have the necessary components to build the R documentation.
>>>>>> >>
>>>>>> >> FAQ
>>>>>> >>
>>>>>> >>
>>>>>> >> =========================
>>>>>> >> How can I help test this release?
>>>>>> >> =========================
>>>>>> >>
>>>>>> >> If you are a Spark user, you can help us test this release by
>>>>>> taking
>>>>>> >> an existing Spark workload and running on this release candidate,
>>>>>> then
>>>>>> >> reporting any regressions.
>>>>>> >>
>>>>>> >> If you're working in PySpark you can set up a virtual env and
>>>>>> install
>>>>>> >> the current RC and see if anything important breaks, in the
>>>>>> Java/Scala
>>>>>> >> you can add the staging repository to your projects resolvers and
>>>>>> test
>>>>>> >> with the RC (make sure to clean up the artifact cache before/after
>>>>>> so
>>>>>> >> you don't end up building with an out of date RC going forward).
>>>>>> >>
>>>>>> >> ===========================================
>>>>>> >> What should happen to JIRA tickets still targeting 3.1.3?
>>>>>> >> ===========================================
>>>>>> >>
>>>>>> >> The current list of open tickets targeted at 3.2.1 can be found at:
>>>>>> >> https://issues.apache.org/jira/projects/SPARK and search for
>>>>>> "Target
>>>>>> >> Version/s" = 3.1.3
>>>>>> >>
>>>>>> >> Committers should look at those and triage. Extremely important bug
>>>>>> >> fixes, documentation, and API tweaks that impact compatibility
>>>>>> should
>>>>>> >> be worked on immediately. Everything else please retarget to an
>>>>>> >> appropriate release.
>>>>>> >>
>>>>>> >> ==================
>>>>>> >> But my bug isn't fixed?
>>>>>> >> ==================
>>>>>> >>
>>>>>> >> In order to make timely releases, we will typically not hold the
>>>>>> >> release unless the bug in question is a regression from the
>>>>>> previous
>>>>>> >> release. That being said, if there is something that is a
>>>>>> regression
>>>>>> >> that has not been correctly targeted please ping me or a committer
>>>>>> to
>>>>>> >> help target the issue.
>>>>>> >>
>>>>>> >> ==================
>>>>>> >> What happened to RC1 & RC2?
>>>>>> >> ==================
>>>>>> >>
>>>>>> >> When I first went to build RC1 the build process failed due to the
>>>>>> >> lack of the R markdown package in my local rm container. By the
>>>>>> time
>>>>>> >> I had time to debug and rebuild there was already another bug fix
>>>>>> commit in
>>>>>> >> branch-3.1 so I decided to skip ahead to RC2 and pick it up
>>>>>> directly.
>>>>>> >> When I went to go send the RC2 vote e-mail I noticed a correctness
>>>>>> issue had
>>>>>> >> been fixed in branch-3.1 so I rolled RC3 to contain the
>>>>>> correctness fix.
>>>>>> >>
>>>>>> >> --
>>>>>> >> Twitter: https://twitter.com/holdenkarau
>>>>>> >> Books (Learning Spark, High Performance Spark, etc.):
>>>>>> https://amzn.to/2MaRAG9
>>>>>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>>>
>>>>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
> --
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [VOTE] Spark 3.1.3 RC3
Posted by Wenchen Fan <cl...@gmail.com>.
Shall we use the release scripts of branch 3.1 to release 3.1?
On Fri, Feb 4, 2022 at 4:57 AM Holden Karau <ho...@pigscanfly.ca> wrote:
> Good catch Dongjoon :)
>
> This release candidate fails, but feel free to keep testing for any other
> potential blockers.
>
> I’ll roll RC4 next week with the older release scripts (but the more
> modern image since the legacy image didn’t have a good time with the R doc
> packaging).
>
> On Thu, Feb 3, 2022 at 3:53 PM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> Unfortunately, -1 for 3.1.3 RC3 due to the packaging issue.
>>
>> It seems that the master branch release script didn't work properly for
>> Hadoop 2 binary distribution, Holden.
>>
>> $ curl -s
>> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/spark-3.1.3-bin-hadoop2.tgz
>> | tar tz | grep hadoop-common
>> spark-3.1.3-bin-hadoop2/jars/hadoop-common-3.2.0.jar
>>
>> Apache Spark didn't drop Apache Hadoop 2 based binary distribution yet.
>>
>> Dongjoon
>>
>>
>> On Wed, Feb 2, 2022 at 3:38 PM Mridul Muralidharan <mr...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Minor nit: the tag mentioned under [1] looks like a typo - I used
>>> "v3.1.3-rc3" for my vote (3.2.1 is mentioned in a couple of places, treat
>>> them as 3.1.3 instead)
>>>
>>> +1
>>> Signatures, digests, etc check out fine.
>>> Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes
>>>
>>> Regards,
>>> Mridul
>>>
>>> [1] "The tag to be voted on is v3.2.1-rc1" - the commit hash and git url
>>> are correct.
>>>
>>>
>>> On Wed, Feb 2, 2022 at 9:30 AM Mridul Muralidharan <mr...@gmail.com>
>>> wrote:
>>>
>>>>
>>>> Thanks Tom !
>>>> I missed [1] (or probably forgot) the 3.1 part of the discussion given
>>>> it centered around 3.2 ...
>>>>
>>>>
>>>> Regards,
>>>> Mridul
>>>>
>>>> [1] https://www.mail-archive.com/dev@spark.apache.org/msg28484.html
>>>>
>>>> On Wed, Feb 2, 2022 at 8:55 AM Thomas Graves <tg...@gmail.com>
>>>> wrote:
>>>>
>>>>> It was discussed doing all the maintenance lines back at beginning of
>>>>> December (Dec 6) when we were talking about release 3.2.1.
>>>>>
>>>>> Tom
>>>>>
>>>>> On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mr...@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> > Hi Holden,
>>>>> >
>>>>> > Not that I am against releasing 3.1.3 (given the fixes that have
>>>>> already gone in), but did we discuss releasing it ? I might have missed the
>>>>> thread ...
>>>>> >
>>>>> > Regards,
>>>>> > Mridul
>>>>> >
>>>>> > On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca>
>>>>> wrote:
>>>>> >>
>>>>> >> Please vote on releasing the following candidate as Apache Spark
>>>>> version 3.1.3.
>>>>> >>
>>>>> >> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and
>>>>> passes if a majority
>>>>> >> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
>>>>> >>
>>>>> >> [ ] +1 Release this package as Apache Spark 3.1.3
>>>>> >> [ ] -1 Do not release this package because ...
>>>>> >>
>>>>> >> To learn more about Apache Spark, please see
>>>>> http://spark.apache.org/
>>>>> >>
>>>>> >> There are currently no open issues targeting 3.1.3 in Spark's JIRA
>>>>> https://issues.apache.org/jira/browse
>>>>> >> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in
>>>>> (Open, Reopened, "In Progress"))
>>>>> >> at https://s.apache.org/n79dw
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> The tag to be voted on is v3.2.1-rc1 (commit
>>>>> >> b8c0799a8cef22c56132d94033759c9f82b0cc86):
>>>>> >> https://github.com/apache/spark/tree/v3.1.3-rc3
>>>>> >>
>>>>> >> The release files, including signatures, digests, etc. can be found
>>>>> at:
>>>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
>>>>> >>
>>>>> >> Signatures used for Spark RCs can be found in this file:
>>>>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>> >>
>>>>> >> The staging repository for this release can be found at
>>>>> >> :
>>>>> https://repository.apache.org/content/repositories/orgapachespark-1400/
>>>>> >>
>>>>> >> The documentation corresponding to this release can be found at:
>>>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
>>>>> >>
>>>>> >> The list of bug fixes going into 3.1.3 can be found at the
>>>>> following URL:
>>>>> >> https://s.apache.org/x0q9b
>>>>> >>
>>>>> >> This release is using the release script in master as of
>>>>> ddc77fb906cb3ce1567d277c2d0850104c89ac25
>>>>> >> The release docker container was rebuilt since the previous version
>>>>> didn't have the necessary components to build the R documentation.
>>>>> >>
>>>>> >> FAQ
>>>>> >>
>>>>> >>
>>>>> >> =========================
>>>>> >> How can I help test this release?
>>>>> >> =========================
>>>>> >>
>>>>> >> If you are a Spark user, you can help us test this release by taking
>>>>> >> an existing Spark workload and running on this release candidate,
>>>>> then
>>>>> >> reporting any regressions.
>>>>> >>
>>>>> >> If you're working in PySpark you can set up a virtual env and
>>>>> install
>>>>> >> the current RC and see if anything important breaks, in the
>>>>> Java/Scala
>>>>> >> you can add the staging repository to your projects resolvers and
>>>>> test
>>>>> >> with the RC (make sure to clean up the artifact cache before/after
>>>>> so
>>>>> >> you don't end up building with an out of date RC going forward).
>>>>> >>
>>>>> >> ===========================================
>>>>> >> What should happen to JIRA tickets still targeting 3.1.3?
>>>>> >> ===========================================
>>>>> >>
>>>>> >> The current list of open tickets targeted at 3.2.1 can be found at:
>>>>> >> https://issues.apache.org/jira/projects/SPARK and search for
>>>>> "Target
>>>>> >> Version/s" = 3.1.3
>>>>> >>
>>>>> >> Committers should look at those and triage. Extremely important bug
>>>>> >> fixes, documentation, and API tweaks that impact compatibility
>>>>> should
>>>>> >> be worked on immediately. Everything else please retarget to an
>>>>> >> appropriate release.
>>>>> >>
>>>>> >> ==================
>>>>> >> But my bug isn't fixed?
>>>>> >> ==================
>>>>> >>
>>>>> >> In order to make timely releases, we will typically not hold the
>>>>> >> release unless the bug in question is a regression from the previous
>>>>> >> release. That being said, if there is something that is a regression
>>>>> >> that has not been correctly targeted please ping me or a committer
>>>>> to
>>>>> >> help target the issue.
>>>>> >>
>>>>> >> ==================
>>>>> >> What happened to RC1 & RC2?
>>>>> >> ==================
>>>>> >>
>>>>> >> When I first went to build RC1 the build process failed due to the
>>>>> >> lack of the R markdown package in my local rm container. By the time
>>>>> >> I had time to debug and rebuild there was already another bug fix
>>>>> commit in
>>>>> >> branch-3.1 so I decided to skip ahead to RC2 and pick it up
>>>>> directly.
>>>>> >> When I went to go send the RC2 vote e-mail I noticed a correctness
>>>>> issue had
>>>>> >> been fixed in branch-3.1 so I rolled RC3 to contain the correctness
>>>>> fix.
>>>>> >>
>>>>> >> --
>>>>> >> Twitter: https://twitter.com/holdenkarau
>>>>> >> Books (Learning Spark, High Performance Spark, etc.):
>>>>> https://amzn.to/2MaRAG9
>>>>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>>
>>>> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
Re: [VOTE] Spark 3.1.3 RC3
Posted by Holden Karau <ho...@pigscanfly.ca>.
Good catch Dongjoon :)
This release candidate fails, but feel free to keep testing for any other
potential blockers.
I’ll roll RC4 next week with the older release scripts (but the more modern
image since the legacy image didn’t have a good time with the R doc
packaging).
On Thu, Feb 3, 2022 at 3:53 PM Dongjoon Hyun <do...@gmail.com>
wrote:
> Unfortunately, -1 for 3.1.3 RC3 due to the packaging issue.
>
> It seems that the master branch release script didn't work properly for
> Hadoop 2 binary distribution, Holden.
>
> $ curl -s
> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/spark-3.1.3-bin-hadoop2.tgz
> | tar tz | grep hadoop-common
> spark-3.1.3-bin-hadoop2/jars/hadoop-common-3.2.0.jar
>
> Apache Spark didn't drop Apache Hadoop 2 based binary distribution yet.
>
> Dongjoon
>
>
> On Wed, Feb 2, 2022 at 3:38 PM Mridul Muralidharan <mr...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Minor nit: the tag mentioned under [1] looks like a typo - I used
>> "v3.1.3-rc3" for my vote (3.2.1 is mentioned in a couple of places, treat
>> them as 3.1.3 instead)
>>
>> +1
>> Signatures, digests, etc check out fine.
>> Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes
>>
>> Regards,
>> Mridul
>>
>> [1] "The tag to be voted on is v3.2.1-rc1" - the commit hash and git url
>> are correct.
>>
>>
>> On Wed, Feb 2, 2022 at 9:30 AM Mridul Muralidharan <mr...@gmail.com>
>> wrote:
>>
>>>
>>> Thanks Tom !
>>> I missed [1] (or probably forgot) the 3.1 part of the discussion given
>>> it centered around 3.2 ...
>>>
>>>
>>> Regards,
>>> Mridul
>>>
>>> [1] https://www.mail-archive.com/dev@spark.apache.org/msg28484.html
>>>
>>> On Wed, Feb 2, 2022 at 8:55 AM Thomas Graves <tg...@gmail.com>
>>> wrote:
>>>
>>>> It was discussed doing all the maintenance lines back at beginning of
>>>> December (Dec 6) when we were talking about release 3.2.1.
>>>>
>>>> Tom
>>>>
>>>> On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mr...@gmail.com>
>>>> wrote:
>>>> >
>>>> > Hi Holden,
>>>> >
>>>> > Not that I am against releasing 3.1.3 (given the fixes that have
>>>> already gone in), but did we discuss releasing it ? I might have missed the
>>>> thread ...
>>>> >
>>>> > Regards,
>>>> > Mridul
>>>> >
>>>> > On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca>
>>>> wrote:
>>>> >>
>>>> >> Please vote on releasing the following candidate as Apache Spark
>>>> version 3.1.3.
>>>> >>
>>>> >> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and
>>>> passes if a majority
>>>> >> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
>>>> >>
>>>> >> [ ] +1 Release this package as Apache Spark 3.1.3
>>>> >> [ ] -1 Do not release this package because ...
>>>> >>
>>>> >> To learn more about Apache Spark, please see
>>>> http://spark.apache.org/
>>>> >>
>>>> >> There are currently no open issues targeting 3.1.3 in Spark's JIRA
>>>> https://issues.apache.org/jira/browse
>>>> >> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in
>>>> (Open, Reopened, "In Progress"))
>>>> >> at https://s.apache.org/n79dw
>>>> >>
>>>> >>
>>>> >>
>>>> >> The tag to be voted on is v3.2.1-rc1 (commit
>>>> >> b8c0799a8cef22c56132d94033759c9f82b0cc86):
>>>> >> https://github.com/apache/spark/tree/v3.1.3-rc3
>>>> >>
>>>> >> The release files, including signatures, digests, etc. can be found
>>>> at:
>>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
>>>> >>
>>>> >> Signatures used for Spark RCs can be found in this file:
>>>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>> >>
>>>> >> The staging repository for this release can be found at
>>>> >> :
>>>> https://repository.apache.org/content/repositories/orgapachespark-1400/
>>>> >>
>>>> >> The documentation corresponding to this release can be found at:
>>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
>>>> >>
>>>> >> The list of bug fixes going into 3.1.3 can be found at the following
>>>> URL:
>>>> >> https://s.apache.org/x0q9b
>>>> >>
>>>> >> This release is using the release script in master as of
>>>> ddc77fb906cb3ce1567d277c2d0850104c89ac25
>>>> >> The release docker container was rebuilt since the previous version
>>>> didn't have the necessary components to build the R documentation.
>>>> >>
>>>> >> FAQ
>>>> >>
>>>> >>
>>>> >> =========================
>>>> >> How can I help test this release?
>>>> >> =========================
>>>> >>
>>>> >> If you are a Spark user, you can help us test this release by taking
>>>> >> an existing Spark workload and running on this release candidate,
>>>> then
>>>> >> reporting any regressions.
>>>> >>
>>>> >> If you're working in PySpark you can set up a virtual env and install
>>>> >> the current RC and see if anything important breaks, in the
>>>> Java/Scala
>>>> >> you can add the staging repository to your projects resolvers and
>>>> test
>>>> >> with the RC (make sure to clean up the artifact cache before/after so
>>>> >> you don't end up building with an out of date RC going forward).
>>>> >>
>>>> >> ===========================================
>>>> >> What should happen to JIRA tickets still targeting 3.1.3?
>>>> >> ===========================================
>>>> >>
>>>> >> The current list of open tickets targeted at 3.2.1 can be found at:
>>>> >> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>>> >> Version/s" = 3.1.3
>>>> >>
>>>> >> Committers should look at those and triage. Extremely important bug
>>>> >> fixes, documentation, and API tweaks that impact compatibility should
>>>> >> be worked on immediately. Everything else please retarget to an
>>>> >> appropriate release.
>>>> >>
>>>> >> ==================
>>>> >> But my bug isn't fixed?
>>>> >> ==================
>>>> >>
>>>> >> In order to make timely releases, we will typically not hold the
>>>> >> release unless the bug in question is a regression from the previous
>>>> >> release. That being said, if there is something that is a regression
>>>> >> that has not been correctly targeted please ping me or a committer to
>>>> >> help target the issue.
>>>> >>
>>>> >> ==================
>>>> >> What happened to RC1 & RC2?
>>>> >> ==================
>>>> >>
>>>> >> When I first went to build RC1 the build process failed due to the
>>>> >> lack of the R markdown package in my local rm container. By the time
>>>> >> I had time to debug and rebuild there was already another bug fix
>>>> commit in
>>>> >> branch-3.1 so I decided to skip ahead to RC2 and pick it up directly.
>>>> >> When I went to go send the RC2 vote e-mail I noticed a correctness
>>>> issue had
>>>> >> been fixed in branch-3.1 so I rolled RC3 to contain the correctness
>>>> fix.
>>>> >>
>>>> >> --
>>>> >> Twitter: https://twitter.com/holdenkarau
>>>> >> Books (Learning Spark, High Performance Spark, etc.):
>>>> https://amzn.to/2MaRAG9
>>>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>
>>> --
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [VOTE] Spark 3.1.3 RC3
Posted by Dongjoon Hyun <do...@gmail.com>.
Unfortunately, -1 for 3.1.3 RC3 due to the packaging issue.
It seems that the master branch release script didn't work properly for
Hadoop 2 binary distribution, Holden.
$ curl -s
https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/spark-3.1.3-bin-hadoop2.tgz
| tar tz | grep hadoop-common
spark-3.1.3-bin-hadoop2/jars/hadoop-common-3.2.0.jar
Apache Spark didn't drop Apache Hadoop 2 based binary distribution yet.
Dongjoon
On Wed, Feb 2, 2022 at 3:38 PM Mridul Muralidharan <mr...@gmail.com> wrote:
> Hi,
>
> Minor nit: the tag mentioned under [1] looks like a typo - I used
> "v3.1.3-rc3" for my vote (3.2.1 is mentioned in a couple of places, treat
> them as 3.1.3 instead)
>
> +1
> Signatures, digests, etc check out fine.
> Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes
>
> Regards,
> Mridul
>
> [1] "The tag to be voted on is v3.2.1-rc1" - the commit hash and git url
> are correct.
>
>
> On Wed, Feb 2, 2022 at 9:30 AM Mridul Muralidharan <mr...@gmail.com>
> wrote:
>
>>
>> Thanks Tom !
>> I missed [1] (or probably forgot) the 3.1 part of the discussion given it
>> centered around 3.2 ...
>>
>>
>> Regards,
>> Mridul
>>
>> [1] https://www.mail-archive.com/dev@spark.apache.org/msg28484.html
>>
>> On Wed, Feb 2, 2022 at 8:55 AM Thomas Graves <tg...@gmail.com>
>> wrote:
>>
>>> It was discussed doing all the maintenance lines back at beginning of
>>> December (Dec 6) when we were talking about release 3.2.1.
>>>
>>> Tom
>>>
>>> On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mr...@gmail.com>
>>> wrote:
>>> >
>>> > Hi Holden,
>>> >
>>> > Not that I am against releasing 3.1.3 (given the fixes that have
>>> already gone in), but did we discuss releasing it ? I might have missed the
>>> thread ...
>>> >
>>> > Regards,
>>> > Mridul
>>> >
>>> > On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca>
>>> wrote:
>>> >>
>>> >> Please vote on releasing the following candidate as Apache Spark
>>> version 3.1.3.
>>> >>
>>> >> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and
>>> passes if a majority
>>> >> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
>>> >>
>>> >> [ ] +1 Release this package as Apache Spark 3.1.3
>>> >> [ ] -1 Do not release this package because ...
>>> >>
>>> >> To learn more about Apache Spark, please see http://spark.apache.org/
>>> >>
>>> >> There are currently no open issues targeting 3.1.3 in Spark's JIRA
>>> https://issues.apache.org/jira/browse
>>> >> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in
>>> (Open, Reopened, "In Progress"))
>>> >> at https://s.apache.org/n79dw
>>> >>
>>> >>
>>> >>
>>> >> The tag to be voted on is v3.2.1-rc1 (commit
>>> >> b8c0799a8cef22c56132d94033759c9f82b0cc86):
>>> >> https://github.com/apache/spark/tree/v3.1.3-rc3
>>> >>
>>> >> The release files, including signatures, digests, etc. can be found
>>> at:
>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
>>> >>
>>> >> Signatures used for Spark RCs can be found in this file:
>>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>> >>
>>> >> The staging repository for this release can be found at
>>> >> :
>>> https://repository.apache.org/content/repositories/orgapachespark-1400/
>>> >>
>>> >> The documentation corresponding to this release can be found at:
>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
>>> >>
>>> >> The list of bug fixes going into 3.1.3 can be found at the following
>>> URL:
>>> >> https://s.apache.org/x0q9b
>>> >>
>>> >> This release is using the release script in master as of
>>> ddc77fb906cb3ce1567d277c2d0850104c89ac25
>>> >> The release docker container was rebuilt since the previous version
>>> didn't have the necessary components to build the R documentation.
>>> >>
>>> >> FAQ
>>> >>
>>> >>
>>> >> =========================
>>> >> How can I help test this release?
>>> >> =========================
>>> >>
>>> >> If you are a Spark user, you can help us test this release by taking
>>> >> an existing Spark workload and running on this release candidate, then
>>> >> reporting any regressions.
>>> >>
>>> >> If you're working in PySpark you can set up a virtual env and install
>>> >> the current RC and see if anything important breaks, in the Java/Scala
>>> >> you can add the staging repository to your projects resolvers and test
>>> >> with the RC (make sure to clean up the artifact cache before/after so
>>> >> you don't end up building with an out of date RC going forward).
>>> >>
>>> >> ===========================================
>>> >> What should happen to JIRA tickets still targeting 3.1.3?
>>> >> ===========================================
>>> >>
>>> >> The current list of open tickets targeted at 3.2.1 can be found at:
>>> >> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>> >> Version/s" = 3.1.3
>>> >>
>>> >> Committers should look at those and triage. Extremely important bug
>>> >> fixes, documentation, and API tweaks that impact compatibility should
>>> >> be worked on immediately. Everything else please retarget to an
>>> >> appropriate release.
>>> >>
>>> >> ==================
>>> >> But my bug isn't fixed?
>>> >> ==================
>>> >>
>>> >> In order to make timely releases, we will typically not hold the
>>> >> release unless the bug in question is a regression from the previous
>>> >> release. That being said, if there is something that is a regression
>>> >> that has not been correctly targeted please ping me or a committer to
>>> >> help target the issue.
>>> >>
>>> >> ==================
>>> >> What happened to RC1 & RC2?
>>> >> ==================
>>> >>
>>> >> When I first went to build RC1 the build process failed due to the
>>> >> lack of the R markdown package in my local rm container. By the time
>>> >> I had time to debug and rebuild there was already another bug fix
>>> commit in
>>> >> branch-3.1 so I decided to skip ahead to RC2 and pick it up directly.
>>> >> When I went to go send the RC2 vote e-mail I noticed a correctness
>>> issue had
>>> >> been fixed in branch-3.1 so I rolled RC3 to contain the correctness
>>> fix.
>>> >>
>>> >> --
>>> >> Twitter: https://twitter.com/holdenkarau
>>> >> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9
>>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>
Re: [VOTE] Spark 3.1.3 RC3
Posted by Mridul Muralidharan <mr...@gmail.com>.
Hi,
Minor nit: the tag mentioned under [1] looks like a typo - I used
"v3.1.3-rc3" for my vote (3.2.1 is mentioned in a couple of places, treat
them as 3.1.3 instead)
+1
Signatures, digests, etc check out fine.
Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes
Regards,
Mridul
[1] "The tag to be voted on is v3.2.1-rc1" - the commit hash and git url
are correct.
On Wed, Feb 2, 2022 at 9:30 AM Mridul Muralidharan <mr...@gmail.com> wrote:
>
> Thanks Tom !
> I missed [1] (or probably forgot) the 3.1 part of the discussion given it
> centered around 3.2 ...
>
>
> Regards,
> Mridul
>
> [1] https://www.mail-archive.com/dev@spark.apache.org/msg28484.html
>
> On Wed, Feb 2, 2022 at 8:55 AM Thomas Graves <tg...@gmail.com> wrote:
>
>> It was discussed doing all the maintenance lines back at beginning of
>> December (Dec 6) when we were talking about release 3.2.1.
>>
>> Tom
>>
>> On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mr...@gmail.com>
>> wrote:
>> >
>> > Hi Holden,
>> >
>> > Not that I am against releasing 3.1.3 (given the fixes that have
>> already gone in), but did we discuss releasing it ? I might have missed the
>> thread ...
>> >
>> > Regards,
>> > Mridul
>> >
>> > On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca>
>> wrote:
>> >>
>> >> Please vote on releasing the following candidate as Apache Spark
>> version 3.1.3.
>> >>
>> >> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and
>> passes if a majority
>> >> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
>> >>
>> >> [ ] +1 Release this package as Apache Spark 3.1.3
>> >> [ ] -1 Do not release this package because ...
>> >>
>> >> To learn more about Apache Spark, please see http://spark.apache.org/
>> >>
>> >> There are currently no open issues targeting 3.1.3 in Spark's JIRA
>> https://issues.apache.org/jira/browse
>> >> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in
>> (Open, Reopened, "In Progress"))
>> >> at https://s.apache.org/n79dw
>> >>
>> >>
>> >>
>> >> The tag to be voted on is v3.2.1-rc1 (commit
>> >> b8c0799a8cef22c56132d94033759c9f82b0cc86):
>> >> https://github.com/apache/spark/tree/v3.1.3-rc3
>> >>
>> >> The release files, including signatures, digests, etc. can be found at:
>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
>> >>
>> >> Signatures used for Spark RCs can be found in this file:
>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
>> >>
>> >> The staging repository for this release can be found at
>> >> :
>> https://repository.apache.org/content/repositories/orgapachespark-1400/
>> >>
>> >> The documentation corresponding to this release can be found at:
>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
>> >>
>> >> The list of bug fixes going into 3.1.3 can be found at the following
>> URL:
>> >> https://s.apache.org/x0q9b
>> >>
>> >> This release is using the release script in master as of
>> ddc77fb906cb3ce1567d277c2d0850104c89ac25
>> >> The release docker container was rebuilt since the previous version
>> didn't have the necessary components to build the R documentation.
>> >>
>> >> FAQ
>> >>
>> >>
>> >> =========================
>> >> How can I help test this release?
>> >> =========================
>> >>
>> >> If you are a Spark user, you can help us test this release by taking
>> >> an existing Spark workload and running on this release candidate, then
>> >> reporting any regressions.
>> >>
>> >> If you're working in PySpark you can set up a virtual env and install
>> >> the current RC and see if anything important breaks, in the Java/Scala
>> >> you can add the staging repository to your projects resolvers and test
>> >> with the RC (make sure to clean up the artifact cache before/after so
>> >> you don't end up building with an out of date RC going forward).
>> >>
>> >> ===========================================
>> >> What should happen to JIRA tickets still targeting 3.1.3?
>> >> ===========================================
>> >>
>> >> The current list of open tickets targeted at 3.2.1 can be found at:
>> >> https://issues.apache.org/jira/projects/SPARK and search for "Target
>> >> Version/s" = 3.1.3
>> >>
>> >> Committers should look at those and triage. Extremely important bug
>> >> fixes, documentation, and API tweaks that impact compatibility should
>> >> be worked on immediately. Everything else please retarget to an
>> >> appropriate release.
>> >>
>> >> ==================
>> >> But my bug isn't fixed?
>> >> ==================
>> >>
>> >> In order to make timely releases, we will typically not hold the
>> >> release unless the bug in question is a regression from the previous
>> >> release. That being said, if there is something that is a regression
>> >> that has not been correctly targeted please ping me or a committer to
>> >> help target the issue.
>> >>
>> >> ==================
>> >> What happened to RC1 & RC2?
>> >> ==================
>> >>
>> >> When I first went to build RC1 the build process failed due to the
>> >> lack of the R markdown package in my local rm container. By the time
>> >> I had time to debug and rebuild there was already another bug fix
>> commit in
>> >> branch-3.1 so I decided to skip ahead to RC2 and pick it up directly.
>> >> When I went to go send the RC2 vote e-mail I noticed a correctness
>> issue had
>> >> been fixed in branch-3.1 so I rolled RC3 to contain the correctness
>> fix.
>> >>
>> >> --
>> >> Twitter: https://twitter.com/holdenkarau
>> >> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9
>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
Re: [VOTE] Spark 3.1.3 RC3
Posted by Mridul Muralidharan <mr...@gmail.com>.
Thanks Tom !
I missed [1] (or probably forgot) the 3.1 part of the discussion given it
centered around 3.2 ...
Regards,
Mridul
[1] https://www.mail-archive.com/dev@spark.apache.org/msg28484.html
On Wed, Feb 2, 2022 at 8:55 AM Thomas Graves <tg...@gmail.com> wrote:
> It was discussed doing all the maintenance lines back at beginning of
> December (Dec 6) when we were talking about release 3.2.1.
>
> Tom
>
> On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mr...@gmail.com>
> wrote:
> >
> > Hi Holden,
> >
> > Not that I am against releasing 3.1.3 (given the fixes that have
> already gone in), but did we discuss releasing it ? I might have missed the
> thread ...
> >
> > Regards,
> > Mridul
> >
> > On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca>
> wrote:
> >>
> >> Please vote on releasing the following candidate as Apache Spark
> version 3.1.3.
> >>
> >> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and
> passes if a majority
> >> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
> >>
> >> [ ] +1 Release this package as Apache Spark 3.1.3
> >> [ ] -1 Do not release this package because ...
> >>
> >> To learn more about Apache Spark, please see http://spark.apache.org/
> >>
> >> There are currently no open issues targeting 3.1.3 in Spark's JIRA
> https://issues.apache.org/jira/browse
> >> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in
> (Open, Reopened, "In Progress"))
> >> at https://s.apache.org/n79dw
> >>
> >>
> >>
> >> The tag to be voted on is v3.2.1-rc1 (commit
> >> b8c0799a8cef22c56132d94033759c9f82b0cc86):
> >> https://github.com/apache/spark/tree/v3.1.3-rc3
> >>
> >> The release files, including signatures, digests, etc. can be found at:
> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
> >>
> >> Signatures used for Spark RCs can be found in this file:
> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
> >>
> >> The staging repository for this release can be found at
> >> :
> https://repository.apache.org/content/repositories/orgapachespark-1400/
> >>
> >> The documentation corresponding to this release can be found at:
> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
> >>
> >> The list of bug fixes going into 3.1.3 can be found at the following
> URL:
> >> https://s.apache.org/x0q9b
> >>
> >> This release is using the release script in master as of
> ddc77fb906cb3ce1567d277c2d0850104c89ac25
> >> The release docker container was rebuilt since the previous version
> didn't have the necessary components to build the R documentation.
> >>
> >> FAQ
> >>
> >>
> >> =========================
> >> How can I help test this release?
> >> =========================
> >>
> >> If you are a Spark user, you can help us test this release by taking
> >> an existing Spark workload and running on this release candidate, then
> >> reporting any regressions.
> >>
> >> If you're working in PySpark you can set up a virtual env and install
> >> the current RC and see if anything important breaks, in the Java/Scala
> >> you can add the staging repository to your projects resolvers and test
> >> with the RC (make sure to clean up the artifact cache before/after so
> >> you don't end up building with an out of date RC going forward).
> >>
> >> ===========================================
> >> What should happen to JIRA tickets still targeting 3.1.3?
> >> ===========================================
> >>
> >> The current list of open tickets targeted at 3.2.1 can be found at:
> >> https://issues.apache.org/jira/projects/SPARK and search for "Target
> >> Version/s" = 3.1.3
> >>
> >> Committers should look at those and triage. Extremely important bug
> >> fixes, documentation, and API tweaks that impact compatibility should
> >> be worked on immediately. Everything else please retarget to an
> >> appropriate release.
> >>
> >> ==================
> >> But my bug isn't fixed?
> >> ==================
> >>
> >> In order to make timely releases, we will typically not hold the
> >> release unless the bug in question is a regression from the previous
> >> release. That being said, if there is something that is a regression
> >> that has not been correctly targeted please ping me or a committer to
> >> help target the issue.
> >>
> >> ==================
> >> What happened to RC1 & RC2?
> >> ==================
> >>
> >> When I first went to build RC1 the build process failed due to the
> >> lack of the R markdown package in my local rm container. By the time
> >> I had time to debug and rebuild there was already another bug fix
> commit in
> >> branch-3.1 so I decided to skip ahead to RC2 and pick it up directly.
> >> When I went to go send the RC2 vote e-mail I noticed a correctness
> issue had
> >> been fixed in branch-3.1 so I rolled RC3 to contain the correctness fix.
> >>
> >> --
> >> Twitter: https://twitter.com/holdenkarau
> >> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9
> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
Re: [VOTE] Spark 3.1.3 RC3
Posted by Thomas Graves <tg...@gmail.com>.
It was discussed doing all the maintenance lines back at beginning of
December (Dec 6) when we were talking about release 3.2.1.
Tom
On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mr...@gmail.com> wrote:
>
> Hi Holden,
>
> Not that I am against releasing 3.1.3 (given the fixes that have already gone in), but did we discuss releasing it ? I might have missed the thread ...
>
> Regards,
> Mridul
>
> On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca> wrote:
>>
>> Please vote on releasing the following candidate as Apache Spark version 3.1.3.
>>
>> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and passes if a majority
>> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
>>
>> [ ] +1 Release this package as Apache Spark 3.1.3
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see http://spark.apache.org/
>>
>> There are currently no open issues targeting 3.1.3 in Spark's JIRA https://issues.apache.org/jira/browse
>> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in (Open, Reopened, "In Progress"))
>> at https://s.apache.org/n79dw
>>
>>
>>
>> The tag to be voted on is v3.2.1-rc1 (commit
>> b8c0799a8cef22c56132d94033759c9f82b0cc86):
>> https://github.com/apache/spark/tree/v3.1.3-rc3
>>
>> The release files, including signatures, digests, etc. can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
>>
>> Signatures used for Spark RCs can be found in this file:
>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>
>> The staging repository for this release can be found at
>> :https://repository.apache.org/content/repositories/orgapachespark-1400/
>>
>> The documentation corresponding to this release can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
>>
>> The list of bug fixes going into 3.1.3 can be found at the following URL:
>> https://s.apache.org/x0q9b
>>
>> This release is using the release script in master as of ddc77fb906cb3ce1567d277c2d0850104c89ac25
>> The release docker container was rebuilt since the previous version didn't have the necessary components to build the R documentation.
>>
>> FAQ
>>
>>
>> =========================
>> How can I help test this release?
>> =========================
>>
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> If you're working in PySpark you can set up a virtual env and install
>> the current RC and see if anything important breaks, in the Java/Scala
>> you can add the staging repository to your projects resolvers and test
>> with the RC (make sure to clean up the artifact cache before/after so
>> you don't end up building with an out of date RC going forward).
>>
>> ===========================================
>> What should happen to JIRA tickets still targeting 3.1.3?
>> ===========================================
>>
>> The current list of open tickets targeted at 3.2.1 can be found at:
>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>> Version/s" = 3.1.3
>>
>> Committers should look at those and triage. Extremely important bug
>> fixes, documentation, and API tweaks that impact compatibility should
>> be worked on immediately. Everything else please retarget to an
>> appropriate release.
>>
>> ==================
>> But my bug isn't fixed?
>> ==================
>>
>> In order to make timely releases, we will typically not hold the
>> release unless the bug in question is a regression from the previous
>> release. That being said, if there is something that is a regression
>> that has not been correctly targeted please ping me or a committer to
>> help target the issue.
>>
>> ==================
>> What happened to RC1 & RC2?
>> ==================
>>
>> When I first went to build RC1 the build process failed due to the
>> lack of the R markdown package in my local rm container. By the time
>> I had time to debug and rebuild there was already another bug fix commit in
>> branch-3.1 so I decided to skip ahead to RC2 and pick it up directly.
>> When I went to go send the RC2 vote e-mail I noticed a correctness issue had
>> been fixed in branch-3.1 so I rolled RC3 to contain the correctness fix.
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [VOTE] Spark 3.1.3 RC3
Posted by Mridul Muralidharan <mr...@gmail.com>.
Hi Holden,
Not that I am against releasing 3.1.3 (given the fixes that have already
gone in), but did we discuss releasing it ? I might have missed the thread
...
Regards,
Mridul
On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <ho...@pigscanfly.ca> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 3.1.3.
>
> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and passes
> if a majority
> +1 PMC votes are cast, with a minimum of 3 + 1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.1.3
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> There are currently no open issues targeting 3.1.3 in Spark's JIRA
> https://issues.apache.org/jira/browse
> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in (Open,
> Reopened, "In Progress"))
> at https://s.apache.org/n79dw
>
>
>
> The tag to be voted on is v3.2.1-rc1 (commit
> b8c0799a8cef22c56132d94033759c9f82b0cc86):
> https://github.com/apache/spark/tree/v3.1.3-rc3
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at
> :https://repository.apache.org/content/repositories/orgapachespark-1400/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/
>
> The list of bug fixes going into 3.1.3 can be found at the following URL:
> https://s.apache.org/x0q9b
>
> This release is using the release script in master as
> of ddc77fb906cb3ce1567d277c2d0850104c89ac25
> The release docker container was rebuilt since the previous version didn't
> have the necessary components to build the R documentation.
>
> FAQ
>
>
> =========================
> How can I help test this release?
> =========================
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===========================================
> What should happen to JIRA tickets still targeting 3.1.3?
> ===========================================
>
> The current list of open tickets targeted at 3.2.1 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.1.3
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==================
> But my bug isn't fixed?
> ==================
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something that is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> ==================
> What happened to RC1 & RC2?
> ==================
>
> When I first went to build RC1 the build process failed due to the
> lack of the R markdown package in my local rm container. By the time
> I had time to debug and rebuild there was already another bug fix commit in
> branch-3.1 so I decided to skip ahead to RC2 and pick it up directly.
> When I went to go send the RC2 vote e-mail I noticed a correctness issue
> had
> been fixed in branch-3.1 so I rolled RC3 to contain the correctness fix.
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>