You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Prashant Sharma <sc...@gmail.com> on 2020/08/06 12:06:37 UTC
[VOTE] Release Spark 2.4.7 (RC1)
Please vote on releasing the following candidate as Apache Spark
version 2.4.7.
The vote is open until Aug 9th at 9AM PST and passes if a majority +1 PMC
votes are cast, with a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 2.4.7
[ ] -1 Do not release this package because ...
To learn more about Apache Spark, please see http://spark.apache.org/
There are currently no issues targeting 2.4.7 (try project = SPARK AND
"Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In Progress"))
The tag to be voted on is v2.4.7-rc1 (commit
dc04bf53fe821b7a07f817966c6c173f3b3788c6):
https://github.com/apache/spark/tree/v2.4.7-rc1
The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS
The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1352/
The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
The list of bug fixes going into 2.4.7 can be found at the following URL:
https://s.apache.org/spark-v2.4.7-rc1
This release is using the release script of the tag v2.4.7-rc1.
FAQ
=========================
How can I help test this release?
=========================
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.
If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with an out of date RC going forward).
===========================================
What should happen to JIRA tickets still targeting 2.4.7?
===========================================
The current list of open tickets targeted at 2.4.7 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target
Version/s" = 2.4.7
Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.
==================
But my bug isn't fixed?
==================
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Micah Kornfield <em...@gmail.com>.
Thank you for the clarification.
On Wed, Sep 2, 2020 at 6:18 AM Sean Owen <sr...@gmail.com> wrote:
> If you mean, will there be another 2.4.8 release? yes eventually. I am
> sure we'll do maintenance releases into early next year.
> What is the policy for holding a release for more changes? generally for
> correctness issues and/or regressions vs the previous maintenance release.
> Those entail an element of judgment call; other really important changes
> might hold it up.
> That looks like a change that should go into 2.4.x but not
> necessarily block 2.4.7. It doesn't look ready to commit, even.
>
> On Wed, Sep 2, 2020 at 1:29 AM Micah Kornfield <em...@gmail.com>
> wrote:
>
>> Just curious what is the policy for releases off of the 2.4 branch?
>>
>> It would be nice if a resolution to SPARK-32708
>> <https://issues.apache.org/jira/browse/SPARK-32708> (Reusing exchanges
>> for DataSourceV2) [1] could make it into the release. But if it can't what
>> would determine the need for a new release?
>>
>> Thanks,
>> Micah
>>
>> [1] https://issues.apache.org/jira/browse/SPARK-32708
>>
>> On Tue, Sep 1, 2020 at 11:19 PM Prashant Sharma <sc...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> FYI, currently there are no blockers to release 2.4.7 at the moment.
>>> Soon there will be RC3, I am awaiting some correctness bugs to be merged.
>>>
>>> Thanks,
>>>
>>> On Tue, Sep 1, 2020 at 6:43 PM Nicholas Marion <nm...@us.ibm.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I see that a 2.4.7-rc2 tag was created over a week ago, wondering if
>>>> there is an active vote going through for it?
>>>>
>>>>
>>>> Regards,
>>>>
>>>> *NICHOLAS T. MARION *
>>>> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
>>>> ------------------------------
>>>> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
>>>> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
>>>> *Find me on:* [image: LinkedIn:
>>>> http://www.linkedin.com/in/nicholasmarion]
>>>> <http://www.linkedin.com/in/nicholasmarion>
>>>> [image: IBM]
>>>>
>>>> 2455 South Rd
>>>> Poughkeepie, New York 12601-5400
>>>> United States
>>>> [image: IBM Redbooks Silver Author][image: Data Science Foundations -
>>>> Level 1]
>>>>
>>>>
>>>> [image: Inactive hide details for Tom Graves ---08/21/2020 03:09:49
>>>> PM--- There is a correctness issue with caching that should go into]Tom
>>>> Graves ---08/21/2020 03:09:49 PM--- There is a correctness issue with
>>>> caching that should go into this if possible: https://urldefense.
>>>>
>>>> From: Tom Graves <tg...@yahoo.com.INVALID>
>>>> To: Nicholas Marion <nm...@us.ibm.com>, Wenchen Fan <
>>>> cloud0fan@gmail.com>
>>>> Cc: Xiao Li <li...@databricks.com>, dev <de...@spark.apache.org>,
>>>> Takeshi Yamamuro <li...@gmail.com>, Prashant Sharma <
>>>> scrapcodes@gmail.com>
>>>> Date: 08/21/2020 03:09 PM
>>>> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
>>>>
>>>> ------------------------------
>>>>
>>>>
>>>>
>>>> There is a correctness issue with caching that should go into this if
>>>> possible: *https://github.com/apache/spark/pull/29506*
>>>> <https://github.com/apache/spark/pull/29506>
>>>>
>>>> Tom
>>>>
>>>> On Wednesday, August 19, 2020, 11:18:37 AM CDT, Wenchen Fan <
>>>> cloud0fan@gmail.com> wrote:
>>>>
>>>>
>>>> I think so. I don't see other bug reports for 2.4.
>>>>
>>>> On Thu, Aug 20, 2020 at 12:11 AM Nicholas Marion <*nmarion@us.ibm.com*
>>>> <nm...@us.ibm.com>> wrote:
>>>>
>>>> It appears all 3 issues slated for Spark 2.4.7 have been merged.
>>>> Should we be looking at getting RC2 ready?
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> *NICHOLAS T. MARION *
>>>> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
>>>> ------------------------------
>>>> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
>>>> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
>>>> *Find me on:* <http://www.linkedin.com/in/nicholasmarion>
>>>>
>>>>
>>>> 2455 South Rd
>>>> Poughkeepie, New York 12601-5400
>>>> United States
>>>>
>>>>
>>>>
>>>>
>>>> [image: Inactive hide details for Xiao Li ---08/17/2020 11:33:30
>>>> AM---INVALID URI REMOVED]Xiao Li ---08/17/2020 11:33:30 AM---
>>>> *https://issues.apache.org/jira/browse/SPARK-32609*
>>>> <https://issues.apache.org/jira/browse/SPARK-32609>
>>>>
>>>> From: Xiao Li <*lixiao@databricks.com* <li...@databricks.com>>
>>>> To: Prashant Sharma <*scrapcodes@gmail.com* <sc...@gmail.com>>
>>>> Cc: Takeshi Yamamuro <*linguin.m.s@gmail.com*
>>>> <li...@gmail.com>>, dev <*dev@spark.apache.org*
>>>> <de...@spark.apache.org>>
>>>> Date: 08/17/2020 11:33 AM
>>>> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
>>>> ------------------------------
>>>>
>>>>
>>>>
>>>> *https://issues.apache.org/jira/browse/SPARK-32609*
>>>> <https://issues.apache.org/jira/browse/SPARK-32609> got merged.
>>>> This is to fix a correctness bug in DSV2 of Spark 2.4. Please include it in
>>>> the upcoming Spark 2.4.7 release.
>>>>
>>>> Thanks,
>>>>
>>>> Xiao
>>>>
>>>> On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <
>>>> *scrapcodes@gmail.com* <sc...@gmail.com>> wrote:
>>>> Thanks for letting us know. So this vote is cancelled in favor of
>>>> RC2.
>>>>
>>>>
>>>>
>>>> On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <
>>>> *linguin.m.s@gmail.com* <li...@gmail.com>> wrote:
>>>> Thanks for letting us know about the two issues above, Dongjoon.
>>>>
>>>> ----
>>>> I've checked the release materials (signatures, tag, ...) and it
>>>> looks fine, too.
>>>> Also, I run the tests on my local Mac (java 1.8.0) with the options
>>>> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos
>>>> -Pkubernetes -Psparkr`
>>>> and they passed.
>>>>
>>>> Bests,
>>>> Takeshi
>>>>
>>>>
>>>>
>>>> On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <
>>>> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
>>>> Another instance is SPARK-31703 which filed on May 13th and the PR
>>>> arrived two days ago.
>>>>
>>>> [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on
>>>> big endian platforms
>>>> *https://github.com/apache/spark/pull/29383*
>>>> <https://github.com/apache/spark/pull/29383>
>>>>
>>>> It seems that the patch is already ready in this case.
>>>> I raised the priority of SPARK-31703 to `Blocker` for both Apache
>>>> Spark 2.4.7 and 3.0.1.
>>>>
>>>> Bests,
>>>> Dongjoon.
>>>>
>>>>
>>>> On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <*holden@pigscanfly.ca*
>>>> <ho...@pigscanfly.ca>> wrote:
>>>> I'm going to go ahead and vote -0 then based on that then.
>>>>
>>>> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <
>>>> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
>>>> Hi, All.
>>>>
>>>> Unfortunately, there is an on-going discussion about the new
>>>> decimal correctness.
>>>>
>>>> Although we fixed one correctness issue at master and backported it
>>>> partially to 3.0/2.4, it turns out that it needs more patched to be
>>>> complete.
>>>>
>>>> Please see *https://github.com/apache/spark/pull/29125*
>>>> <https://github.com/apache/spark/pull/29125> for on-going
>>>> discussion for both 3.0/2.4.
>>>>
>>>> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
>>>> overflowed value
>>>>
>>>> I also confirmed that 2.4.7 RC1 is affected.
>>>>
>>>> Bests,
>>>> Dongjoon.
>>>>
>>>>
>>>> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <*srowen@apache.org*
>>>> <sr...@apache.org>> wrote:
>>>> +1 from me. The same as usual. Licenses and sigs look OK, builds and
>>>> passes tests on a standard selection of profiles.
>>>>
>>>> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <
>>>> *scrapcodes@gmail.com* <sc...@gmail.com>> wrote:
>>>> >
>>>> > Please vote on releasing the following candidate as Apache Spark
>>>> version 2.4.7.
>>>> >
>>>> > The vote is open until Aug 9th at 9AM PST and passes if a
>>>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>> >
>>>> > [ ] +1 Release this package as Apache Spark 2.4.7
>>>> > [ ] -1 Do not release this package because ...
>>>> >
>>>> > To learn more about Apache Spark, please see
>>>> *http://spark.apache.org/* <http://spark.apache.org/>
>>>> >
>>>> > There are currently no issues targeting 2.4.7 (try project =
>>>> SPARK AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
>>>> Progress"))
>>>> >
>>>> > The tag to be voted on is v2.4.7-rc1 (commit
>>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>>>> > *https://github.com/apache/spark/tree/v2.4.7-rc1*
>>>> <https://github.com/apache/spark/tree/v2.4.7-rc1>
>>>> >
>>>> > The release files, including signatures, digests, etc. can be
>>>> found at:
>>>> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/*
>>>> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/>
>>>> >
>>>> > Signatures used for Spark RCs can be found in this file:
>>>> > *https://dist.apache.org/repos/dist/dev/spark/KEYS*
>>>> <https://dist.apache.org/repos/dist/dev/spark/KEYS>
>>>> >
>>>> > The staging repository for this release can be found at:
>>>> >
>>>> *https://repository.apache.org/content/repositories/orgapachespark-1352/*
>>>> <https://repository.apache.org/content/repositories/orgapachespark-1352/>
>>>> >
>>>> > The documentation corresponding to this release can be found at:
>>>> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/*
>>>> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/>
>>>> >
>>>> > The list of bug fixes going into 2.4.7 can be found at the
>>>> following URL:
>>>> > *https://s.apache.org/spark-v2.4.7-rc1*
>>>> <https://s.apache.org/spark-v2.4.7-rc1>
>>>> >
>>>> > This release is using the release script of the tag v2.4.7-rc1.
>>>> >
>>>> > FAQ
>>>> >
>>>> >
>>>> > =========================
>>>> > How can I help test this release?
>>>> > =========================
>>>> >
>>>> > If you are a Spark user, you can help us test this release by
>>>> taking
>>>> > an existing Spark workload and running on this release candidate,
>>>> then
>>>> > reporting any regressions.
>>>> >
>>>> > If you're working in PySpark you can set up a virtual env and
>>>> install
>>>> > the current RC and see if anything important breaks, in the
>>>> Java/Scala
>>>> > you can add the staging repository to your projects resolvers and
>>>> test
>>>> > with the RC (make sure to clean up the artifact cache
>>>> before/after so
>>>> > you don't end up building with an out of date RC going forward).
>>>> >
>>>> > ===========================================
>>>> > What should happen to JIRA tickets still targeting 2.4.7?
>>>> > ===========================================
>>>> >
>>>> > The current list of open tickets targeted at 2.4.7 can be found
>>>> at:
>>>> > *https://issues.apache.org/jira/projects/SPARK*
>>>> <https://issues.apache.org/jira/projects/SPARK> and search for
>>>> "Target Version/s" = 2.4.7
>>>> >
>>>> > Committers should look at those and triage. Extremely important
>>>> bug
>>>> > fixes, documentation, and API tweaks that impact compatibility
>>>> should
>>>> > be worked on immediately. Everything else please retarget to an
>>>> > appropriate release.
>>>> >
>>>> > ==================
>>>> > But my bug isn't fixed?
>>>> > ==================
>>>> >
>>>> > In order to make timely releases, we will typically not hold the
>>>> > release unless the bug in question is a regression from the
>>>> previous
>>>> > release. That being said, if there is something which is a
>>>> regression
>>>> > that has not been correctly targeted please ping me or a
>>>> committer to
>>>> > help target the issue.
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: *dev-unsubscribe@spark.apache.org*
>>>> <de...@spark.apache.org>
>>>>
>>>>
>>>> --
>>>> Twitter: *https://twitter.com/holdenkarau*
>>>> <https://twitter.com/holdenkarau>
>>>> Books (Learning Spark, High Performance Spark, etc.): *https://amzn.to/2MaRAG9
>>>> * <https://amzn.to/2MaRAG9>
>>>> YouTube Live Streams: *https://www.youtube.com/user/holdenkarau*
>>>> <https://www.youtube.com/user/holdenkarau>
>>>>
>>>> --
>>>> ---
>>>> Takeshi Yamamuro
>>>>
>>>> --
>>>> <https://databricks.com/sparkaisummit/north-america>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>
>>>>
>>>>
>>>>
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Sean Owen <sr...@gmail.com>.
If you mean, will there be another 2.4.8 release? yes eventually. I am sure
we'll do maintenance releases into early next year.
What is the policy for holding a release for more changes? generally for
correctness issues and/or regressions vs the previous maintenance release.
Those entail an element of judgment call; other really important changes
might hold it up.
That looks like a change that should go into 2.4.x but not
necessarily block 2.4.7. It doesn't look ready to commit, even.
On Wed, Sep 2, 2020 at 1:29 AM Micah Kornfield <em...@gmail.com>
wrote:
> Just curious what is the policy for releases off of the 2.4 branch?
>
> It would be nice if a resolution to SPARK-32708
> <https://issues.apache.org/jira/browse/SPARK-32708> (Reusing exchanges
> for DataSourceV2) [1] could make it into the release. But if it can't what
> would determine the need for a new release?
>
> Thanks,
> Micah
>
> [1] https://issues.apache.org/jira/browse/SPARK-32708
>
> On Tue, Sep 1, 2020 at 11:19 PM Prashant Sharma <sc...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> FYI, currently there are no blockers to release 2.4.7 at the moment. Soon
>> there will be RC3, I am awaiting some correctness bugs to be merged.
>>
>> Thanks,
>>
>> On Tue, Sep 1, 2020 at 6:43 PM Nicholas Marion <nm...@us.ibm.com>
>> wrote:
>>
>>> Hello,
>>>
>>> I see that a 2.4.7-rc2 tag was created over a week ago, wondering if
>>> there is an active vote going through for it?
>>>
>>>
>>> Regards,
>>>
>>> *NICHOLAS T. MARION *
>>> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
>>> ------------------------------
>>> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
>>> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
>>> *Find me on:* [image: LinkedIn:
>>> http://www.linkedin.com/in/nicholasmarion]
>>> <http://www.linkedin.com/in/nicholasmarion>
>>> [image: IBM]
>>>
>>> 2455 South Rd
>>> Poughkeepie, New York 12601-5400
>>> United States
>>> [image: IBM Redbooks Silver Author][image: Data Science Foundations -
>>> Level 1]
>>>
>>>
>>> [image: Inactive hide details for Tom Graves ---08/21/2020 03:09:49
>>> PM--- There is a correctness issue with caching that should go into]Tom
>>> Graves ---08/21/2020 03:09:49 PM--- There is a correctness issue with
>>> caching that should go into this if possible: https://urldefense.
>>>
>>> From: Tom Graves <tg...@yahoo.com.INVALID>
>>> To: Nicholas Marion <nm...@us.ibm.com>, Wenchen Fan <
>>> cloud0fan@gmail.com>
>>> Cc: Xiao Li <li...@databricks.com>, dev <de...@spark.apache.org>,
>>> Takeshi Yamamuro <li...@gmail.com>, Prashant Sharma <
>>> scrapcodes@gmail.com>
>>> Date: 08/21/2020 03:09 PM
>>> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
>>>
>>> ------------------------------
>>>
>>>
>>>
>>> There is a correctness issue with caching that should go into this if
>>> possible: *https://github.com/apache/spark/pull/29506*
>>> <https://github.com/apache/spark/pull/29506>
>>>
>>> Tom
>>>
>>> On Wednesday, August 19, 2020, 11:18:37 AM CDT, Wenchen Fan <
>>> cloud0fan@gmail.com> wrote:
>>>
>>>
>>> I think so. I don't see other bug reports for 2.4.
>>>
>>> On Thu, Aug 20, 2020 at 12:11 AM Nicholas Marion <*nmarion@us.ibm.com*
>>> <nm...@us.ibm.com>> wrote:
>>>
>>> It appears all 3 issues slated for Spark 2.4.7 have been merged.
>>> Should we be looking at getting RC2 ready?
>>>
>>>
>>>
>>> Regards,
>>>
>>> *NICHOLAS T. MARION *
>>> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
>>> ------------------------------
>>> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
>>> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
>>> *Find me on:* <http://www.linkedin.com/in/nicholasmarion>
>>>
>>>
>>> 2455 South Rd
>>> Poughkeepie, New York 12601-5400
>>> United States
>>>
>>>
>>>
>>>
>>> [image: Inactive hide details for Xiao Li ---08/17/2020 11:33:30
>>> AM---INVALID URI REMOVED]Xiao Li ---08/17/2020 11:33:30 AM---
>>> *https://issues.apache.org/jira/browse/SPARK-32609*
>>> <https://issues.apache.org/jira/browse/SPARK-32609>
>>>
>>> From: Xiao Li <*lixiao@databricks.com* <li...@databricks.com>>
>>> To: Prashant Sharma <*scrapcodes@gmail.com* <sc...@gmail.com>>
>>> Cc: Takeshi Yamamuro <*linguin.m.s@gmail.com* <li...@gmail.com>>,
>>> dev <*dev@spark.apache.org* <de...@spark.apache.org>>
>>> Date: 08/17/2020 11:33 AM
>>> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
>>> ------------------------------
>>>
>>>
>>>
>>> *https://issues.apache.org/jira/browse/SPARK-32609*
>>> <https://issues.apache.org/jira/browse/SPARK-32609> got merged. This
>>> is to fix a correctness bug in DSV2 of Spark 2.4. Please include it in the
>>> upcoming Spark 2.4.7 release.
>>>
>>> Thanks,
>>>
>>> Xiao
>>>
>>> On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <
>>> *scrapcodes@gmail.com* <sc...@gmail.com>> wrote:
>>> Thanks for letting us know. So this vote is cancelled in favor of
>>> RC2.
>>>
>>>
>>>
>>> On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <
>>> *linguin.m.s@gmail.com* <li...@gmail.com>> wrote:
>>> Thanks for letting us know about the two issues above, Dongjoon.
>>>
>>> ----
>>> I've checked the release materials (signatures, tag, ...) and it
>>> looks fine, too.
>>> Also, I run the tests on my local Mac (java 1.8.0) with the options
>>> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
>>> -Psparkr`
>>> and they passed.
>>>
>>> Bests,
>>> Takeshi
>>>
>>>
>>>
>>> On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <
>>> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
>>> Another instance is SPARK-31703 which filed on May 13th and the PR
>>> arrived two days ago.
>>>
>>> [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on
>>> big endian platforms
>>> *https://github.com/apache/spark/pull/29383*
>>> <https://github.com/apache/spark/pull/29383>
>>>
>>> It seems that the patch is already ready in this case.
>>> I raised the priority of SPARK-31703 to `Blocker` for both Apache
>>> Spark 2.4.7 and 3.0.1.
>>>
>>> Bests,
>>> Dongjoon.
>>>
>>>
>>> On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <*holden@pigscanfly.ca*
>>> <ho...@pigscanfly.ca>> wrote:
>>> I'm going to go ahead and vote -0 then based on that then.
>>>
>>> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <
>>> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
>>> Hi, All.
>>>
>>> Unfortunately, there is an on-going discussion about the new decimal
>>> correctness.
>>>
>>> Although we fixed one correctness issue at master and backported it
>>> partially to 3.0/2.4, it turns out that it needs more patched to be
>>> complete.
>>>
>>> Please see *https://github.com/apache/spark/pull/29125*
>>> <https://github.com/apache/spark/pull/29125> for on-going discussion
>>> for both 3.0/2.4.
>>>
>>> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
>>> overflowed value
>>>
>>> I also confirmed that 2.4.7 RC1 is affected.
>>>
>>> Bests,
>>> Dongjoon.
>>>
>>>
>>> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <*srowen@apache.org*
>>> <sr...@apache.org>> wrote:
>>> +1 from me. The same as usual. Licenses and sigs look OK, builds and
>>> passes tests on a standard selection of profiles.
>>>
>>> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <
>>> *scrapcodes@gmail.com* <sc...@gmail.com>> wrote:
>>> >
>>> > Please vote on releasing the following candidate as Apache Spark
>>> version 2.4.7.
>>> >
>>> > The vote is open until Aug 9th at 9AM PST and passes if a majority
>>> +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>> >
>>> > [ ] +1 Release this package as Apache Spark 2.4.7
>>> > [ ] -1 Do not release this package because ...
>>> >
>>> > To learn more about Apache Spark, please see
>>> *http://spark.apache.org/* <http://spark.apache.org/>
>>> >
>>> > There are currently no issues targeting 2.4.7 (try project = SPARK
>>> AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
>>> Progress"))
>>> >
>>> > The tag to be voted on is v2.4.7-rc1 (commit
>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>>> > *https://github.com/apache/spark/tree/v2.4.7-rc1*
>>> <https://github.com/apache/spark/tree/v2.4.7-rc1>
>>> >
>>> > The release files, including signatures, digests, etc. can be
>>> found at:
>>> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/*
>>> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/>
>>> >
>>> > Signatures used for Spark RCs can be found in this file:
>>> > *https://dist.apache.org/repos/dist/dev/spark/KEYS*
>>> <https://dist.apache.org/repos/dist/dev/spark/KEYS>
>>> >
>>> > The staging repository for this release can be found at:
>>> >
>>> *https://repository.apache.org/content/repositories/orgapachespark-1352/*
>>> <https://repository.apache.org/content/repositories/orgapachespark-1352/>
>>> >
>>> > The documentation corresponding to this release can be found at:
>>> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/*
>>> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/>
>>> >
>>> > The list of bug fixes going into 2.4.7 can be found at the
>>> following URL:
>>> > *https://s.apache.org/spark-v2.4.7-rc1*
>>> <https://s.apache.org/spark-v2.4.7-rc1>
>>> >
>>> > This release is using the release script of the tag v2.4.7-rc1.
>>> >
>>> > FAQ
>>> >
>>> >
>>> > =========================
>>> > How can I help test this release?
>>> > =========================
>>> >
>>> > If you are a Spark user, you can help us test this release by
>>> taking
>>> > an existing Spark workload and running on this release candidate,
>>> then
>>> > reporting any regressions.
>>> >
>>> > If you're working in PySpark you can set up a virtual env and
>>> install
>>> > the current RC and see if anything important breaks, in the
>>> Java/Scala
>>> > you can add the staging repository to your projects resolvers and
>>> test
>>> > with the RC (make sure to clean up the artifact cache before/after
>>> so
>>> > you don't end up building with an out of date RC going forward).
>>> >
>>> > ===========================================
>>> > What should happen to JIRA tickets still targeting 2.4.7?
>>> > ===========================================
>>> >
>>> > The current list of open tickets targeted at 2.4.7 can be found at:
>>> > *https://issues.apache.org/jira/projects/SPARK*
>>> <https://issues.apache.org/jira/projects/SPARK> and search for
>>> "Target Version/s" = 2.4.7
>>> >
>>> > Committers should look at those and triage. Extremely important bug
>>> > fixes, documentation, and API tweaks that impact compatibility
>>> should
>>> > be worked on immediately. Everything else please retarget to an
>>> > appropriate release.
>>> >
>>> > ==================
>>> > But my bug isn't fixed?
>>> > ==================
>>> >
>>> > In order to make timely releases, we will typically not hold the
>>> > release unless the bug in question is a regression from the
>>> previous
>>> > release. That being said, if there is something which is a
>>> regression
>>> > that has not been correctly targeted please ping me or a committer
>>> to
>>> > help target the issue.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: *dev-unsubscribe@spark.apache.org*
>>> <de...@spark.apache.org>
>>>
>>>
>>> --
>>> Twitter: *https://twitter.com/holdenkarau*
>>> <https://twitter.com/holdenkarau>
>>> Books (Learning Spark, High Performance Spark, etc.): *https://amzn.to/2MaRAG9
>>> * <https://amzn.to/2MaRAG9>
>>> YouTube Live Streams: *https://www.youtube.com/user/holdenkarau*
>>> <https://www.youtube.com/user/holdenkarau>
>>>
>>> --
>>> ---
>>> Takeshi Yamamuro
>>>
>>> --
>>> <https://databricks.com/sparkaisummit/north-america>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>
>>>
>>>
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Micah Kornfield <em...@gmail.com>.
Just curious what is the policy for releases off of the 2.4 branch?
It would be nice if a resolution to SPARK-32708
<https://issues.apache.org/jira/browse/SPARK-32708> (Reusing exchanges for
DataSourceV2) [1] could make it into the release. But if it can't what
would determine the need for a new release?
Thanks,
Micah
[1] https://issues.apache.org/jira/browse/SPARK-32708
On Tue, Sep 1, 2020 at 11:19 PM Prashant Sharma <sc...@gmail.com>
wrote:
> Hi All,
>
> FYI, currently there are no blockers to release 2.4.7 at the moment. Soon
> there will be RC3, I am awaiting some correctness bugs to be merged.
>
> Thanks,
>
> On Tue, Sep 1, 2020 at 6:43 PM Nicholas Marion <nm...@us.ibm.com> wrote:
>
>> Hello,
>>
>> I see that a 2.4.7-rc2 tag was created over a week ago, wondering if
>> there is an active vote going through for it?
>>
>>
>> Regards,
>>
>> *NICHOLAS T. MARION *
>> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
>> ------------------------------
>> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
>> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
>> *Find me on:* [image: LinkedIn:
>> http://www.linkedin.com/in/nicholasmarion]
>> <http://www.linkedin.com/in/nicholasmarion>
>> [image: IBM]
>>
>> 2455 South Rd
>> Poughkeepie, New York 12601-5400
>> United States
>> [image: IBM Redbooks Silver Author][image: Data Science Foundations -
>> Level 1]
>>
>>
>> [image: Inactive hide details for Tom Graves ---08/21/2020 03:09:49 PM---
>> There is a correctness issue with caching that should go into]Tom Graves
>> ---08/21/2020 03:09:49 PM--- There is a correctness issue with caching that
>> should go into this if possible: https://urldefense.
>>
>> From: Tom Graves <tg...@yahoo.com.INVALID>
>> To: Nicholas Marion <nm...@us.ibm.com>, Wenchen Fan <
>> cloud0fan@gmail.com>
>> Cc: Xiao Li <li...@databricks.com>, dev <de...@spark.apache.org>, Takeshi
>> Yamamuro <li...@gmail.com>, Prashant Sharma <sc...@gmail.com>
>> Date: 08/21/2020 03:09 PM
>> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
>>
>> ------------------------------
>>
>>
>>
>> There is a correctness issue with caching that should go into this if
>> possible: *https://github.com/apache/spark/pull/29506*
>> <https://github.com/apache/spark/pull/29506>
>>
>> Tom
>>
>> On Wednesday, August 19, 2020, 11:18:37 AM CDT, Wenchen Fan <
>> cloud0fan@gmail.com> wrote:
>>
>>
>> I think so. I don't see other bug reports for 2.4.
>>
>> On Thu, Aug 20, 2020 at 12:11 AM Nicholas Marion <*nmarion@us.ibm.com*
>> <nm...@us.ibm.com>> wrote:
>>
>> It appears all 3 issues slated for Spark 2.4.7 have been merged.
>> Should we be looking at getting RC2 ready?
>>
>>
>>
>> Regards,
>>
>> *NICHOLAS T. MARION *
>> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
>> ------------------------------
>> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
>> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
>> *Find me on:* <http://www.linkedin.com/in/nicholasmarion>
>>
>>
>> 2455 South Rd
>> Poughkeepie, New York 12601-5400
>> United States
>>
>>
>>
>>
>> [image: Inactive hide details for Xiao Li ---08/17/2020 11:33:30
>> AM---INVALID URI REMOVED]Xiao Li ---08/17/2020 11:33:30 AM---
>> *https://issues.apache.org/jira/browse/SPARK-32609*
>> <https://issues.apache.org/jira/browse/SPARK-32609>
>>
>> From: Xiao Li <*lixiao@databricks.com* <li...@databricks.com>>
>> To: Prashant Sharma <*scrapcodes@gmail.com* <sc...@gmail.com>>
>> Cc: Takeshi Yamamuro <*linguin.m.s@gmail.com* <li...@gmail.com>>,
>> dev <*dev@spark.apache.org* <de...@spark.apache.org>>
>> Date: 08/17/2020 11:33 AM
>> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
>> ------------------------------
>>
>>
>>
>> *https://issues.apache.org/jira/browse/SPARK-32609*
>> <https://issues.apache.org/jira/browse/SPARK-32609> got merged. This
>> is to fix a correctness bug in DSV2 of Spark 2.4. Please include it in the
>> upcoming Spark 2.4.7 release.
>>
>> Thanks,
>>
>> Xiao
>>
>> On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <
>> *scrapcodes@gmail.com* <sc...@gmail.com>> wrote:
>> Thanks for letting us know. So this vote is cancelled in favor of RC2.
>>
>>
>>
>> On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <
>> *linguin.m.s@gmail.com* <li...@gmail.com>> wrote:
>> Thanks for letting us know about the two issues above, Dongjoon.
>>
>> ----
>> I've checked the release materials (signatures, tag, ...) and it
>> looks fine, too.
>> Also, I run the tests on my local Mac (java 1.8.0) with the options
>> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
>> -Psparkr`
>> and they passed.
>>
>> Bests,
>> Takeshi
>>
>>
>>
>> On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <
>> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
>> Another instance is SPARK-31703 which filed on May 13th and the PR
>> arrived two days ago.
>>
>> [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on
>> big endian platforms
>> *https://github.com/apache/spark/pull/29383*
>> <https://github.com/apache/spark/pull/29383>
>>
>> It seems that the patch is already ready in this case.
>> I raised the priority of SPARK-31703 to `Blocker` for both Apache
>> Spark 2.4.7 and 3.0.1.
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <*holden@pigscanfly.ca*
>> <ho...@pigscanfly.ca>> wrote:
>> I'm going to go ahead and vote -0 then based on that then.
>>
>> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <
>> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
>> Hi, All.
>>
>> Unfortunately, there is an on-going discussion about the new decimal
>> correctness.
>>
>> Although we fixed one correctness issue at master and backported it
>> partially to 3.0/2.4, it turns out that it needs more patched to be
>> complete.
>>
>> Please see *https://github.com/apache/spark/pull/29125*
>> <https://github.com/apache/spark/pull/29125> for on-going discussion
>> for both 3.0/2.4.
>>
>> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
>> overflowed value
>>
>> I also confirmed that 2.4.7 RC1 is affected.
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <*srowen@apache.org*
>> <sr...@apache.org>> wrote:
>> +1 from me. The same as usual. Licenses and sigs look OK, builds and
>> passes tests on a standard selection of profiles.
>>
>> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <*scrapcodes@gmail.com*
>> <sc...@gmail.com>> wrote:
>> >
>> > Please vote on releasing the following candidate as Apache Spark
>> version 2.4.7.
>> >
>> > The vote is open until Aug 9th at 9AM PST and passes if a majority
>> +1 PMC votes are cast, with a minimum of 3 +1 votes.
>> >
>> > [ ] +1 Release this package as Apache Spark 2.4.7
>> > [ ] -1 Do not release this package because ...
>> >
>> > To learn more about Apache Spark, please see
>> *http://spark.apache.org/* <http://spark.apache.org/>
>> >
>> > There are currently no issues targeting 2.4.7 (try project = SPARK
>> AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
>> Progress"))
>> >
>> > The tag to be voted on is v2.4.7-rc1 (commit
>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>> > *https://github.com/apache/spark/tree/v2.4.7-rc1*
>> <https://github.com/apache/spark/tree/v2.4.7-rc1>
>> >
>> > The release files, including signatures, digests, etc. can be found
>> at:
>> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/*
>> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/>
>> >
>> > Signatures used for Spark RCs can be found in this file:
>> > *https://dist.apache.org/repos/dist/dev/spark/KEYS*
>> <https://dist.apache.org/repos/dist/dev/spark/KEYS>
>> >
>> > The staging repository for this release can be found at:
>> >
>> *https://repository.apache.org/content/repositories/orgapachespark-1352/*
>> <https://repository.apache.org/content/repositories/orgapachespark-1352/>
>> >
>> > The documentation corresponding to this release can be found at:
>> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/*
>> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/>
>> >
>> > The list of bug fixes going into 2.4.7 can be found at the
>> following URL:
>> > *https://s.apache.org/spark-v2.4.7-rc1*
>> <https://s.apache.org/spark-v2.4.7-rc1>
>> >
>> > This release is using the release script of the tag v2.4.7-rc1.
>> >
>> > FAQ
>> >
>> >
>> > =========================
>> > How can I help test this release?
>> > =========================
>> >
>> > If you are a Spark user, you can help us test this release by taking
>> > an existing Spark workload and running on this release candidate,
>> then
>> > reporting any regressions.
>> >
>> > If you're working in PySpark you can set up a virtual env and
>> install
>> > the current RC and see if anything important breaks, in the
>> Java/Scala
>> > you can add the staging repository to your projects resolvers and
>> test
>> > with the RC (make sure to clean up the artifact cache before/after
>> so
>> > you don't end up building with an out of date RC going forward).
>> >
>> > ===========================================
>> > What should happen to JIRA tickets still targeting 2.4.7?
>> > ===========================================
>> >
>> > The current list of open tickets targeted at 2.4.7 can be found at:
>> > *https://issues.apache.org/jira/projects/SPARK*
>> <https://issues.apache.org/jira/projects/SPARK> and search for
>> "Target Version/s" = 2.4.7
>> >
>> > Committers should look at those and triage. Extremely important bug
>> > fixes, documentation, and API tweaks that impact compatibility
>> should
>> > be worked on immediately. Everything else please retarget to an
>> > appropriate release.
>> >
>> > ==================
>> > But my bug isn't fixed?
>> > ==================
>> >
>> > In order to make timely releases, we will typically not hold the
>> > release unless the bug in question is a regression from the previous
>> > release. That being said, if there is something which is a
>> regression
>> > that has not been correctly targeted please ping me or a committer
>> to
>> > help target the issue.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: *dev-unsubscribe@spark.apache.org*
>> <de...@spark.apache.org>
>>
>>
>> --
>> Twitter: *https://twitter.com/holdenkarau*
>> <https://twitter.com/holdenkarau>
>> Books (Learning Spark, High Performance Spark, etc.): *https://amzn.to/2MaRAG9
>> * <https://amzn.to/2MaRAG9>
>> YouTube Live Streams: *https://www.youtube.com/user/holdenkarau*
>> <https://www.youtube.com/user/holdenkarau>
>>
>> --
>> ---
>> Takeshi Yamamuro
>>
>> --
>> <https://databricks.com/sparkaisummit/north-america>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
>>
>>
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Prashant Sharma <sc...@gmail.com>.
Hi All,
FYI, currently there are no blockers to release 2.4.7 at the moment. Soon
there will be RC3, I am awaiting some correctness bugs to be merged.
Thanks,
On Tue, Sep 1, 2020 at 6:43 PM Nicholas Marion <nm...@us.ibm.com> wrote:
> Hello,
>
> I see that a 2.4.7-rc2 tag was created over a week ago, wondering if there
> is an active vote going through for it?
>
>
> Regards,
>
> *NICHOLAS T. MARION *
> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
> ------------------------------
> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
> *Find me on:* [image: LinkedIn: http://www.linkedin.com/in/nicholasmarion]
> <http://www.linkedin.com/in/nicholasmarion>
> [image: IBM]
>
> 2455 South Rd
> Poughkeepie, New York 12601-5400
> United States
> [image: IBM Redbooks Silver Author][image: Data Science Foundations -
> Level 1]
>
>
> [image: Inactive hide details for Tom Graves ---08/21/2020 03:09:49 PM---
> There is a correctness issue with caching that should go into]Tom Graves
> ---08/21/2020 03:09:49 PM--- There is a correctness issue with caching that
> should go into this if possible: https://urldefense.
>
> From: Tom Graves <tg...@yahoo.com.INVALID>
> To: Nicholas Marion <nm...@us.ibm.com>, Wenchen Fan <cloud0fan@gmail.com
> >
> Cc: Xiao Li <li...@databricks.com>, dev <de...@spark.apache.org>, Takeshi
> Yamamuro <li...@gmail.com>, Prashant Sharma <sc...@gmail.com>
> Date: 08/21/2020 03:09 PM
> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
>
> ------------------------------
>
>
>
> There is a correctness issue with caching that should go into this if
> possible: *https://github.com/apache/spark/pull/29506*
> <https://github.com/apache/spark/pull/29506>
>
> Tom
>
> On Wednesday, August 19, 2020, 11:18:37 AM CDT, Wenchen Fan <
> cloud0fan@gmail.com> wrote:
>
>
> I think so. I don't see other bug reports for 2.4.
>
> On Thu, Aug 20, 2020 at 12:11 AM Nicholas Marion <*nmarion@us.ibm.com*
> <nm...@us.ibm.com>> wrote:
>
> It appears all 3 issues slated for Spark 2.4.7 have been merged.
> Should we be looking at getting RC2 ready?
>
>
>
> Regards,
>
> *NICHOLAS T. MARION *
> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
> ------------------------------
> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
> *Find me on:* <http://www.linkedin.com/in/nicholasmarion>
>
>
> 2455 South Rd
> Poughkeepie, New York 12601-5400
> United States
>
>
>
>
> [image: Inactive hide details for Xiao Li ---08/17/2020 11:33:30
> AM---INVALID URI REMOVED]Xiao Li ---08/17/2020 11:33:30 AM---
> *https://issues.apache.org/jira/browse/SPARK-32609*
> <https://issues.apache.org/jira/browse/SPARK-32609>
>
> From: Xiao Li <*lixiao@databricks.com* <li...@databricks.com>>
> To: Prashant Sharma <*scrapcodes@gmail.com* <sc...@gmail.com>>
> Cc: Takeshi Yamamuro <*linguin.m.s@gmail.com* <li...@gmail.com>>,
> dev <*dev@spark.apache.org* <de...@spark.apache.org>>
> Date: 08/17/2020 11:33 AM
> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
> ------------------------------
>
>
>
> *https://issues.apache.org/jira/browse/SPARK-32609*
> <https://issues.apache.org/jira/browse/SPARK-32609> got merged. This
> is to fix a correctness bug in DSV2 of Spark 2.4. Please include it in the
> upcoming Spark 2.4.7 release.
>
> Thanks,
>
> Xiao
>
> On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <*scrapcodes@gmail.com*
> <sc...@gmail.com>> wrote:
> Thanks for letting us know. So this vote is cancelled in favor of RC2.
>
>
>
> On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <
> *linguin.m.s@gmail.com* <li...@gmail.com>> wrote:
> Thanks for letting us know about the two issues above, Dongjoon.
>
> ----
> I've checked the release materials (signatures, tag, ...) and it looks
> fine, too.
> Also, I run the tests on my local Mac (java 1.8.0) with the options
> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
> -Psparkr`
> and they passed.
>
> Bests,
> Takeshi
>
>
>
> On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <
> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
> Another instance is SPARK-31703 which filed on May 13th and the PR
> arrived two days ago.
>
> [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on
> big endian platforms
> *https://github.com/apache/spark/pull/29383*
> <https://github.com/apache/spark/pull/29383>
>
> It seems that the patch is already ready in this case.
> I raised the priority of SPARK-31703 to `Blocker` for both Apache
> Spark 2.4.7 and 3.0.1.
>
> Bests,
> Dongjoon.
>
>
> On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <*holden@pigscanfly.ca*
> <ho...@pigscanfly.ca>> wrote:
> I'm going to go ahead and vote -0 then based on that then.
>
> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <
> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
> Hi, All.
>
> Unfortunately, there is an on-going discussion about the new decimal
> correctness.
>
> Although we fixed one correctness issue at master and backported it
> partially to 3.0/2.4, it turns out that it needs more patched to be
> complete.
>
> Please see *https://github.com/apache/spark/pull/29125*
> <https://github.com/apache/spark/pull/29125> for on-going discussion
> for both 3.0/2.4.
>
> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
> overflowed value
>
> I also confirmed that 2.4.7 RC1 is affected.
>
> Bests,
> Dongjoon.
>
>
> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <*srowen@apache.org*
> <sr...@apache.org>> wrote:
> +1 from me. The same as usual. Licenses and sigs look OK, builds and
> passes tests on a standard selection of profiles.
>
> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <*scrapcodes@gmail.com*
> <sc...@gmail.com>> wrote:
> >
> > Please vote on releasing the following candidate as Apache Spark
> version 2.4.7.
> >
> > The vote is open until Aug 9th at 9AM PST and passes if a majority
> +1 PMC votes are cast, with a minimum of 3 +1 votes.
> >
> > [ ] +1 Release this package as Apache Spark 2.4.7
> > [ ] -1 Do not release this package because ...
> >
> > To learn more about Apache Spark, please see
> *http://spark.apache.org/* <http://spark.apache.org/>
> >
> > There are currently no issues targeting 2.4.7 (try project = SPARK
> AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
> Progress"))
> >
> > The tag to be voted on is v2.4.7-rc1 (commit
> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
> > *https://github.com/apache/spark/tree/v2.4.7-rc1*
> <https://github.com/apache/spark/tree/v2.4.7-rc1>
> >
> > The release files, including signatures, digests, etc. can be found
> at:
> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/*
> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/>
> >
> > Signatures used for Spark RCs can be found in this file:
> > *https://dist.apache.org/repos/dist/dev/spark/KEYS*
> <https://dist.apache.org/repos/dist/dev/spark/KEYS>
> >
> > The staging repository for this release can be found at:
> >
> *https://repository.apache.org/content/repositories/orgapachespark-1352/*
> <https://repository.apache.org/content/repositories/orgapachespark-1352/>
> >
> > The documentation corresponding to this release can be found at:
> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/*
> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/>
> >
> > The list of bug fixes going into 2.4.7 can be found at the following
> URL:
> > *https://s.apache.org/spark-v2.4.7-rc1*
> <https://s.apache.org/spark-v2.4.7-rc1>
> >
> > This release is using the release script of the tag v2.4.7-rc1.
> >
> > FAQ
> >
> >
> > =========================
> > How can I help test this release?
> > =========================
> >
> > If you are a Spark user, you can help us test this release by taking
> > an existing Spark workload and running on this release candidate,
> then
> > reporting any regressions.
> >
> > If you're working in PySpark you can set up a virtual env and install
> > the current RC and see if anything important breaks, in the
> Java/Scala
> > you can add the staging repository to your projects resolvers and
> test
> > with the RC (make sure to clean up the artifact cache before/after so
> > you don't end up building with an out of date RC going forward).
> >
> > ===========================================
> > What should happen to JIRA tickets still targeting 2.4.7?
> > ===========================================
> >
> > The current list of open tickets targeted at 2.4.7 can be found at:
> > *https://issues.apache.org/jira/projects/SPARK*
> <https://issues.apache.org/jira/projects/SPARK> and search for "Target
> Version/s" = 2.4.7
> >
> > Committers should look at those and triage. Extremely important bug
> > fixes, documentation, and API tweaks that impact compatibility should
> > be worked on immediately. Everything else please retarget to an
> > appropriate release.
> >
> > ==================
> > But my bug isn't fixed?
> > ==================
> >
> > In order to make timely releases, we will typically not hold the
> > release unless the bug in question is a regression from the previous
> > release. That being said, if there is something which is a regression
> > that has not been correctly targeted please ping me or a committer to
> > help target the issue.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: *dev-unsubscribe@spark.apache.org*
> <de...@spark.apache.org>
>
>
> --
> Twitter: *https://twitter.com/holdenkarau*
> <https://twitter.com/holdenkarau>
> Books (Learning Spark, High Performance Spark, etc.): *https://amzn.to/2MaRAG9
> * <https://amzn.to/2MaRAG9>
> YouTube Live Streams: *https://www.youtube.com/user/holdenkarau*
> <https://www.youtube.com/user/holdenkarau>
>
> --
> ---
> Takeshi Yamamuro
>
> --
> <https://databricks.com/sparkaisummit/north-america>
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
>
>
RE: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Nicholas Marion <nm...@us.ibm.com>.
Hello,
I see that a 2.4.7-rc2 tag was created over a week ago, wondering if there
is an active vote going through for it?
Regards,
NICHOLAS T. MARION
IBM Open Data Analytics for z/OS - CPO and Service Team Lead
Phone: 1-845-433-5010 | Tie-Line: 293-5010 IBM
E-mail: nmarion@us.ibm.com
Find me on: LinkedIn: 2455 South Rd
http://www.linkedin.com/in/nicholasmarion Poughkeepie, New York 12601-5400
United States
IBM Redbooks Silver Author
Data Science Foundations - Level 1
From: Tom Graves <tg...@yahoo.com.INVALID>
To: Nicholas Marion <nm...@us.ibm.com>, Wenchen Fan
<cl...@gmail.com>
Cc: Xiao Li <li...@databricks.com>, dev <de...@spark.apache.org>,
Takeshi Yamamuro <li...@gmail.com>, Prashant Sharma
<sc...@gmail.com>
Date: 08/21/2020 03:09 PM
Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
There is a correctness issue with caching that should go into this if
possible: https://github.com/apache/spark/pull/29506
Tom
On Wednesday, August 19, 2020, 11:18:37 AM CDT, Wenchen Fan
<cl...@gmail.com> wrote:
I think so. I don't see other bug reports for 2.4.
On Thu, Aug 20, 2020 at 12:11 AM Nicholas Marion <nm...@us.ibm.com>
wrote:
It appears all 3 issues slated for Spark 2.4.7 have been merged. Should
we be looking at getting RC2 ready?
Regards,
NICHOLAS T. MARION
IBM Open Data Analytics for z/OS - CPO and Service Team Lead
Phone: 1-845-433-5010 | Tie-Line: 293-5010
E-mail: nmarion@us.ibm.com
Find me on: 2455 South Rd
Poughkeepie, New York
12601-5400
United States
Inactive hide details for Xiao Li ---08/17/2020 11:33:30 AM---INVALID URI
REMOVEDXiao Li ---08/17/2020 11:33:30 AM---
https://issues.apache.org/jira/browse/SPARK-32609
From: Xiao Li <li...@databricks.com>
To: Prashant Sharma <sc...@gmail.com>
Cc: Takeshi Yamamuro <li...@gmail.com>, dev <de...@spark.apache.org>
Date: 08/17/2020 11:33 AM
Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
https://issues.apache.org/jira/browse/SPARK-32609 got merged. This is to
fix a correctness bug in DSV2 of Spark 2.4. Please include it in the
upcoming Spark 2.4.7 release.
Thanks,
Xiao
On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <sc...@gmail.com>
wrote:
Thanks for letting us know. So this vote is cancelled in favor of RC2.
On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <li...@gmail.com>
wrote:
Thanks for letting us know about the two issues above, Dongjoon.
----
I've checked the release materials (signatures, tag, ...) and it looks
fine, too.
Also, I run the tests on my local Mac (java 1.8.0) with the options
`-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
-Psparkr`
and they passed.
Bests,
Takeshi
On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
wrote:
Another instance is SPARK-31703 which filed on May 13th and the PR
arrived two days ago.
[SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on
big endian platforms
https://github.com/apache/spark/pull/29383
It seems that the patch is already ready in this case.
I raised the priority of SPARK-31703 to `Blocker` for both Apache Spark
2.4.7 and 3.0.1.
Bests,
Dongjoon.
On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <ho...@pigscanfly.ca> wrote:
I'm going to go ahead and vote -0 then based on that then.
On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <do...@gmail.com>
wrote:
Hi, All.
Unfortunately, there is an on-going discussion about the new decimal
correctness.
Although we fixed one correctness issue at master and backported it
partially to 3.0/2.4, it turns out that it needs more patched to be
complete.
Please see https://github.com/apache/spark/pull/29125 for on-going
discussion for both 3.0/2.4.
[SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
overflowed value
I also confirmed that 2.4.7 RC1 is affected.
Bests,
Dongjoon.
On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
+1 from me. The same as usual. Licenses and sigs look OK, builds and
passes tests on a standard selection of profiles.
On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com>
wrote:
>
> Please vote on releasing the following candidate as Apache Spark
version 2.4.7.
>
> The vote is open until Aug 9th at 9AM PST and passes if a majority +1
PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 2.4.7
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> There are currently no issues targeting 2.4.7 (try project = SPARK AND
"Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
Progress"))
>
> The tag to be voted on is v2.4.7-rc1 (commit
dc04bf53fe821b7a07f817966c6c173f3b3788c6):
> https://github.com/apache/spark/tree/v2.4.7-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1352/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>
> The list of bug fixes going into 2.4.7 can be found at the following
URL:
> https://s.apache.org/spark-v2.4.7-rc1
>
> This release is using the release script of the tag v2.4.7-rc1.
>
> FAQ
>
>
> =========================
> How can I help test this release?
> =========================
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===========================================
> What should happen to JIRA tickets still targeting 2.4.7?
> ===========================================
>
> The current list of open tickets targeted at 2.4.7 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
Version/s" = 2.4.7
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==================
> But my bug isn't fixed?
> ==================
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
--
---
Takeshi Yamamuro
--
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Tom Graves <tg...@yahoo.com.INVALID>.
There is a correctness issue with caching that should go into this if possible: https://github.com/apache/spark/pull/29506
Tom
On Wednesday, August 19, 2020, 11:18:37 AM CDT, Wenchen Fan <cl...@gmail.com> wrote:
I think so. I don't see other bug reports for 2.4.
On Thu, Aug 20, 2020 at 12:11 AM Nicholas Marion <nm...@us.ibm.com> wrote:
It appears all 3 issues slated for Spark 2.4.7 have been merged. Should we be looking at getting RC2 ready?
| Regards,
NICHOLAS T. MARION
IBM Open Data Analytics for z/OS - CPO and Service Team Lead |
| |
| Phone: 1-845-433-5010 | Tie-Line: 293-5010
E-mail: nmarion@us.ibm.com
Find me on: |
2455 South Rd
Poughkeepie, New York 12601-5400
United States |
| |
| |
Xiao Li ---08/17/2020 11:33:30 AM---https://issues.apache.org/jira/browse/SPARK-32609
From: Xiao Li <li...@databricks.com>
To: Prashant Sharma <sc...@gmail.com>
Cc: Takeshi Yamamuro <li...@gmail.com>, dev <de...@spark.apache.org>
Date: 08/17/2020 11:33 AM
Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
https://issues.apache.org/jira/browse/SPARK-32609 got merged. This is to fix a correctness bug in DSV2 of Spark 2.4. Please include it in the upcoming Spark 2.4.7 release.
Thanks,
Xiao
On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <sc...@gmail.com> wrote:
Thanks for letting us know. So this vote is cancelled in favor of RC2.
On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <li...@gmail.com> wrote:
Thanks for letting us know about the two issues above, Dongjoon.
----
I've checked the release materials (signatures, tag, ...) and it looks fine, too.
Also, I run the tests on my local Mac (java 1.8.0) with the options
`-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes -Psparkr`
and they passed.
Bests,
Takeshi
On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <do...@gmail.com> wrote:
Another instance is SPARK-31703 which filed on May 13th and the PR arrived two days ago.
[SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms
https://github.com/apache/spark/pull/29383
It seems that the patch is already ready in this case.
I raised the priority of SPARK-31703 to `Blocker` for both Apache Spark 2.4.7 and 3.0.1.
Bests,
Dongjoon.
On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <ho...@pigscanfly.ca> wrote:
I'm going to go ahead and vote -0 then based on that then.
On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <do...@gmail.com> wrote:
Hi, All.
Unfortunately, there is an on-going discussion about the new decimal correctness.
Although we fixed one correctness issue at master and backported it partially to 3.0/2.4, it turns out that it needs more patched to be complete.
Please see https://github.com/apache/spark/pull/29125 for on-going discussion for both 3.0/2.4.
[SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with overflowed value
I also confirmed that 2.4.7 RC1 is affected.
Bests,
Dongjoon.
On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
+1 from me. The same as usual. Licenses and sigs look OK, builds and
passes tests on a standard selection of profiles.
On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com> wrote:
>
> Please vote on releasing the following candidate as Apache Spark version 2.4.7.
>
> The vote is open until Aug 9th at 9AM PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 2.4.7
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> There are currently no issues targeting 2.4.7 (try project = SPARK AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In Progress"))
>
> The tag to be voted on is v2.4.7-rc1 (commit dc04bf53fe821b7a07f817966c6c173f3b3788c6):
> https://github.com/apache/spark/tree/v2.4.7-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1352/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>
> The list of bug fixes going into 2.4.7 can be found at the following URL:
> https://s.apache.org/spark-v2.4.7-rc1
>
> This release is using the release script of the tag v2.4.7-rc1.
>
> FAQ
>
>
> =========================
> How can I help test this release?
> =========================
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===========================================
> What should happen to JIRA tickets still targeting 2.4.7?
> ===========================================
>
> The current list of open tickets targeted at 2.4.7 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target Version/s" = 2.4.7
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==================
> But my bug isn't fixed?
> ==================
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
--
---
Takeshi Yamamuro
--
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Wenchen Fan <cl...@gmail.com>.
I think so. I don't see other bug reports for 2.4.
On Thu, Aug 20, 2020 at 12:11 AM Nicholas Marion <nm...@us.ibm.com> wrote:
> It appears all 3 issues slated for Spark 2.4.7 have been merged. Should we
> be looking at getting RC2 ready?
>
>
> Regards,
>
> *NICHOLAS T. MARION *
> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
> ------------------------------
> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
> *E-mail:* *nmarion@us.ibm.com* <nm...@us.ibm.com>
> *Find me on:* <http://www.linkedin.com/in/nicholasmarion>
>
>
> 2455 South Rd
> Poughkeepie, New York 12601-5400
> United States
>
>
>
> [image: Inactive hide details for Xiao Li ---08/17/2020 11:33:30
> AM---INVALID URI REMOVED]Xiao Li ---08/17/2020 11:33:30 AM---
> https://issues.apache.org/jira/browse/SPARK-32609
>
> From: Xiao Li <li...@databricks.com>
> To: Prashant Sharma <sc...@gmail.com>
> Cc: Takeshi Yamamuro <li...@gmail.com>, dev <de...@spark.apache.org>
> Date: 08/17/2020 11:33 AM
> Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
>
> ------------------------------
>
>
>
> *https://issues.apache.org/jira/browse/SPARK-32609*
> <https://issues.apache.org/jira/browse/SPARK-32609> got merged. This is
> to fix a correctness bug in DSV2 of Spark 2.4. Please include it in the
> upcoming Spark 2.4.7 release.
>
> Thanks,
>
> Xiao
>
> On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <*scrapcodes@gmail.com*
> <sc...@gmail.com>> wrote:
>
> Thanks for letting us know. So this vote is cancelled in favor of RC2.
>
>
>
> On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <
> *linguin.m.s@gmail.com* <li...@gmail.com>> wrote:
> Thanks for letting us know about the two issues above, Dongjoon.
>
> ----
> I've checked the release materials (signatures, tag, ...) and it looks
> fine, too.
> Also, I run the tests on my local Mac (java 1.8.0) with the options
> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
> -Psparkr`
> and they passed.
>
> Bests,
> Takeshi
>
>
>
> On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <
> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
> Another instance is SPARK-31703 which filed on May 13th and the PR
> arrived two days ago.
>
> [SPARK-31703][SQL] Parquet RLE float/double are read
> incorrectly on big endian platforms
> *https://github.com/apache/spark/pull/29383*
> <https://github.com/apache/spark/pull/29383>
>
> It seems that the patch is already ready in this case.
> I raised the priority of SPARK-31703 to `Blocker` for both Apache
> Spark 2.4.7 and 3.0.1.
>
> Bests,
> Dongjoon.
>
>
> On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <*holden@pigscanfly.ca*
> <ho...@pigscanfly.ca>> wrote:
> I'm going to go ahead and vote -0 then based on that then.
>
> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <
> *dongjoon.hyun@gmail.com* <do...@gmail.com>> wrote:
> Hi, All.
>
> Unfortunately, there is an on-going discussion about the new
> decimal correctness.
>
> Although we fixed one correctness issue at master and backported
> it partially to 3.0/2.4, it turns out that it needs more patched to be
> complete.
>
> Please see *https://github.com/apache/spark/pull/29125*
> <https://github.com/apache/spark/pull/29125> for on-going
> discussion for both 3.0/2.4.
>
> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null
> with overflowed value
>
> I also confirmed that 2.4.7 RC1 is affected.
>
> Bests,
> Dongjoon.
>
>
> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <*srowen@apache.org*
> <sr...@apache.org>> wrote:
> +1 from me. The same as usual. Licenses and sigs look OK, builds
> and
> passes tests on a standard selection of profiles.
>
> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <
> *scrapcodes@gmail.com* <sc...@gmail.com>> wrote:
> >
> > Please vote on releasing the following candidate as Apache
> Spark version 2.4.7.
> >
> > The vote is open until Aug 9th at 9AM PST and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
> >
> > [ ] +1 Release this package as Apache Spark 2.4.7
> > [ ] -1 Do not release this package because ...
> >
> > To learn more about Apache Spark, please see
> *http://spark.apache.org/* <http://spark.apache.org/>
> >
> > There are currently no issues targeting 2.4.7 (try project =
> SPARK AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
> Progress"))
> >
> > The tag to be voted on is v2.4.7-rc1 (commit
> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
> > *https://github.com/apache/spark/tree/v2.4.7-rc1*
> <https://github.com/apache/spark/tree/v2.4.7-rc1>
> >
> > The release files, including signatures, digests, etc. can be
> found at:
> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/*
> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/>
> >
> > Signatures used for Spark RCs can be found in this file:
> > *https://dist.apache.org/repos/dist/dev/spark/KEYS*
> <https://dist.apache.org/repos/dist/dev/spark/KEYS>
> >
> > The staging repository for this release can be found at:
> >
> *https://repository.apache.org/content/repositories/orgapachespark-1352/*
> <https://repository.apache.org/content/repositories/orgapachespark-1352/>
> >
> > The documentation corresponding to this release can be found
> at:
> > *https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/*
> <https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/>
> >
> > The list of bug fixes going into 2.4.7 can be found at the
> following URL:
> > *https://s.apache.org/spark-v2.4.7-rc1*
> <https://s.apache.org/spark-v2.4.7-rc1>
> >
> > This release is using the release script of the tag v2.4.7-rc1.
> >
> > FAQ
> >
> >
> > =========================
> > How can I help test this release?
> > =========================
> >
> > If you are a Spark user, you can help us test this release by
> taking
> > an existing Spark workload and running on this release
> candidate, then
> > reporting any regressions.
> >
> > If you're working in PySpark you can set up a virtual env and
> install
> > the current RC and see if anything important breaks, in the
> Java/Scala
> > you can add the staging repository to your projects resolvers
> and test
> > with the RC (make sure to clean up the artifact cache
> before/after so
> > you don't end up building with an out of date RC going
> forward).
> >
> > ===========================================
> > What should happen to JIRA tickets still targeting 2.4.7?
> > ===========================================
> >
> > The current list of open tickets targeted at 2.4.7 can be
> found at:
> > *https://issues.apache.org/jira/projects/SPARK*
> <https://issues.apache.org/jira/projects/SPARK> and search for
> "Target Version/s" = 2.4.7
> >
> > Committers should look at those and triage. Extremely
> important bug
> > fixes, documentation, and API tweaks that impact compatibility
> should
> > be worked on immediately. Everything else please retarget to an
> > appropriate release.
> >
> > ==================
> > But my bug isn't fixed?
> > ==================
> >
> > In order to make timely releases, we will typically not hold
> the
> > release unless the bug in question is a regression from the
> previous
> > release. That being said, if there is something which is a
> regression
> > that has not been correctly targeted please ping me or a
> committer to
> > help target the issue.
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: *dev-unsubscribe@spark.apache.org*
> <de...@spark.apache.org>
>
>
> --
> Twitter: *https://twitter.com/holdenkarau*
> <https://twitter.com/holdenkarau>
> Books (Learning Spark, High Performance Spark, etc.):
> *https://amzn.to/2MaRAG9 * <https://amzn.to/2MaRAG9>
> YouTube Live Streams: *https://www.youtube.com/user/holdenkarau*
> <https://www.youtube.com/user/holdenkarau>
>
>
> --
> ---
> Takeshi Yamamuro
>
>
>
> --
> <https://databricks.com/sparkaisummit/north-america>
>
>
>
RE: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Nicholas Marion <nm...@us.ibm.com>.
It appears all 3 issues slated for Spark 2.4.7 have been merged. Should we
be looking at getting RC2 ready?
Regards,
NICHOLAS T. MARION
IBM Open Data Analytics for z/OS - CPO and Service Team Lead
Phone: 1-845-433-5010 | Tie-Line: 293-5010
E-mail: nmarion@us.ibm.com
Find me on: 2455 South Rd
Poughkeepie, New York 12601-5400
United States
From: Xiao Li <li...@databricks.com>
To: Prashant Sharma <sc...@gmail.com>
Cc: Takeshi Yamamuro <li...@gmail.com>, dev
<de...@spark.apache.org>
Date: 08/17/2020 11:33 AM
Subject: [EXTERNAL] Re: [VOTE] Release Spark 2.4.7 (RC1)
https://issues.apache.org/jira/browse/SPARK-32609 got merged. This is to
fix a correctness bug in DSV2 of Spark 2.4. Please include it in the
upcoming Spark 2.4.7 release.
Thanks,
Xiao
On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <sc...@gmail.com>
wrote:
Thanks for letting us know. So this vote is cancelled in favor of RC2.
On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <li...@gmail.com>
wrote:
Thanks for letting us know about the two issues above, Dongjoon.
----
I've checked the release materials (signatures, tag, ...) and it looks
fine, too.
Also, I run the tests on my local Mac (java 1.8.0) with the options
`-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
-Psparkr`
and they passed.
Bests,
Takeshi
On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
wrote:
Another instance is SPARK-31703 which filed on May 13th and the PR
arrived two days ago.
[SPARK-31703][SQL] Parquet RLE float/double are read incorrectly
on big endian platforms
https://github.com/apache/spark/pull/29383
It seems that the patch is already ready in this case.
I raised the priority of SPARK-31703 to `Blocker` for both Apache
Spark 2.4.7 and 3.0.1.
Bests,
Dongjoon.
On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <ho...@pigscanfly.ca>
wrote:
I'm going to go ahead and vote -0 then based on that then.
On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <
dongjoon.hyun@gmail.com> wrote:
Hi, All.
Unfortunately, there is an on-going discussion about the new
decimal correctness.
Although we fixed one correctness issue at master and backported it
partially to 3.0/2.4, it turns out that it needs more patched to be
complete.
Please see https://github.com/apache/spark/pull/29125 for on-going
discussion for both 3.0/2.4.
[SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null
with overflowed value
I also confirmed that 2.4.7 RC1 is affected.
Bests,
Dongjoon.
On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
+1 from me. The same as usual. Licenses and sigs look OK, builds
and
passes tests on a standard selection of profiles.
On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <
scrapcodes@gmail.com> wrote:
>
> Please vote on releasing the following candidate as Apache
Spark version 2.4.7.
>
> The vote is open until Aug 9th at 9AM PST and passes if a
majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 2.4.7
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see
http://spark.apache.org/
>
> There are currently no issues targeting 2.4.7 (try project =
SPARK AND "Target Version/s" = "2.4.7" AND status in (Open,
Reopened, "In Progress"))
>
> The tag to be voted on is v2.4.7-rc1 (commit
dc04bf53fe821b7a07f817966c6c173f3b3788c6):
> https://github.com/apache/spark/tree/v2.4.7-rc1
>
> The release files, including signatures, digests, etc. can be
found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
>
https://repository.apache.org/content/repositories/orgapachespark-1352/
>
> The documentation corresponding to this release can be found
at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>
> The list of bug fixes going into 2.4.7 can be found at the
following URL:
> https://s.apache.org/spark-v2.4.7-rc1
>
> This release is using the release script of the tag v2.4.7-rc1.
>
> FAQ
>
>
> =========================
> How can I help test this release?
> =========================
>
> If you are a Spark user, you can help us test this release by
taking
> an existing Spark workload and running on this release
candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and
install
> the current RC and see if anything important breaks, in the
Java/Scala
> you can add the staging repository to your projects resolvers
and test
> with the RC (make sure to clean up the artifact cache
before/after so
> you don't end up building with an out of date RC going
forward).
>
> ===========================================
> What should happen to JIRA tickets still targeting 2.4.7?
> ===========================================
>
> The current list of open tickets targeted at 2.4.7 can be found
at:
> https://issues.apache.org/jira/projects/SPARK and search for
"Target Version/s" = 2.4.7
>
> Committers should look at those and triage. Extremely important
bug
> fixes, documentation, and API tweaks that impact compatibility
should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==================
> But my bug isn't fixed?
> ==================
>
> In order to make timely releases, we will typically not hold
the
> release unless the bug in question is a regression from the
previous
> release. That being said, if there is something which is a
regression
> that has not been correctly targeted please ping me or a
committer to
> help target the issue.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
--
---
Takeshi Yamamuro
--
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Xiao Li <li...@databricks.com>.
https://issues.apache.org/jira/browse/SPARK-32609 got merged. This is to
fix a correctness bug in DSV2 of Spark 2.4. Please include it in the
upcoming Spark 2.4.7 release.
Thanks,
Xiao
On Sun, Aug 9, 2020 at 10:26 PM Prashant Sharma <sc...@gmail.com>
wrote:
> Thanks for letting us know. So this vote is cancelled in favor of RC2.
>
>
>
> On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <li...@gmail.com>
> wrote:
>
>> Thanks for letting us know about the two issues above, Dongjoon.
>>
>> ----
>> I've checked the release materials (signatures, tag, ...) and it looks
>> fine, too.
>> Also, I run the tests on my local Mac (java 1.8.0) with the options
>> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
>> -Psparkr`
>> and they passed.
>>
>> Bests,
>> Takeshi
>>
>>
>>
>> On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> Another instance is SPARK-31703 which filed on May 13th and the PR
>>> arrived two days ago.
>>>
>>> [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on
>>> big endian platforms
>>> https://github.com/apache/spark/pull/29383
>>>
>>> It seems that the patch is already ready in this case.
>>> I raised the priority of SPARK-31703 to `Blocker` for both Apache Spark
>>> 2.4.7 and 3.0.1.
>>>
>>> Bests,
>>> Dongjoon.
>>>
>>>
>>> On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <ho...@pigscanfly.ca>
>>> wrote:
>>>
>>>> I'm going to go ahead and vote -0 then based on that then.
>>>>
>>>> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <do...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi, All.
>>>>>
>>>>> Unfortunately, there is an on-going discussion about the new decimal
>>>>> correctness.
>>>>>
>>>>> Although we fixed one correctness issue at master and backported it
>>>>> partially to 3.0/2.4, it turns out that it needs more patched to be
>>>>> complete.
>>>>>
>>>>> Please see https://github.com/apache/spark/pull/29125 for on-going
>>>>> discussion for both 3.0/2.4.
>>>>>
>>>>> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
>>>>> overflowed value
>>>>>
>>>>> I also confirmed that 2.4.7 RC1 is affected.
>>>>>
>>>>> Bests,
>>>>> Dongjoon.
>>>>>
>>>>>
>>>>> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
>>>>>
>>>>>> +1 from me. The same as usual. Licenses and sigs look OK, builds and
>>>>>> passes tests on a standard selection of profiles.
>>>>>>
>>>>>> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > Please vote on releasing the following candidate as Apache Spark
>>>>>> version 2.4.7.
>>>>>> >
>>>>>> > The vote is open until Aug 9th at 9AM PST and passes if a majority
>>>>>> +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>>> >
>>>>>> > [ ] +1 Release this package as Apache Spark 2.4.7
>>>>>> > [ ] -1 Do not release this package because ...
>>>>>> >
>>>>>> > To learn more about Apache Spark, please see
>>>>>> http://spark.apache.org/
>>>>>> >
>>>>>> > There are currently no issues targeting 2.4.7 (try project = SPARK
>>>>>> AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
>>>>>> Progress"))
>>>>>> >
>>>>>> > The tag to be voted on is v2.4.7-rc1 (commit
>>>>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>>>>>> > https://github.com/apache/spark/tree/v2.4.7-rc1
>>>>>> >
>>>>>> > The release files, including signatures, digests, etc. can be found
>>>>>> at:
>>>>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>>>>>> >
>>>>>> > Signatures used for Spark RCs can be found in this file:
>>>>>> > https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>> >
>>>>>> > The staging repository for this release can be found at:
>>>>>> >
>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1352/
>>>>>> >
>>>>>> > The documentation corresponding to this release can be found at:
>>>>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>>>>>> >
>>>>>> > The list of bug fixes going into 2.4.7 can be found at the
>>>>>> following URL:
>>>>>> > https://s.apache.org/spark-v2.4.7-rc1
>>>>>> >
>>>>>> > This release is using the release script of the tag v2.4.7-rc1.
>>>>>> >
>>>>>> > FAQ
>>>>>> >
>>>>>> >
>>>>>> > =========================
>>>>>> > How can I help test this release?
>>>>>> > =========================
>>>>>> >
>>>>>> > If you are a Spark user, you can help us test this release by taking
>>>>>> > an existing Spark workload and running on this release candidate,
>>>>>> then
>>>>>> > reporting any regressions.
>>>>>> >
>>>>>> > If you're working in PySpark you can set up a virtual env and
>>>>>> install
>>>>>> > the current RC and see if anything important breaks, in the
>>>>>> Java/Scala
>>>>>> > you can add the staging repository to your projects resolvers and
>>>>>> test
>>>>>> > with the RC (make sure to clean up the artifact cache before/after
>>>>>> so
>>>>>> > you don't end up building with an out of date RC going forward).
>>>>>> >
>>>>>> > ===========================================
>>>>>> > What should happen to JIRA tickets still targeting 2.4.7?
>>>>>> > ===========================================
>>>>>> >
>>>>>> > The current list of open tickets targeted at 2.4.7 can be found at:
>>>>>> > https://issues.apache.org/jira/projects/SPARK and search for
>>>>>> "Target Version/s" = 2.4.7
>>>>>> >
>>>>>> > Committers should look at those and triage. Extremely important bug
>>>>>> > fixes, documentation, and API tweaks that impact compatibility
>>>>>> should
>>>>>> > be worked on immediately. Everything else please retarget to an
>>>>>> > appropriate release.
>>>>>> >
>>>>>> > ==================
>>>>>> > But my bug isn't fixed?
>>>>>> > ==================
>>>>>> >
>>>>>> > In order to make timely releases, we will typically not hold the
>>>>>> > release unless the bug in question is a regression from the previous
>>>>>> > release. That being said, if there is something which is a
>>>>>> regression
>>>>>> > that has not been correctly targeted please ping me or a committer
>>>>>> to
>>>>>> > help target the issue.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>
>>>>>>
>>>>
>>>> --
>>>> Twitter: https://twitter.com/holdenkarau
>>>> Books (Learning Spark, High Performance Spark, etc.):
>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>
>>>
>>
>> --
>> ---
>> Takeshi Yamamuro
>>
>
--
<https://databricks.com/sparkaisummit/north-america>
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Prashant Sharma <sc...@gmail.com>.
Thanks for letting us know. So this vote is cancelled in favor of RC2.
On Sun, Aug 9, 2020 at 8:31 AM Takeshi Yamamuro <li...@gmail.com>
wrote:
> Thanks for letting us know about the two issues above, Dongjoon.
>
> ----
> I've checked the release materials (signatures, tag, ...) and it looks
> fine, too.
> Also, I run the tests on my local Mac (java 1.8.0) with the options
> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
> -Psparkr`
> and they passed.
>
> Bests,
> Takeshi
>
>
>
> On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> Another instance is SPARK-31703 which filed on May 13th and the PR
>> arrived two days ago.
>>
>> [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on
>> big endian platforms
>> https://github.com/apache/spark/pull/29383
>>
>> It seems that the patch is already ready in this case.
>> I raised the priority of SPARK-31703 to `Blocker` for both Apache Spark
>> 2.4.7 and 3.0.1.
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <ho...@pigscanfly.ca> wrote:
>>
>>> I'm going to go ahead and vote -0 then based on that then.
>>>
>>> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <do...@gmail.com>
>>> wrote:
>>>
>>>> Hi, All.
>>>>
>>>> Unfortunately, there is an on-going discussion about the new decimal
>>>> correctness.
>>>>
>>>> Although we fixed one correctness issue at master and backported it
>>>> partially to 3.0/2.4, it turns out that it needs more patched to be
>>>> complete.
>>>>
>>>> Please see https://github.com/apache/spark/pull/29125 for on-going
>>>> discussion for both 3.0/2.4.
>>>>
>>>> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
>>>> overflowed value
>>>>
>>>> I also confirmed that 2.4.7 RC1 is affected.
>>>>
>>>> Bests,
>>>> Dongjoon.
>>>>
>>>>
>>>> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
>>>>
>>>>> +1 from me. The same as usual. Licenses and sigs look OK, builds and
>>>>> passes tests on a standard selection of profiles.
>>>>>
>>>>> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> > Please vote on releasing the following candidate as Apache Spark
>>>>> version 2.4.7.
>>>>> >
>>>>> > The vote is open until Aug 9th at 9AM PST and passes if a majority
>>>>> +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>> >
>>>>> > [ ] +1 Release this package as Apache Spark 2.4.7
>>>>> > [ ] -1 Do not release this package because ...
>>>>> >
>>>>> > To learn more about Apache Spark, please see
>>>>> http://spark.apache.org/
>>>>> >
>>>>> > There are currently no issues targeting 2.4.7 (try project = SPARK
>>>>> AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
>>>>> Progress"))
>>>>> >
>>>>> > The tag to be voted on is v2.4.7-rc1 (commit
>>>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>>>>> > https://github.com/apache/spark/tree/v2.4.7-rc1
>>>>> >
>>>>> > The release files, including signatures, digests, etc. can be found
>>>>> at:
>>>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>>>>> >
>>>>> > Signatures used for Spark RCs can be found in this file:
>>>>> > https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>> >
>>>>> > The staging repository for this release can be found at:
>>>>> >
>>>>> https://repository.apache.org/content/repositories/orgapachespark-1352/
>>>>> >
>>>>> > The documentation corresponding to this release can be found at:
>>>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>>>>> >
>>>>> > The list of bug fixes going into 2.4.7 can be found at the following
>>>>> URL:
>>>>> > https://s.apache.org/spark-v2.4.7-rc1
>>>>> >
>>>>> > This release is using the release script of the tag v2.4.7-rc1.
>>>>> >
>>>>> > FAQ
>>>>> >
>>>>> >
>>>>> > =========================
>>>>> > How can I help test this release?
>>>>> > =========================
>>>>> >
>>>>> > If you are a Spark user, you can help us test this release by taking
>>>>> > an existing Spark workload and running on this release candidate,
>>>>> then
>>>>> > reporting any regressions.
>>>>> >
>>>>> > If you're working in PySpark you can set up a virtual env and install
>>>>> > the current RC and see if anything important breaks, in the
>>>>> Java/Scala
>>>>> > you can add the staging repository to your projects resolvers and
>>>>> test
>>>>> > with the RC (make sure to clean up the artifact cache before/after so
>>>>> > you don't end up building with an out of date RC going forward).
>>>>> >
>>>>> > ===========================================
>>>>> > What should happen to JIRA tickets still targeting 2.4.7?
>>>>> > ===========================================
>>>>> >
>>>>> > The current list of open tickets targeted at 2.4.7 can be found at:
>>>>> > https://issues.apache.org/jira/projects/SPARK and search for
>>>>> "Target Version/s" = 2.4.7
>>>>> >
>>>>> > Committers should look at those and triage. Extremely important bug
>>>>> > fixes, documentation, and API tweaks that impact compatibility should
>>>>> > be worked on immediately. Everything else please retarget to an
>>>>> > appropriate release.
>>>>> >
>>>>> > ==================
>>>>> > But my bug isn't fixed?
>>>>> > ==================
>>>>> >
>>>>> > In order to make timely releases, we will typically not hold the
>>>>> > release unless the bug in question is a regression from the previous
>>>>> > release. That being said, if there is something which is a regression
>>>>> > that has not been correctly targeted please ping me or a committer to
>>>>> > help target the issue.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>
>>>>>
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>
>
> --
> ---
> Takeshi Yamamuro
>
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Takeshi Yamamuro <li...@gmail.com>.
Thanks for letting us know about the two issues above, Dongjoon.
----
I've checked the release materials (signatures, tag, ...) and it looks
fine, too.
Also, I run the tests on my local Mac (java 1.8.0) with the options
`-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
-Psparkr`
and they passed.
Bests,
Takeshi
On Sun, Aug 9, 2020 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
wrote:
> Another instance is SPARK-31703 which filed on May 13th and the PR arrived
> two days ago.
>
> [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on
> big endian platforms
> https://github.com/apache/spark/pull/29383
>
> It seems that the patch is already ready in this case.
> I raised the priority of SPARK-31703 to `Blocker` for both Apache Spark
> 2.4.7 and 3.0.1.
>
> Bests,
> Dongjoon.
>
>
> On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <ho...@pigscanfly.ca> wrote:
>
>> I'm going to go ahead and vote -0 then based on that then.
>>
>> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> Hi, All.
>>>
>>> Unfortunately, there is an on-going discussion about the new decimal
>>> correctness.
>>>
>>> Although we fixed one correctness issue at master and backported it
>>> partially to 3.0/2.4, it turns out that it needs more patched to be
>>> complete.
>>>
>>> Please see https://github.com/apache/spark/pull/29125 for on-going
>>> discussion for both 3.0/2.4.
>>>
>>> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
>>> overflowed value
>>>
>>> I also confirmed that 2.4.7 RC1 is affected.
>>>
>>> Bests,
>>> Dongjoon.
>>>
>>>
>>> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
>>>
>>>> +1 from me. The same as usual. Licenses and sigs look OK, builds and
>>>> passes tests on a standard selection of profiles.
>>>>
>>>> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com>
>>>> wrote:
>>>> >
>>>> > Please vote on releasing the following candidate as Apache Spark
>>>> version 2.4.7.
>>>> >
>>>> > The vote is open until Aug 9th at 9AM PST and passes if a majority +1
>>>> PMC votes are cast, with a minimum of 3 +1 votes.
>>>> >
>>>> > [ ] +1 Release this package as Apache Spark 2.4.7
>>>> > [ ] -1 Do not release this package because ...
>>>> >
>>>> > To learn more about Apache Spark, please see http://spark.apache.org/
>>>> >
>>>> > There are currently no issues targeting 2.4.7 (try project = SPARK
>>>> AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In
>>>> Progress"))
>>>> >
>>>> > The tag to be voted on is v2.4.7-rc1 (commit
>>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>>>> > https://github.com/apache/spark/tree/v2.4.7-rc1
>>>> >
>>>> > The release files, including signatures, digests, etc. can be found
>>>> at:
>>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>>>> >
>>>> > Signatures used for Spark RCs can be found in this file:
>>>> > https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>> >
>>>> > The staging repository for this release can be found at:
>>>> >
>>>> https://repository.apache.org/content/repositories/orgapachespark-1352/
>>>> >
>>>> > The documentation corresponding to this release can be found at:
>>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>>>> >
>>>> > The list of bug fixes going into 2.4.7 can be found at the following
>>>> URL:
>>>> > https://s.apache.org/spark-v2.4.7-rc1
>>>> >
>>>> > This release is using the release script of the tag v2.4.7-rc1.
>>>> >
>>>> > FAQ
>>>> >
>>>> >
>>>> > =========================
>>>> > How can I help test this release?
>>>> > =========================
>>>> >
>>>> > If you are a Spark user, you can help us test this release by taking
>>>> > an existing Spark workload and running on this release candidate, then
>>>> > reporting any regressions.
>>>> >
>>>> > If you're working in PySpark you can set up a virtual env and install
>>>> > the current RC and see if anything important breaks, in the Java/Scala
>>>> > you can add the staging repository to your projects resolvers and test
>>>> > with the RC (make sure to clean up the artifact cache before/after so
>>>> > you don't end up building with an out of date RC going forward).
>>>> >
>>>> > ===========================================
>>>> > What should happen to JIRA tickets still targeting 2.4.7?
>>>> > ===========================================
>>>> >
>>>> > The current list of open tickets targeted at 2.4.7 can be found at:
>>>> > https://issues.apache.org/jira/projects/SPARK and search for "Target
>>>> Version/s" = 2.4.7
>>>> >
>>>> > Committers should look at those and triage. Extremely important bug
>>>> > fixes, documentation, and API tweaks that impact compatibility should
>>>> > be worked on immediately. Everything else please retarget to an
>>>> > appropriate release.
>>>> >
>>>> > ==================
>>>> > But my bug isn't fixed?
>>>> > ==================
>>>> >
>>>> > In order to make timely releases, we will typically not hold the
>>>> > release unless the bug in question is a regression from the previous
>>>> > release. That being said, if there is something which is a regression
>>>> > that has not been correctly targeted please ping me or a committer to
>>>> > help target the issue.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>
>>>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
--
---
Takeshi Yamamuro
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Dongjoon Hyun <do...@gmail.com>.
Another instance is SPARK-31703 which filed on May 13th and the PR arrived
two days ago.
[SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big
endian platforms
https://github.com/apache/spark/pull/29383
It seems that the patch is already ready in this case.
I raised the priority of SPARK-31703 to `Blocker` for both Apache Spark
2.4.7 and 3.0.1.
Bests,
Dongjoon.
On Sat, Aug 8, 2020 at 6:10 AM Holden Karau <ho...@pigscanfly.ca> wrote:
> I'm going to go ahead and vote -0 then based on that then.
>
> On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> Hi, All.
>>
>> Unfortunately, there is an on-going discussion about the new decimal
>> correctness.
>>
>> Although we fixed one correctness issue at master and backported it
>> partially to 3.0/2.4, it turns out that it needs more patched to be
>> complete.
>>
>> Please see https://github.com/apache/spark/pull/29125 for on-going
>> discussion for both 3.0/2.4.
>>
>> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
>> overflowed value
>>
>> I also confirmed that 2.4.7 RC1 is affected.
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
>>
>>> +1 from me. The same as usual. Licenses and sigs look OK, builds and
>>> passes tests on a standard selection of profiles.
>>>
>>> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com>
>>> wrote:
>>> >
>>> > Please vote on releasing the following candidate as Apache Spark
>>> version 2.4.7.
>>> >
>>> > The vote is open until Aug 9th at 9AM PST and passes if a majority +1
>>> PMC votes are cast, with a minimum of 3 +1 votes.
>>> >
>>> > [ ] +1 Release this package as Apache Spark 2.4.7
>>> > [ ] -1 Do not release this package because ...
>>> >
>>> > To learn more about Apache Spark, please see http://spark.apache.org/
>>> >
>>> > There are currently no issues targeting 2.4.7 (try project = SPARK AND
>>> "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In Progress"))
>>> >
>>> > The tag to be voted on is v2.4.7-rc1 (commit
>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>>> > https://github.com/apache/spark/tree/v2.4.7-rc1
>>> >
>>> > The release files, including signatures, digests, etc. can be found at:
>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>>> >
>>> > Signatures used for Spark RCs can be found in this file:
>>> > https://dist.apache.org/repos/dist/dev/spark/KEYS
>>> >
>>> > The staging repository for this release can be found at:
>>> >
>>> https://repository.apache.org/content/repositories/orgapachespark-1352/
>>> >
>>> > The documentation corresponding to this release can be found at:
>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>>> >
>>> > The list of bug fixes going into 2.4.7 can be found at the following
>>> URL:
>>> > https://s.apache.org/spark-v2.4.7-rc1
>>> >
>>> > This release is using the release script of the tag v2.4.7-rc1.
>>> >
>>> > FAQ
>>> >
>>> >
>>> > =========================
>>> > How can I help test this release?
>>> > =========================
>>> >
>>> > If you are a Spark user, you can help us test this release by taking
>>> > an existing Spark workload and running on this release candidate, then
>>> > reporting any regressions.
>>> >
>>> > If you're working in PySpark you can set up a virtual env and install
>>> > the current RC and see if anything important breaks, in the Java/Scala
>>> > you can add the staging repository to your projects resolvers and test
>>> > with the RC (make sure to clean up the artifact cache before/after so
>>> > you don't end up building with an out of date RC going forward).
>>> >
>>> > ===========================================
>>> > What should happen to JIRA tickets still targeting 2.4.7?
>>> > ===========================================
>>> >
>>> > The current list of open tickets targeted at 2.4.7 can be found at:
>>> > https://issues.apache.org/jira/projects/SPARK and search for "Target
>>> Version/s" = 2.4.7
>>> >
>>> > Committers should look at those and triage. Extremely important bug
>>> > fixes, documentation, and API tweaks that impact compatibility should
>>> > be worked on immediately. Everything else please retarget to an
>>> > appropriate release.
>>> >
>>> > ==================
>>> > But my bug isn't fixed?
>>> > ==================
>>> >
>>> > In order to make timely releases, we will typically not hold the
>>> > release unless the bug in question is a regression from the previous
>>> > release. That being said, if there is something which is a regression
>>> > that has not been correctly targeted please ping me or a committer to
>>> > help target the issue.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Holden Karau <ho...@pigscanfly.ca>.
I'm going to go ahead and vote -0 then based on that then.
On Fri, Aug 7, 2020 at 11:36 PM Dongjoon Hyun <do...@gmail.com>
wrote:
> Hi, All.
>
> Unfortunately, there is an on-going discussion about the new decimal
> correctness.
>
> Although we fixed one correctness issue at master and backported it
> partially to 3.0/2.4, it turns out that it needs more patched to be
> complete.
>
> Please see https://github.com/apache/spark/pull/29125 for on-going
> discussion for both 3.0/2.4.
>
> [SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
> overflowed value
>
> I also confirmed that 2.4.7 RC1 is affected.
>
> Bests,
> Dongjoon.
>
>
> On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
>
>> +1 from me. The same as usual. Licenses and sigs look OK, builds and
>> passes tests on a standard selection of profiles.
>>
>> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com>
>> wrote:
>> >
>> > Please vote on releasing the following candidate as Apache Spark
>> version 2.4.7.
>> >
>> > The vote is open until Aug 9th at 9AM PST and passes if a majority +1
>> PMC votes are cast, with a minimum of 3 +1 votes.
>> >
>> > [ ] +1 Release this package as Apache Spark 2.4.7
>> > [ ] -1 Do not release this package because ...
>> >
>> > To learn more about Apache Spark, please see http://spark.apache.org/
>> >
>> > There are currently no issues targeting 2.4.7 (try project = SPARK AND
>> "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In Progress"))
>> >
>> > The tag to be voted on is v2.4.7-rc1 (commit
>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>> > https://github.com/apache/spark/tree/v2.4.7-rc1
>> >
>> > The release files, including signatures, digests, etc. can be found at:
>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>> >
>> > Signatures used for Spark RCs can be found in this file:
>> > https://dist.apache.org/repos/dist/dev/spark/KEYS
>> >
>> > The staging repository for this release can be found at:
>> > https://repository.apache.org/content/repositories/orgapachespark-1352/
>> >
>> > The documentation corresponding to this release can be found at:
>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>> >
>> > The list of bug fixes going into 2.4.7 can be found at the following
>> URL:
>> > https://s.apache.org/spark-v2.4.7-rc1
>> >
>> > This release is using the release script of the tag v2.4.7-rc1.
>> >
>> > FAQ
>> >
>> >
>> > =========================
>> > How can I help test this release?
>> > =========================
>> >
>> > If you are a Spark user, you can help us test this release by taking
>> > an existing Spark workload and running on this release candidate, then
>> > reporting any regressions.
>> >
>> > If you're working in PySpark you can set up a virtual env and install
>> > the current RC and see if anything important breaks, in the Java/Scala
>> > you can add the staging repository to your projects resolvers and test
>> > with the RC (make sure to clean up the artifact cache before/after so
>> > you don't end up building with an out of date RC going forward).
>> >
>> > ===========================================
>> > What should happen to JIRA tickets still targeting 2.4.7?
>> > ===========================================
>> >
>> > The current list of open tickets targeted at 2.4.7 can be found at:
>> > https://issues.apache.org/jira/projects/SPARK and search for "Target
>> Version/s" = 2.4.7
>> >
>> > Committers should look at those and triage. Extremely important bug
>> > fixes, documentation, and API tweaks that impact compatibility should
>> > be worked on immediately. Everything else please retarget to an
>> > appropriate release.
>> >
>> > ==================
>> > But my bug isn't fixed?
>> > ==================
>> >
>> > In order to make timely releases, we will typically not hold the
>> > release unless the bug in question is a regression from the previous
>> > release. That being said, if there is something which is a regression
>> > that has not been correctly targeted please ping me or a committer to
>> > help target the issue.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Dongjoon Hyun <do...@gmail.com>.
Hi, All.
Unfortunately, there is an on-going discussion about the new decimal
correctness.
Although we fixed one correctness issue at master and backported it
partially to 3.0/2.4, it turns out that it needs more patched to be
complete.
Please see https://github.com/apache/spark/pull/29125 for on-going
discussion for both 3.0/2.4.
[SPARK-32018][SQL][3.0] UnsafeRow.setDecimal should set null with
overflowed value
I also confirmed that 2.4.7 RC1 is affected.
Bests,
Dongjoon.
On Thu, Aug 6, 2020 at 2:48 PM Sean Owen <sr...@apache.org> wrote:
> +1 from me. The same as usual. Licenses and sigs look OK, builds and
> passes tests on a standard selection of profiles.
>
> On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com>
> wrote:
> >
> > Please vote on releasing the following candidate as Apache Spark version
> 2.4.7.
> >
> > The vote is open until Aug 9th at 9AM PST and passes if a majority +1
> PMC votes are cast, with a minimum of 3 +1 votes.
> >
> > [ ] +1 Release this package as Apache Spark 2.4.7
> > [ ] -1 Do not release this package because ...
> >
> > To learn more about Apache Spark, please see http://spark.apache.org/
> >
> > There are currently no issues targeting 2.4.7 (try project = SPARK AND
> "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In Progress"))
> >
> > The tag to be voted on is v2.4.7-rc1 (commit
> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
> > https://github.com/apache/spark/tree/v2.4.7-rc1
> >
> > The release files, including signatures, digests, etc. can be found at:
> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
> >
> > Signatures used for Spark RCs can be found in this file:
> > https://dist.apache.org/repos/dist/dev/spark/KEYS
> >
> > The staging repository for this release can be found at:
> > https://repository.apache.org/content/repositories/orgapachespark-1352/
> >
> > The documentation corresponding to this release can be found at:
> > https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
> >
> > The list of bug fixes going into 2.4.7 can be found at the following URL:
> > https://s.apache.org/spark-v2.4.7-rc1
> >
> > This release is using the release script of the tag v2.4.7-rc1.
> >
> > FAQ
> >
> >
> > =========================
> > How can I help test this release?
> > =========================
> >
> > If you are a Spark user, you can help us test this release by taking
> > an existing Spark workload and running on this release candidate, then
> > reporting any regressions.
> >
> > If you're working in PySpark you can set up a virtual env and install
> > the current RC and see if anything important breaks, in the Java/Scala
> > you can add the staging repository to your projects resolvers and test
> > with the RC (make sure to clean up the artifact cache before/after so
> > you don't end up building with an out of date RC going forward).
> >
> > ===========================================
> > What should happen to JIRA tickets still targeting 2.4.7?
> > ===========================================
> >
> > The current list of open tickets targeted at 2.4.7 can be found at:
> > https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 2.4.7
> >
> > Committers should look at those and triage. Extremely important bug
> > fixes, documentation, and API tweaks that impact compatibility should
> > be worked on immediately. Everything else please retarget to an
> > appropriate release.
> >
> > ==================
> > But my bug isn't fixed?
> > ==================
> >
> > In order to make timely releases, we will typically not hold the
> > release unless the bug in question is a regression from the previous
> > release. That being said, if there is something which is a regression
> > that has not been correctly targeted please ping me or a committer to
> > help target the issue.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: [VOTE] Release Spark 2.4.7 (RC1)
Posted by Sean Owen <sr...@apache.org>.
+1 from me. The same as usual. Licenses and sigs look OK, builds and
passes tests on a standard selection of profiles.
On Thu, Aug 6, 2020 at 7:07 AM Prashant Sharma <sc...@gmail.com> wrote:
>
> Please vote on releasing the following candidate as Apache Spark version 2.4.7.
>
> The vote is open until Aug 9th at 9AM PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 2.4.7
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> There are currently no issues targeting 2.4.7 (try project = SPARK AND "Target Version/s" = "2.4.7" AND status in (Open, Reopened, "In Progress"))
>
> The tag to be voted on is v2.4.7-rc1 (commit dc04bf53fe821b7a07f817966c6c173f3b3788c6):
> https://github.com/apache/spark/tree/v2.4.7-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1352/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v2.4.7-rc1-docs/
>
> The list of bug fixes going into 2.4.7 can be found at the following URL:
> https://s.apache.org/spark-v2.4.7-rc1
>
> This release is using the release script of the tag v2.4.7-rc1.
>
> FAQ
>
>
> =========================
> How can I help test this release?
> =========================
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with an out of date RC going forward).
>
> ===========================================
> What should happen to JIRA tickets still targeting 2.4.7?
> ===========================================
>
> The current list of open tickets targeted at 2.4.7 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target Version/s" = 2.4.7
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==================
> But my bug isn't fixed?
> ==================
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org