You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Maxim Gekk <ma...@databricks.com.INVALID> on 2022/05/05 11:00:27 UTC
[VOTE] Release Spark 3.3.0 (RC1)
Please vote on releasing the following candidate as Apache Spark version 3.3
.0.
The vote is open until 11:59pm Pacific time May 10th and passes if a
majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 3.3.0
[ ] -1 Do not release this package because ...
To learn more about Apache Spark, please see http://spark.apache.org/
The tag to be voted on is v3.3.0-rc1 (commit
482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
https://github.com/apache/spark/tree/v3.3.0-rc1
The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS
The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1402
The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
The list of bug fixes going into 3.3.0 can be found at the following URL:
https://issues.apache.org/jira/projects/SPARK/versions/12350369
This release is using the release script of the tag v3.3.0-rc1.
FAQ
=========================
How can I help test this release?
=========================
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.
If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with a out of date RC going forward).
===========================================
What should happen to JIRA tickets still targeting 3.3.0?
===========================================
The current list of open tickets targeted at 3.3.0 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target
Version/s" = 3.3.0
Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.
==================
But my bug isn't fixed?
==================
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.
Maxim Gekk
Software Engineer
Databricks, Inc.
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Dongjoon Hyun <do...@gmail.com>.
Actually, I want to add this for discussion because there were many
concerns about the new Kafka 3.1 client.
https://github.com/apache/spark/pull/36517
Revert "[SPARK-36837][BUILD] Upgrade Kafka to 3.1.0"
Dongjoon.
On Wed, May 11, 2022 at 7:55 AM Holden Karau <ho...@pigscanfly.ca> wrote:
> Do we have everything we want for RC2 targeted to 3.3.0 for tracking?
>
> On Wed, May 11, 2022 at 6:44 AM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
>
>> Hi All,
>>
>> The vote has failed. I will create RC2 in a couple of days.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>>
>> On Wed, May 11, 2022 at 4:23 AM Hyukjin Kwon <gu...@gmail.com> wrote:
>>
>>> I expect to see RC2 too. I guess he just sticks to the standard, leaving
>>> the vote open till the end.
>>> It hasn't got enough +1s anyway :-).
>>>
>>> On Wed, 11 May 2022 at 10:17, Holden Karau <ho...@pigscanfly.ca> wrote:
>>>
>>>> Technically release don't follow vetos (see
>>>> https://www.apache.org/foundation/voting.html ) it's up to the RM if
>>>> they get the minimum number of binding +1s (although they are encouraged to
>>>> cancel the release if any serious issues are raised).
>>>>
>>>> That being said I'll add my -1 based on the issues reported in this
>>>> thread.
>>>>
>>>> On Tue, May 10, 2022 at 6:07 PM Sean Owen <sr...@gmail.com> wrote:
>>>>
>>>>> There's a -1 vote here, so I think this RC fails anyway.
>>>>>
>>>>> On Fri, May 6, 2022 at 10:30 AM Gengliang Wang <lt...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Maxim,
>>>>>>
>>>>>> Thanks for the work!
>>>>>> There is a bug fix from Bruce merged on branch-3.3 right after the
>>>>>> RC1 is cut:
>>>>>> SPARK-39093: Dividing interval by integral can result in codegen
>>>>>> compilation error
>>>>>> <https://github.com/apache/spark/commit/fd998c8a6783c0c8aceed8dcde4017cd479e42c8>
>>>>>>
>>>>>> So -1 from me. We should have RC2 to include the fix.
>>>>>>
>>>>>> Thanks
>>>>>> Gengliang
>>>>>>
>>>>>> On Fri, May 6, 2022 at 6:15 PM Maxim Gekk
>>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>>
>>>>>>> Hi Dongjoon,
>>>>>>>
>>>>>>> > https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>> > Since RC1 is started, could you move them out from the 3.3.0
>>>>>>> milestone?
>>>>>>>
>>>>>>> I have removed the 3.3.0 label from Fix version(s). Thank you,
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>> Maxim Gekk
>>>>>>>
>>>>>>> Software Engineer
>>>>>>>
>>>>>>> Databricks, Inc.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <
>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi, Sean.
>>>>>>>> It's interesting. I didn't see those failures from my side.
>>>>>>>>
>>>>>>>> Hi, Maxim.
>>>>>>>> In the following link, there are 17 in-progress and 6 to-do JIRA
>>>>>>>> issues which look irrelevant to this RC1 vote.
>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>>>
>>>>>>>> Since RC1 is started, could you move them out from the 3.3.0
>>>>>>>> milestone?
>>>>>>>> Otherwise, we cannot distinguish new real blocker issues from those
>>>>>>>> obsolete JIRA issues.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I looked back at the first one (SPARK-37618), it expects/assumes a
>>>>>>>>> 0022 umask to correctly test the behavior. I'm not sure how to get that to
>>>>>>>>> not fail or be ignored with a more open umask.
>>>>>>>>>
>>>>>>>>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I'm seeing test failures; is anyone seeing ones like this? This
>>>>>>>>>> is Java 8 / Scala 2.12 / Ubuntu 22.04:
>>>>>>>>>>
>>>>>>>>>> - SPARK-37618: Sub dirs are group writable when removing from
>>>>>>>>>> shuffle service enabled *** FAILED ***
>>>>>>>>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE,
>>>>>>>>>> OTHERS_READ, OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained
>>>>>>>>>> GROUP_WRITE (DiskBlockManagerSuite.scala:155)
>>>>>>>>>>
>>>>>>>>>> - Check schemas for expression examples *** FAILED ***
>>>>>>>>>> 396 did not equal 398 Expected 396 blocks in result file but
>>>>>>>>>> got 398. Try regenerating the result files.
>>>>>>>>>> (ExpressionsSchemaSuite.scala:161)
>>>>>>>>>>
>>>>>>>>>> Function 'bloom_filter_agg', Expression class
>>>>>>>>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>>>>>>>>> "" did not start with "
>>>>>>>>>> Examples:
>>>>>>>>>> " (ExpressionInfoSuite.scala:142)
>>>>>>>>>>
>>>>>>>>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>>>>>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>>>>>>
>>>>>>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>>>>>>> version 3.3.0.
>>>>>>>>>>>
>>>>>>>>>>> The vote is open until 11:59pm Pacific time May 10th and passes
>>>>>>>>>>> if a majority +1 PMC votes are cast, with a minimum of 3 +1
>>>>>>>>>>> votes.
>>>>>>>>>>>
>>>>>>>>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>>>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>>>>>
>>>>>>>>>>> To learn more about Apache Spark, please see http://spark
>>>>>>>>>>> .apache.org/
>>>>>>>>>>>
>>>>>>>>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>>>>>>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>>>>>>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>>>>>>>>
>>>>>>>>>>> The release files, including signatures, digests, etc. can be
>>>>>>>>>>> found at:
>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>>>>>>>>
>>>>>>>>>>> Signatures used for Spark RCs can be found in this file:
>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>>>>>>>
>>>>>>>>>>> The staging repository for this release can be found at:
>>>>>>>>>>>
>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>>>>>>>>
>>>>>>>>>>> The documentation corresponding to this release can be found at:
>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>>>>>>>>
>>>>>>>>>>> The list of bug fixes going into 3.3.0 can be found at the
>>>>>>>>>>> following URL:
>>>>>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>>>>>>
>>>>>>>>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> FAQ
>>>>>>>>>>>
>>>>>>>>>>> =========================
>>>>>>>>>>> How can I help test this release?
>>>>>>>>>>> =========================
>>>>>>>>>>> If you are a Spark user, you can help us test this release by
>>>>>>>>>>> taking
>>>>>>>>>>> an existing Spark workload and running on this release candidate,
>>>>>>>>>>> then
>>>>>>>>>>> reporting any regressions.
>>>>>>>>>>>
>>>>>>>>>>> If you're working in PySpark you can set up a virtual env and
>>>>>>>>>>> install
>>>>>>>>>>> the current RC and see if anything important breaks, in the
>>>>>>>>>>> Java/Scala
>>>>>>>>>>> you can add the staging repository to your projects resolvers
>>>>>>>>>>> and test
>>>>>>>>>>> with the RC (make sure to clean up the artifact cache
>>>>>>>>>>> before/after so
>>>>>>>>>>> you don't end up building with a out of date RC going forward).
>>>>>>>>>>>
>>>>>>>>>>> ===========================================
>>>>>>>>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>>>>>>>>> ===========================================
>>>>>>>>>>> The current list of open tickets targeted at 3.3.0 can be found
>>>>>>>>>>> at:
>>>>>>>>>>> https://issues.apache.org/jira/projects/SPARK and search for
>>>>>>>>>>> "Target Version/s" = 3.3.0
>>>>>>>>>>>
>>>>>>>>>>> Committers should look at those and triage. Extremely important
>>>>>>>>>>> bug
>>>>>>>>>>> fixes, documentation, and API tweaks that impact compatibility
>>>>>>>>>>> should
>>>>>>>>>>> be worked on immediately. Everything else please retarget to an
>>>>>>>>>>> appropriate release.
>>>>>>>>>>>
>>>>>>>>>>> ==================
>>>>>>>>>>> But my bug isn't fixed?
>>>>>>>>>>> ==================
>>>>>>>>>>> In order to make timely releases, we will typically not hold the
>>>>>>>>>>> release unless the bug in question is a regression from the
>>>>>>>>>>> previous
>>>>>>>>>>> release. That being said, if there is something which is a
>>>>>>>>>>> regression
>>>>>>>>>>> that has not been correctly targeted please ping me or a
>>>>>>>>>>> committer to
>>>>>>>>>>> help target the issue.
>>>>>>>>>>>
>>>>>>>>>>> Maxim Gekk
>>>>>>>>>>>
>>>>>>>>>>> Software Engineer
>>>>>>>>>>>
>>>>>>>>>>> Databricks, Inc.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Adam Binford
>>>>>>>>>
>>>>>>>>
>>>>
>>>> --
>>>> Twitter: https://twitter.com/holdenkarau
>>>> Books (Learning Spark, High Performance Spark, etc.):
>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>
>>> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Holden Karau <ho...@pigscanfly.ca>.
Do we have everything we want for RC2 targeted to 3.3.0 for tracking?
On Wed, May 11, 2022 at 6:44 AM Maxim Gekk
<ma...@databricks.com.invalid> wrote:
> Hi All,
>
> The vote has failed. I will create RC2 in a couple of days.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
>
> On Wed, May 11, 2022 at 4:23 AM Hyukjin Kwon <gu...@gmail.com> wrote:
>
>> I expect to see RC2 too. I guess he just sticks to the standard, leaving
>> the vote open till the end.
>> It hasn't got enough +1s anyway :-).
>>
>> On Wed, 11 May 2022 at 10:17, Holden Karau <ho...@pigscanfly.ca> wrote:
>>
>>> Technically release don't follow vetos (see
>>> https://www.apache.org/foundation/voting.html ) it's up to the RM if
>>> they get the minimum number of binding +1s (although they are encouraged to
>>> cancel the release if any serious issues are raised).
>>>
>>> That being said I'll add my -1 based on the issues reported in this
>>> thread.
>>>
>>> On Tue, May 10, 2022 at 6:07 PM Sean Owen <sr...@gmail.com> wrote:
>>>
>>>> There's a -1 vote here, so I think this RC fails anyway.
>>>>
>>>> On Fri, May 6, 2022 at 10:30 AM Gengliang Wang <lt...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Maxim,
>>>>>
>>>>> Thanks for the work!
>>>>> There is a bug fix from Bruce merged on branch-3.3 right after the RC1
>>>>> is cut:
>>>>> SPARK-39093: Dividing interval by integral can result in codegen
>>>>> compilation error
>>>>> <https://github.com/apache/spark/commit/fd998c8a6783c0c8aceed8dcde4017cd479e42c8>
>>>>>
>>>>> So -1 from me. We should have RC2 to include the fix.
>>>>>
>>>>> Thanks
>>>>> Gengliang
>>>>>
>>>>> On Fri, May 6, 2022 at 6:15 PM Maxim Gekk
>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>
>>>>>> Hi Dongjoon,
>>>>>>
>>>>>> > https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>> > Since RC1 is started, could you move them out from the 3.3.0
>>>>>> milestone?
>>>>>>
>>>>>> I have removed the 3.3.0 label from Fix version(s). Thank you,
>>>>>> Dongjoon.
>>>>>>
>>>>>> Maxim Gekk
>>>>>>
>>>>>> Software Engineer
>>>>>>
>>>>>> Databricks, Inc.
>>>>>>
>>>>>>
>>>>>> On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <
>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>
>>>>>>> Hi, Sean.
>>>>>>> It's interesting. I didn't see those failures from my side.
>>>>>>>
>>>>>>> Hi, Maxim.
>>>>>>> In the following link, there are 17 in-progress and 6 to-do JIRA
>>>>>>> issues which look irrelevant to this RC1 vote.
>>>>>>>
>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>>
>>>>>>> Since RC1 is started, could you move them out from the 3.3.0
>>>>>>> milestone?
>>>>>>> Otherwise, we cannot distinguish new real blocker issues from those
>>>>>>> obsolete JIRA issues.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I looked back at the first one (SPARK-37618), it expects/assumes a
>>>>>>>> 0022 umask to correctly test the behavior. I'm not sure how to get that to
>>>>>>>> not fail or be ignored with a more open umask.
>>>>>>>>
>>>>>>>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I'm seeing test failures; is anyone seeing ones like this? This is
>>>>>>>>> Java 8 / Scala 2.12 / Ubuntu 22.04:
>>>>>>>>>
>>>>>>>>> - SPARK-37618: Sub dirs are group writable when removing from
>>>>>>>>> shuffle service enabled *** FAILED ***
>>>>>>>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE,
>>>>>>>>> OTHERS_READ, OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained
>>>>>>>>> GROUP_WRITE (DiskBlockManagerSuite.scala:155)
>>>>>>>>>
>>>>>>>>> - Check schemas for expression examples *** FAILED ***
>>>>>>>>> 396 did not equal 398 Expected 396 blocks in result file but got
>>>>>>>>> 398. Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>>>>>>>>
>>>>>>>>> Function 'bloom_filter_agg', Expression class
>>>>>>>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>>>>>>>> "" did not start with "
>>>>>>>>> Examples:
>>>>>>>>> " (ExpressionInfoSuite.scala:142)
>>>>>>>>>
>>>>>>>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>>>>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>>>>>
>>>>>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>>>>>> version 3.3.0.
>>>>>>>>>>
>>>>>>>>>> The vote is open until 11:59pm Pacific time May 10th and passes
>>>>>>>>>> if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>>>>
>>>>>>>>>> To learn more about Apache Spark, please see http://spark
>>>>>>>>>> .apache.org/
>>>>>>>>>>
>>>>>>>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>>>>>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>>>>>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>>>>>>>
>>>>>>>>>> The release files, including signatures, digests, etc. can be
>>>>>>>>>> found at:
>>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>>>>>>>
>>>>>>>>>> Signatures used for Spark RCs can be found in this file:
>>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>>>>>>
>>>>>>>>>> The staging repository for this release can be found at:
>>>>>>>>>>
>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>>>>>>>
>>>>>>>>>> The documentation corresponding to this release can be found at:
>>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>>>>>>>
>>>>>>>>>> The list of bug fixes going into 3.3.0 can be found at the
>>>>>>>>>> following URL:
>>>>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>>>>>
>>>>>>>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> FAQ
>>>>>>>>>>
>>>>>>>>>> =========================
>>>>>>>>>> How can I help test this release?
>>>>>>>>>> =========================
>>>>>>>>>> If you are a Spark user, you can help us test this release by
>>>>>>>>>> taking
>>>>>>>>>> an existing Spark workload and running on this release candidate,
>>>>>>>>>> then
>>>>>>>>>> reporting any regressions.
>>>>>>>>>>
>>>>>>>>>> If you're working in PySpark you can set up a virtual env and
>>>>>>>>>> install
>>>>>>>>>> the current RC and see if anything important breaks, in the
>>>>>>>>>> Java/Scala
>>>>>>>>>> you can add the staging repository to your projects resolvers and
>>>>>>>>>> test
>>>>>>>>>> with the RC (make sure to clean up the artifact cache
>>>>>>>>>> before/after so
>>>>>>>>>> you don't end up building with a out of date RC going forward).
>>>>>>>>>>
>>>>>>>>>> ===========================================
>>>>>>>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>>>>>>>> ===========================================
>>>>>>>>>> The current list of open tickets targeted at 3.3.0 can be found
>>>>>>>>>> at:
>>>>>>>>>> https://issues.apache.org/jira/projects/SPARK and search for
>>>>>>>>>> "Target Version/s" = 3.3.0
>>>>>>>>>>
>>>>>>>>>> Committers should look at those and triage. Extremely important
>>>>>>>>>> bug
>>>>>>>>>> fixes, documentation, and API tweaks that impact compatibility
>>>>>>>>>> should
>>>>>>>>>> be worked on immediately. Everything else please retarget to an
>>>>>>>>>> appropriate release.
>>>>>>>>>>
>>>>>>>>>> ==================
>>>>>>>>>> But my bug isn't fixed?
>>>>>>>>>> ==================
>>>>>>>>>> In order to make timely releases, we will typically not hold the
>>>>>>>>>> release unless the bug in question is a regression from the
>>>>>>>>>> previous
>>>>>>>>>> release. That being said, if there is something which is a
>>>>>>>>>> regression
>>>>>>>>>> that has not been correctly targeted please ping me or a
>>>>>>>>>> committer to
>>>>>>>>>> help target the issue.
>>>>>>>>>>
>>>>>>>>>> Maxim Gekk
>>>>>>>>>>
>>>>>>>>>> Software Engineer
>>>>>>>>>>
>>>>>>>>>> Databricks, Inc.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Adam Binford
>>>>>>>>
>>>>>>>
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>> --
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Maxim Gekk <ma...@databricks.com.INVALID>.
Hi All,
The vote has failed. I will create RC2 in a couple of days.
Maxim Gekk
Software Engineer
Databricks, Inc.
On Wed, May 11, 2022 at 4:23 AM Hyukjin Kwon <gu...@gmail.com> wrote:
> I expect to see RC2 too. I guess he just sticks to the standard, leaving
> the vote open till the end.
> It hasn't got enough +1s anyway :-).
>
> On Wed, 11 May 2022 at 10:17, Holden Karau <ho...@pigscanfly.ca> wrote:
>
>> Technically release don't follow vetos (see
>> https://www.apache.org/foundation/voting.html ) it's up to the RM if
>> they get the minimum number of binding +1s (although they are encouraged to
>> cancel the release if any serious issues are raised).
>>
>> That being said I'll add my -1 based on the issues reported in this
>> thread.
>>
>> On Tue, May 10, 2022 at 6:07 PM Sean Owen <sr...@gmail.com> wrote:
>>
>>> There's a -1 vote here, so I think this RC fails anyway.
>>>
>>> On Fri, May 6, 2022 at 10:30 AM Gengliang Wang <lt...@gmail.com> wrote:
>>>
>>>> Hi Maxim,
>>>>
>>>> Thanks for the work!
>>>> There is a bug fix from Bruce merged on branch-3.3 right after the RC1
>>>> is cut:
>>>> SPARK-39093: Dividing interval by integral can result in codegen
>>>> compilation error
>>>> <https://github.com/apache/spark/commit/fd998c8a6783c0c8aceed8dcde4017cd479e42c8>
>>>>
>>>> So -1 from me. We should have RC2 to include the fix.
>>>>
>>>> Thanks
>>>> Gengliang
>>>>
>>>> On Fri, May 6, 2022 at 6:15 PM Maxim Gekk
>>>> <ma...@databricks.com.invalid> wrote:
>>>>
>>>>> Hi Dongjoon,
>>>>>
>>>>> > https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>> > Since RC1 is started, could you move them out from the 3.3.0
>>>>> milestone?
>>>>>
>>>>> I have removed the 3.3.0 label from Fix version(s). Thank you,
>>>>> Dongjoon.
>>>>>
>>>>> Maxim Gekk
>>>>>
>>>>> Software Engineer
>>>>>
>>>>> Databricks, Inc.
>>>>>
>>>>>
>>>>> On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi, Sean.
>>>>>> It's interesting. I didn't see those failures from my side.
>>>>>>
>>>>>> Hi, Maxim.
>>>>>> In the following link, there are 17 in-progress and 6 to-do JIRA
>>>>>> issues which look irrelevant to this RC1 vote.
>>>>>>
>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>
>>>>>> Since RC1 is started, could you move them out from the 3.3.0
>>>>>> milestone?
>>>>>> Otherwise, we cannot distinguish new real blocker issues from those
>>>>>> obsolete JIRA issues.
>>>>>>
>>>>>> Thanks,
>>>>>> Dongjoon.
>>>>>>
>>>>>>
>>>>>> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I looked back at the first one (SPARK-37618), it expects/assumes a
>>>>>>> 0022 umask to correctly test the behavior. I'm not sure how to get that to
>>>>>>> not fail or be ignored with a more open umask.
>>>>>>>
>>>>>>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>>>>>>
>>>>>>>> I'm seeing test failures; is anyone seeing ones like this? This is
>>>>>>>> Java 8 / Scala 2.12 / Ubuntu 22.04:
>>>>>>>>
>>>>>>>> - SPARK-37618: Sub dirs are group writable when removing from
>>>>>>>> shuffle service enabled *** FAILED ***
>>>>>>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE,
>>>>>>>> OTHERS_READ, OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained
>>>>>>>> GROUP_WRITE (DiskBlockManagerSuite.scala:155)
>>>>>>>>
>>>>>>>> - Check schemas for expression examples *** FAILED ***
>>>>>>>> 396 did not equal 398 Expected 396 blocks in result file but got
>>>>>>>> 398. Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>>>>>>>
>>>>>>>> Function 'bloom_filter_agg', Expression class
>>>>>>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>>>>>>> "" did not start with "
>>>>>>>> Examples:
>>>>>>>> " (ExpressionInfoSuite.scala:142)
>>>>>>>>
>>>>>>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>>>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>>>>
>>>>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>>>>> version 3.3.0.
>>>>>>>>>
>>>>>>>>> The vote is open until 11:59pm Pacific time May 10th and passes
>>>>>>>>> if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>>>>>>
>>>>>>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>>>
>>>>>>>>> To learn more about Apache Spark, please see http://spark
>>>>>>>>> .apache.org/
>>>>>>>>>
>>>>>>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>>>>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>>>>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>>>>>>
>>>>>>>>> The release files, including signatures, digests, etc. can be
>>>>>>>>> found at:
>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>>>>>>
>>>>>>>>> Signatures used for Spark RCs can be found in this file:
>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>>>>>
>>>>>>>>> The staging repository for this release can be found at:
>>>>>>>>>
>>>>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>>>>>>
>>>>>>>>> The documentation corresponding to this release can be found at:
>>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>>>>>>
>>>>>>>>> The list of bug fixes going into 3.3.0 can be found at the
>>>>>>>>> following URL:
>>>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>>>>
>>>>>>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> FAQ
>>>>>>>>>
>>>>>>>>> =========================
>>>>>>>>> How can I help test this release?
>>>>>>>>> =========================
>>>>>>>>> If you are a Spark user, you can help us test this release by
>>>>>>>>> taking
>>>>>>>>> an existing Spark workload and running on this release candidate,
>>>>>>>>> then
>>>>>>>>> reporting any regressions.
>>>>>>>>>
>>>>>>>>> If you're working in PySpark you can set up a virtual env and
>>>>>>>>> install
>>>>>>>>> the current RC and see if anything important breaks, in the
>>>>>>>>> Java/Scala
>>>>>>>>> you can add the staging repository to your projects resolvers and
>>>>>>>>> test
>>>>>>>>> with the RC (make sure to clean up the artifact cache before/after
>>>>>>>>> so
>>>>>>>>> you don't end up building with a out of date RC going forward).
>>>>>>>>>
>>>>>>>>> ===========================================
>>>>>>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>>>>>>> ===========================================
>>>>>>>>> The current list of open tickets targeted at 3.3.0 can be found
>>>>>>>>> at:
>>>>>>>>> https://issues.apache.org/jira/projects/SPARK and search for
>>>>>>>>> "Target Version/s" = 3.3.0
>>>>>>>>>
>>>>>>>>> Committers should look at those and triage. Extremely important bug
>>>>>>>>> fixes, documentation, and API tweaks that impact compatibility
>>>>>>>>> should
>>>>>>>>> be worked on immediately. Everything else please retarget to an
>>>>>>>>> appropriate release.
>>>>>>>>>
>>>>>>>>> ==================
>>>>>>>>> But my bug isn't fixed?
>>>>>>>>> ==================
>>>>>>>>> In order to make timely releases, we will typically not hold the
>>>>>>>>> release unless the bug in question is a regression from the
>>>>>>>>> previous
>>>>>>>>> release. That being said, if there is something which is a
>>>>>>>>> regression
>>>>>>>>> that has not been correctly targeted please ping me or a committer
>>>>>>>>> to
>>>>>>>>> help target the issue.
>>>>>>>>>
>>>>>>>>> Maxim Gekk
>>>>>>>>>
>>>>>>>>> Software Engineer
>>>>>>>>>
>>>>>>>>> Databricks, Inc.
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Adam Binford
>>>>>>>
>>>>>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.):
>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>
>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Hyukjin Kwon <gu...@gmail.com>.
I expect to see RC2 too. I guess he just sticks to the standard, leaving
the vote open till the end.
It hasn't got enough +1s anyway :-).
On Wed, 11 May 2022 at 10:17, Holden Karau <ho...@pigscanfly.ca> wrote:
> Technically release don't follow vetos (see
> https://www.apache.org/foundation/voting.html ) it's up to the RM if they
> get the minimum number of binding +1s (although they are encouraged to
> cancel the release if any serious issues are raised).
>
> That being said I'll add my -1 based on the issues reported in this thread.
>
> On Tue, May 10, 2022 at 6:07 PM Sean Owen <sr...@gmail.com> wrote:
>
>> There's a -1 vote here, so I think this RC fails anyway.
>>
>> On Fri, May 6, 2022 at 10:30 AM Gengliang Wang <lt...@gmail.com> wrote:
>>
>>> Hi Maxim,
>>>
>>> Thanks for the work!
>>> There is a bug fix from Bruce merged on branch-3.3 right after the RC1
>>> is cut:
>>> SPARK-39093: Dividing interval by integral can result in codegen
>>> compilation error
>>> <https://github.com/apache/spark/commit/fd998c8a6783c0c8aceed8dcde4017cd479e42c8>
>>>
>>> So -1 from me. We should have RC2 to include the fix.
>>>
>>> Thanks
>>> Gengliang
>>>
>>> On Fri, May 6, 2022 at 6:15 PM Maxim Gekk
>>> <ma...@databricks.com.invalid> wrote:
>>>
>>>> Hi Dongjoon,
>>>>
>>>> > https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>> > Since RC1 is started, could you move them out from the 3.3.0
>>>> milestone?
>>>>
>>>> I have removed the 3.3.0 label from Fix version(s). Thank you, Dongjoon.
>>>>
>>>> Maxim Gekk
>>>>
>>>> Software Engineer
>>>>
>>>> Databricks, Inc.
>>>>
>>>>
>>>> On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi, Sean.
>>>>> It's interesting. I didn't see those failures from my side.
>>>>>
>>>>> Hi, Maxim.
>>>>> In the following link, there are 17 in-progress and 6 to-do JIRA
>>>>> issues which look irrelevant to this RC1 vote.
>>>>>
>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>
>>>>> Since RC1 is started, could you move them out from the 3.3.0 milestone?
>>>>> Otherwise, we cannot distinguish new real blocker issues from those
>>>>> obsolete JIRA issues.
>>>>>
>>>>> Thanks,
>>>>> Dongjoon.
>>>>>
>>>>>
>>>>> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I looked back at the first one (SPARK-37618), it expects/assumes a
>>>>>> 0022 umask to correctly test the behavior. I'm not sure how to get that to
>>>>>> not fail or be ignored with a more open umask.
>>>>>>
>>>>>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>>>>>
>>>>>>> I'm seeing test failures; is anyone seeing ones like this? This is
>>>>>>> Java 8 / Scala 2.12 / Ubuntu 22.04:
>>>>>>>
>>>>>>> - SPARK-37618: Sub dirs are group writable when removing from
>>>>>>> shuffle service enabled *** FAILED ***
>>>>>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
>>>>>>> OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
>>>>>>> (DiskBlockManagerSuite.scala:155)
>>>>>>>
>>>>>>> - Check schemas for expression examples *** FAILED ***
>>>>>>> 396 did not equal 398 Expected 396 blocks in result file but got
>>>>>>> 398. Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>>>>>>
>>>>>>> Function 'bloom_filter_agg', Expression class
>>>>>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>>>>>> "" did not start with "
>>>>>>> Examples:
>>>>>>> " (ExpressionInfoSuite.scala:142)
>>>>>>>
>>>>>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>>>
>>>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>>>> version 3.3.0.
>>>>>>>>
>>>>>>>> The vote is open until 11:59pm Pacific time May 10th and passes if
>>>>>>>> a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>>>>>
>>>>>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>>
>>>>>>>> To learn more about Apache Spark, please see http://spark
>>>>>>>> .apache.org/
>>>>>>>>
>>>>>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>>>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>>>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>>>>>
>>>>>>>> The release files, including signatures, digests, etc. can be
>>>>>>>> found at:
>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>>>>>
>>>>>>>> Signatures used for Spark RCs can be found in this file:
>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>>>>
>>>>>>>> The staging repository for this release can be found at:
>>>>>>>>
>>>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>>>>>
>>>>>>>> The documentation corresponding to this release can be found at:
>>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>>>>>
>>>>>>>> The list of bug fixes going into 3.3.0 can be found at the
>>>>>>>> following URL:
>>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>>>
>>>>>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>>>>>
>>>>>>>>
>>>>>>>> FAQ
>>>>>>>>
>>>>>>>> =========================
>>>>>>>> How can I help test this release?
>>>>>>>> =========================
>>>>>>>> If you are a Spark user, you can help us test this release by
>>>>>>>> taking
>>>>>>>> an existing Spark workload and running on this release candidate,
>>>>>>>> then
>>>>>>>> reporting any regressions.
>>>>>>>>
>>>>>>>> If you're working in PySpark you can set up a virtual env and
>>>>>>>> install
>>>>>>>> the current RC and see if anything important breaks, in the
>>>>>>>> Java/Scala
>>>>>>>> you can add the staging repository to your projects resolvers and
>>>>>>>> test
>>>>>>>> with the RC (make sure to clean up the artifact cache before/after
>>>>>>>> so
>>>>>>>> you don't end up building with a out of date RC going forward).
>>>>>>>>
>>>>>>>> ===========================================
>>>>>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>>>>>> ===========================================
>>>>>>>> The current list of open tickets targeted at 3.3.0 can be found at:
>>>>>>>> https://issues.apache.org/jira/projects/SPARK and search for
>>>>>>>> "Target Version/s" = 3.3.0
>>>>>>>>
>>>>>>>> Committers should look at those and triage. Extremely important bug
>>>>>>>> fixes, documentation, and API tweaks that impact compatibility
>>>>>>>> should
>>>>>>>> be worked on immediately. Everything else please retarget to an
>>>>>>>> appropriate release.
>>>>>>>>
>>>>>>>> ==================
>>>>>>>> But my bug isn't fixed?
>>>>>>>> ==================
>>>>>>>> In order to make timely releases, we will typically not hold the
>>>>>>>> release unless the bug in question is a regression from the
>>>>>>>> previous
>>>>>>>> release. That being said, if there is something which is a
>>>>>>>> regression
>>>>>>>> that has not been correctly targeted please ping me or a committer
>>>>>>>> to
>>>>>>>> help target the issue.
>>>>>>>>
>>>>>>>> Maxim Gekk
>>>>>>>>
>>>>>>>> Software Engineer
>>>>>>>>
>>>>>>>> Databricks, Inc.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Adam Binford
>>>>>>
>>>>>
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Holden Karau <ho...@pigscanfly.ca>.
Technically release don't follow vetos (see
https://www.apache.org/foundation/voting.html ) it's up to the RM if they
get the minimum number of binding +1s (although they are encouraged to
cancel the release if any serious issues are raised).
That being said I'll add my -1 based on the issues reported in this thread.
On Tue, May 10, 2022 at 6:07 PM Sean Owen <sr...@gmail.com> wrote:
> There's a -1 vote here, so I think this RC fails anyway.
>
> On Fri, May 6, 2022 at 10:30 AM Gengliang Wang <lt...@gmail.com> wrote:
>
>> Hi Maxim,
>>
>> Thanks for the work!
>> There is a bug fix from Bruce merged on branch-3.3 right after the RC1 is
>> cut:
>> SPARK-39093: Dividing interval by integral can result in codegen
>> compilation error
>> <https://github.com/apache/spark/commit/fd998c8a6783c0c8aceed8dcde4017cd479e42c8>
>>
>> So -1 from me. We should have RC2 to include the fix.
>>
>> Thanks
>> Gengliang
>>
>> On Fri, May 6, 2022 at 6:15 PM Maxim Gekk
>> <ma...@databricks.com.invalid> wrote:
>>
>>> Hi Dongjoon,
>>>
>>> > https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>> > Since RC1 is started, could you move them out from the 3.3.0 milestone?
>>>
>>> I have removed the 3.3.0 label from Fix version(s). Thank you, Dongjoon.
>>>
>>> Maxim Gekk
>>>
>>> Software Engineer
>>>
>>> Databricks, Inc.
>>>
>>>
>>> On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
>>> wrote:
>>>
>>>> Hi, Sean.
>>>> It's interesting. I didn't see those failures from my side.
>>>>
>>>> Hi, Maxim.
>>>> In the following link, there are 17 in-progress and 6 to-do JIRA issues
>>>> which look irrelevant to this RC1 vote.
>>>>
>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>
>>>> Since RC1 is started, could you move them out from the 3.3.0 milestone?
>>>> Otherwise, we cannot distinguish new real blocker issues from those
>>>> obsolete JIRA issues.
>>>>
>>>> Thanks,
>>>> Dongjoon.
>>>>
>>>>
>>>> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com> wrote:
>>>>
>>>>> I looked back at the first one (SPARK-37618), it expects/assumes a
>>>>> 0022 umask to correctly test the behavior. I'm not sure how to get that to
>>>>> not fail or be ignored with a more open umask.
>>>>>
>>>>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>>>>
>>>>>> I'm seeing test failures; is anyone seeing ones like this? This is
>>>>>> Java 8 / Scala 2.12 / Ubuntu 22.04:
>>>>>>
>>>>>> - SPARK-37618: Sub dirs are group writable when removing from shuffle
>>>>>> service enabled *** FAILED ***
>>>>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
>>>>>> OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
>>>>>> (DiskBlockManagerSuite.scala:155)
>>>>>>
>>>>>> - Check schemas for expression examples *** FAILED ***
>>>>>> 396 did not equal 398 Expected 396 blocks in result file but got
>>>>>> 398. Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>>>>>
>>>>>> Function 'bloom_filter_agg', Expression class
>>>>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>>>>> "" did not start with "
>>>>>> Examples:
>>>>>> " (ExpressionInfoSuite.scala:142)
>>>>>>
>>>>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>>
>>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>>> version 3.3.0.
>>>>>>>
>>>>>>> The vote is open until 11:59pm Pacific time May 10th and passes if
>>>>>>> a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>>>>
>>>>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>
>>>>>>> To learn more about Apache Spark, please see http://spark
>>>>>>> .apache.org/
>>>>>>>
>>>>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>>>>
>>>>>>> The release files, including signatures, digests, etc. can be found
>>>>>>> at:
>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>>>>
>>>>>>> Signatures used for Spark RCs can be found in this file:
>>>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>>>
>>>>>>> The staging repository for this release can be found at:
>>>>>>>
>>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>>>>
>>>>>>> The documentation corresponding to this release can be found at:
>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>>>>
>>>>>>> The list of bug fixes going into 3.3.0 can be found at the
>>>>>>> following URL:
>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>>
>>>>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>>>>
>>>>>>>
>>>>>>> FAQ
>>>>>>>
>>>>>>> =========================
>>>>>>> How can I help test this release?
>>>>>>> =========================
>>>>>>> If you are a Spark user, you can help us test this release by taking
>>>>>>> an existing Spark workload and running on this release candidate,
>>>>>>> then
>>>>>>> reporting any regressions.
>>>>>>>
>>>>>>> If you're working in PySpark you can set up a virtual env and install
>>>>>>> the current RC and see if anything important breaks, in the
>>>>>>> Java/Scala
>>>>>>> you can add the staging repository to your projects resolvers and
>>>>>>> test
>>>>>>> with the RC (make sure to clean up the artifact cache before/after so
>>>>>>> you don't end up building with a out of date RC going forward).
>>>>>>>
>>>>>>> ===========================================
>>>>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>>>>> ===========================================
>>>>>>> The current list of open tickets targeted at 3.3.0 can be found at:
>>>>>>> https://issues.apache.org/jira/projects/SPARK and search for
>>>>>>> "Target Version/s" = 3.3.0
>>>>>>>
>>>>>>> Committers should look at those and triage. Extremely important bug
>>>>>>> fixes, documentation, and API tweaks that impact compatibility should
>>>>>>> be worked on immediately. Everything else please retarget to an
>>>>>>> appropriate release.
>>>>>>>
>>>>>>> ==================
>>>>>>> But my bug isn't fixed?
>>>>>>> ==================
>>>>>>> In order to make timely releases, we will typically not hold the
>>>>>>> release unless the bug in question is a regression from the previous
>>>>>>> release. That being said, if there is something which is a
>>>>>>> regression
>>>>>>> that has not been correctly targeted please ping me or a committer to
>>>>>>> help target the issue.
>>>>>>>
>>>>>>> Maxim Gekk
>>>>>>>
>>>>>>> Software Engineer
>>>>>>>
>>>>>>> Databricks, Inc.
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Adam Binford
>>>>>
>>>>
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Sean Owen <sr...@gmail.com>.
There's a -1 vote here, so I think this RC fails anyway.
On Fri, May 6, 2022 at 10:30 AM Gengliang Wang <lt...@gmail.com> wrote:
> Hi Maxim,
>
> Thanks for the work!
> There is a bug fix from Bruce merged on branch-3.3 right after the RC1 is
> cut:
> SPARK-39093: Dividing interval by integral can result in codegen
> compilation error
> <https://github.com/apache/spark/commit/fd998c8a6783c0c8aceed8dcde4017cd479e42c8>
>
> So -1 from me. We should have RC2 to include the fix.
>
> Thanks
> Gengliang
>
> On Fri, May 6, 2022 at 6:15 PM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
>
>> Hi Dongjoon,
>>
>> > https://issues.apache.org/jira/projects/SPARK/versions/12350369
>> > Since RC1 is started, could you move them out from the 3.3.0 milestone?
>>
>> I have removed the 3.3.0 label from Fix version(s). Thank you, Dongjoon.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>>
>> On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> Hi, Sean.
>>> It's interesting. I didn't see those failures from my side.
>>>
>>> Hi, Maxim.
>>> In the following link, there are 17 in-progress and 6 to-do JIRA issues
>>> which look irrelevant to this RC1 vote.
>>>
>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>
>>> Since RC1 is started, could you move them out from the 3.3.0 milestone?
>>> Otherwise, we cannot distinguish new real blocker issues from those
>>> obsolete JIRA issues.
>>>
>>> Thanks,
>>> Dongjoon.
>>>
>>>
>>> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com> wrote:
>>>
>>>> I looked back at the first one (SPARK-37618), it expects/assumes a 0022
>>>> umask to correctly test the behavior. I'm not sure how to get that to not
>>>> fail or be ignored with a more open umask.
>>>>
>>>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>>>
>>>>> I'm seeing test failures; is anyone seeing ones like this? This is
>>>>> Java 8 / Scala 2.12 / Ubuntu 22.04:
>>>>>
>>>>> - SPARK-37618: Sub dirs are group writable when removing from shuffle
>>>>> service enabled *** FAILED ***
>>>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
>>>>> OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
>>>>> (DiskBlockManagerSuite.scala:155)
>>>>>
>>>>> - Check schemas for expression examples *** FAILED ***
>>>>> 396 did not equal 398 Expected 396 blocks in result file but got
>>>>> 398. Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>>>>
>>>>> Function 'bloom_filter_agg', Expression class
>>>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>>>> "" did not start with "
>>>>> Examples:
>>>>> " (ExpressionInfoSuite.scala:142)
>>>>>
>>>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>
>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>> version 3.3.0.
>>>>>>
>>>>>> The vote is open until 11:59pm Pacific time May 10th and passes if a
>>>>>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>>>
>>>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>>>> [ ] -1 Do not release this package because ...
>>>>>>
>>>>>> To learn more about Apache Spark, please see http://spark.apache.org/
>>>>>>
>>>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>>>
>>>>>> The release files, including signatures, digests, etc. can be found
>>>>>> at:
>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>>>
>>>>>> Signatures used for Spark RCs can be found in this file:
>>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>>
>>>>>> The staging repository for this release can be found at:
>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>>>
>>>>>> The documentation corresponding to this release can be found at:
>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>>>
>>>>>> The list of bug fixes going into 3.3.0 can be found at the following
>>>>>> URL:
>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>
>>>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>>>
>>>>>>
>>>>>> FAQ
>>>>>>
>>>>>> =========================
>>>>>> How can I help test this release?
>>>>>> =========================
>>>>>> If you are a Spark user, you can help us test this release by taking
>>>>>> an existing Spark workload and running on this release candidate,
>>>>>> then
>>>>>> reporting any regressions.
>>>>>>
>>>>>> If you're working in PySpark you can set up a virtual env and install
>>>>>> the current RC and see if anything important breaks, in the Java/Scala
>>>>>> you can add the staging repository to your projects resolvers and test
>>>>>> with the RC (make sure to clean up the artifact cache before/after so
>>>>>> you don't end up building with a out of date RC going forward).
>>>>>>
>>>>>> ===========================================
>>>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>>>> ===========================================
>>>>>> The current list of open tickets targeted at 3.3.0 can be found at:
>>>>>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>>>>> Version/s" = 3.3.0
>>>>>>
>>>>>> Committers should look at those and triage. Extremely important bug
>>>>>> fixes, documentation, and API tweaks that impact compatibility should
>>>>>> be worked on immediately. Everything else please retarget to an
>>>>>> appropriate release.
>>>>>>
>>>>>> ==================
>>>>>> But my bug isn't fixed?
>>>>>> ==================
>>>>>> In order to make timely releases, we will typically not hold the
>>>>>> release unless the bug in question is a regression from the previous
>>>>>> release. That being said, if there is something which is a regression
>>>>>> that has not been correctly targeted please ping me or a committer to
>>>>>> help target the issue.
>>>>>>
>>>>>> Maxim Gekk
>>>>>>
>>>>>> Software Engineer
>>>>>>
>>>>>> Databricks, Inc.
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Adam Binford
>>>>
>>>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Mridul Muralidharan <mr...@gmail.com>.
I will also try to get a PR out to fix the first test failure that Sean
reported. I will have a PR ready by EOD.
Regards,
Mridul
On Fri, May 6, 2022 at 10:31 AM Gengliang Wang <lt...@gmail.com> wrote:
> Hi Maxim,
>
> Thanks for the work!
> There is a bug fix from Bruce merged on branch-3.3 right after the RC1 is
> cut:
> SPARK-39093: Dividing interval by integral can result in codegen
> compilation error
> <https://github.com/apache/spark/commit/fd998c8a6783c0c8aceed8dcde4017cd479e42c8>
>
> So -1 from me. We should have RC2 to include the fix.
>
> Thanks
> Gengliang
>
> On Fri, May 6, 2022 at 6:15 PM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
>
>> Hi Dongjoon,
>>
>> > https://issues.apache.org/jira/projects/SPARK/versions/12350369
>> > Since RC1 is started, could you move them out from the 3.3.0 milestone?
>>
>> I have removed the 3.3.0 label from Fix version(s). Thank you, Dongjoon.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>>
>> On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> Hi, Sean.
>>> It's interesting. I didn't see those failures from my side.
>>>
>>> Hi, Maxim.
>>> In the following link, there are 17 in-progress and 6 to-do JIRA issues
>>> which look irrelevant to this RC1 vote.
>>>
>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>
>>> Since RC1 is started, could you move them out from the 3.3.0 milestone?
>>> Otherwise, we cannot distinguish new real blocker issues from those
>>> obsolete JIRA issues.
>>>
>>> Thanks,
>>> Dongjoon.
>>>
>>>
>>> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com> wrote:
>>>
>>>> I looked back at the first one (SPARK-37618), it expects/assumes a 0022
>>>> umask to correctly test the behavior. I'm not sure how to get that to not
>>>> fail or be ignored with a more open umask.
>>>>
>>>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>>>
>>>>> I'm seeing test failures; is anyone seeing ones like this? This is
>>>>> Java 8 / Scala 2.12 / Ubuntu 22.04:
>>>>>
>>>>> - SPARK-37618: Sub dirs are group writable when removing from shuffle
>>>>> service enabled *** FAILED ***
>>>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
>>>>> OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
>>>>> (DiskBlockManagerSuite.scala:155)
>>>>>
>>>>> - Check schemas for expression examples *** FAILED ***
>>>>> 396 did not equal 398 Expected 396 blocks in result file but got
>>>>> 398. Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>>>>
>>>>> Function 'bloom_filter_agg', Expression class
>>>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>>>> "" did not start with "
>>>>> Examples:
>>>>> " (ExpressionInfoSuite.scala:142)
>>>>>
>>>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>
>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>> version 3.3.0.
>>>>>>
>>>>>> The vote is open until 11:59pm Pacific time May 10th and passes if a
>>>>>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>>>
>>>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>>>> [ ] -1 Do not release this package because ...
>>>>>>
>>>>>> To learn more about Apache Spark, please see http://spark.apache.org/
>>>>>>
>>>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>>>
>>>>>> The release files, including signatures, digests, etc. can be found
>>>>>> at:
>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>>>
>>>>>> Signatures used for Spark RCs can be found in this file:
>>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>>
>>>>>> The staging repository for this release can be found at:
>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>>>
>>>>>> The documentation corresponding to this release can be found at:
>>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>>>
>>>>>> The list of bug fixes going into 3.3.0 can be found at the following
>>>>>> URL:
>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>>
>>>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>>>
>>>>>>
>>>>>> FAQ
>>>>>>
>>>>>> =========================
>>>>>> How can I help test this release?
>>>>>> =========================
>>>>>> If you are a Spark user, you can help us test this release by taking
>>>>>> an existing Spark workload and running on this release candidate,
>>>>>> then
>>>>>> reporting any regressions.
>>>>>>
>>>>>> If you're working in PySpark you can set up a virtual env and install
>>>>>> the current RC and see if anything important breaks, in the Java/Scala
>>>>>> you can add the staging repository to your projects resolvers and test
>>>>>> with the RC (make sure to clean up the artifact cache before/after so
>>>>>> you don't end up building with a out of date RC going forward).
>>>>>>
>>>>>> ===========================================
>>>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>>>> ===========================================
>>>>>> The current list of open tickets targeted at 3.3.0 can be found at:
>>>>>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>>>>> Version/s" = 3.3.0
>>>>>>
>>>>>> Committers should look at those and triage. Extremely important bug
>>>>>> fixes, documentation, and API tweaks that impact compatibility should
>>>>>> be worked on immediately. Everything else please retarget to an
>>>>>> appropriate release.
>>>>>>
>>>>>> ==================
>>>>>> But my bug isn't fixed?
>>>>>> ==================
>>>>>> In order to make timely releases, we will typically not hold the
>>>>>> release unless the bug in question is a regression from the previous
>>>>>> release. That being said, if there is something which is a regression
>>>>>> that has not been correctly targeted please ping me or a committer to
>>>>>> help target the issue.
>>>>>>
>>>>>> Maxim Gekk
>>>>>>
>>>>>> Software Engineer
>>>>>>
>>>>>> Databricks, Inc.
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Adam Binford
>>>>
>>>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Gengliang Wang <lt...@gmail.com>.
Hi Maxim,
Thanks for the work!
There is a bug fix from Bruce merged on branch-3.3 right after the RC1 is
cut:
SPARK-39093: Dividing interval by integral can result in codegen
compilation error
<https://github.com/apache/spark/commit/fd998c8a6783c0c8aceed8dcde4017cd479e42c8>
So -1 from me. We should have RC2 to include the fix.
Thanks
Gengliang
On Fri, May 6, 2022 at 6:15 PM Maxim Gekk <ma...@databricks.com.invalid>
wrote:
> Hi Dongjoon,
>
> > https://issues.apache.org/jira/projects/SPARK/versions/12350369
> > Since RC1 is started, could you move them out from the 3.3.0 milestone?
>
> I have removed the 3.3.0 label from Fix version(s). Thank you, Dongjoon.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
>
> On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> Hi, Sean.
>> It's interesting. I didn't see those failures from my side.
>>
>> Hi, Maxim.
>> In the following link, there are 17 in-progress and 6 to-do JIRA issues
>> which look irrelevant to this RC1 vote.
>>
>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>
>> Since RC1 is started, could you move them out from the 3.3.0 milestone?
>> Otherwise, we cannot distinguish new real blocker issues from those
>> obsolete JIRA issues.
>>
>> Thanks,
>> Dongjoon.
>>
>>
>> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com> wrote:
>>
>>> I looked back at the first one (SPARK-37618), it expects/assumes a 0022
>>> umask to correctly test the behavior. I'm not sure how to get that to not
>>> fail or be ignored with a more open umask.
>>>
>>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>>
>>>> I'm seeing test failures; is anyone seeing ones like this? This is Java
>>>> 8 / Scala 2.12 / Ubuntu 22.04:
>>>>
>>>> - SPARK-37618: Sub dirs are group writable when removing from shuffle
>>>> service enabled *** FAILED ***
>>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
>>>> OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
>>>> (DiskBlockManagerSuite.scala:155)
>>>>
>>>> - Check schemas for expression examples *** FAILED ***
>>>> 396 did not equal 398 Expected 396 blocks in result file but got 398.
>>>> Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>>>
>>>> Function 'bloom_filter_agg', Expression class
>>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>>> "" did not start with "
>>>> Examples:
>>>> " (ExpressionInfoSuite.scala:142)
>>>>
>>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>>> <ma...@databricks.com.invalid> wrote:
>>>>
>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>> version 3.3.0.
>>>>>
>>>>> The vote is open until 11:59pm Pacific time May 10th and passes if a
>>>>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>>
>>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>>> [ ] -1 Do not release this package because ...
>>>>>
>>>>> To learn more about Apache Spark, please see http://spark.apache.org/
>>>>>
>>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>>
>>>>> The release files, including signatures, digests, etc. can be found
>>>>> at:
>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>>
>>>>> Signatures used for Spark RCs can be found in this file:
>>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>
>>>>> The staging repository for this release can be found at:
>>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>>
>>>>> The documentation corresponding to this release can be found at:
>>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>>
>>>>> The list of bug fixes going into 3.3.0 can be found at the following
>>>>> URL:
>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>>
>>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>>
>>>>>
>>>>> FAQ
>>>>>
>>>>> =========================
>>>>> How can I help test this release?
>>>>> =========================
>>>>> If you are a Spark user, you can help us test this release by taking
>>>>> an existing Spark workload and running on this release candidate, then
>>>>> reporting any regressions.
>>>>>
>>>>> If you're working in PySpark you can set up a virtual env and install
>>>>> the current RC and see if anything important breaks, in the Java/Scala
>>>>> you can add the staging repository to your projects resolvers and test
>>>>> with the RC (make sure to clean up the artifact cache before/after so
>>>>> you don't end up building with a out of date RC going forward).
>>>>>
>>>>> ===========================================
>>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>>> ===========================================
>>>>> The current list of open tickets targeted at 3.3.0 can be found at:
>>>>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>>>> Version/s" = 3.3.0
>>>>>
>>>>> Committers should look at those and triage. Extremely important bug
>>>>> fixes, documentation, and API tweaks that impact compatibility should
>>>>> be worked on immediately. Everything else please retarget to an
>>>>> appropriate release.
>>>>>
>>>>> ==================
>>>>> But my bug isn't fixed?
>>>>> ==================
>>>>> In order to make timely releases, we will typically not hold the
>>>>> release unless the bug in question is a regression from the previous
>>>>> release. That being said, if there is something which is a regression
>>>>> that has not been correctly targeted please ping me or a committer to
>>>>> help target the issue.
>>>>>
>>>>> Maxim Gekk
>>>>>
>>>>> Software Engineer
>>>>>
>>>>> Databricks, Inc.
>>>>>
>>>>
>>>
>>> --
>>> Adam Binford
>>>
>>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Maxim Gekk <ma...@databricks.com.INVALID>.
Hi Dongjoon,
> https://issues.apache.org/jira/projects/SPARK/versions/12350369
> Since RC1 is started, could you move them out from the 3.3.0 milestone?
I have removed the 3.3.0 label from Fix version(s). Thank you, Dongjoon.
Maxim Gekk
Software Engineer
Databricks, Inc.
On Fri, May 6, 2022 at 11:06 AM Dongjoon Hyun <do...@gmail.com>
wrote:
> Hi, Sean.
> It's interesting. I didn't see those failures from my side.
>
> Hi, Maxim.
> In the following link, there are 17 in-progress and 6 to-do JIRA issues
> which look irrelevant to this RC1 vote.
>
> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>
> Since RC1 is started, could you move them out from the 3.3.0 milestone?
> Otherwise, we cannot distinguish new real blocker issues from those
> obsolete JIRA issues.
>
> Thanks,
> Dongjoon.
>
>
> On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com> wrote:
>
>> I looked back at the first one (SPARK-37618), it expects/assumes a 0022
>> umask to correctly test the behavior. I'm not sure how to get that to not
>> fail or be ignored with a more open umask.
>>
>> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>>
>>> I'm seeing test failures; is anyone seeing ones like this? This is Java
>>> 8 / Scala 2.12 / Ubuntu 22.04:
>>>
>>> - SPARK-37618: Sub dirs are group writable when removing from shuffle
>>> service enabled *** FAILED ***
>>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
>>> OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
>>> (DiskBlockManagerSuite.scala:155)
>>>
>>> - Check schemas for expression examples *** FAILED ***
>>> 396 did not equal 398 Expected 396 blocks in result file but got 398.
>>> Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>>
>>> Function 'bloom_filter_agg', Expression class
>>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>>> "" did not start with "
>>> Examples:
>>> " (ExpressionInfoSuite.scala:142)
>>>
>>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>>> <ma...@databricks.com.invalid> wrote:
>>>
>>>> Please vote on releasing the following candidate as Apache Spark
>>>> version 3.3.0.
>>>>
>>>> The vote is open until 11:59pm Pacific time May 10th and passes if a
>>>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>>
>>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>>> [ ] -1 Do not release this package because ...
>>>>
>>>> To learn more about Apache Spark, please see http://spark.apache.org/
>>>>
>>>> The tag to be voted on is v3.3.0-rc1 (commit
>>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>>
>>>> The release files, including signatures, digests, etc. can be found at:
>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>>
>>>> Signatures used for Spark RCs can be found in this file:
>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>
>>>> The staging repository for this release can be found at:
>>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>>
>>>> The documentation corresponding to this release can be found at:
>>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>>
>>>> The list of bug fixes going into 3.3.0 can be found at the following
>>>> URL:
>>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>>
>>>> This release is using the release script of the tag v3.3.0-rc1.
>>>>
>>>>
>>>> FAQ
>>>>
>>>> =========================
>>>> How can I help test this release?
>>>> =========================
>>>> If you are a Spark user, you can help us test this release by taking
>>>> an existing Spark workload and running on this release candidate, then
>>>> reporting any regressions.
>>>>
>>>> If you're working in PySpark you can set up a virtual env and install
>>>> the current RC and see if anything important breaks, in the Java/Scala
>>>> you can add the staging repository to your projects resolvers and test
>>>> with the RC (make sure to clean up the artifact cache before/after so
>>>> you don't end up building with a out of date RC going forward).
>>>>
>>>> ===========================================
>>>> What should happen to JIRA tickets still targeting 3.3.0?
>>>> ===========================================
>>>> The current list of open tickets targeted at 3.3.0 can be found at:
>>>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>>> Version/s" = 3.3.0
>>>>
>>>> Committers should look at those and triage. Extremely important bug
>>>> fixes, documentation, and API tweaks that impact compatibility should
>>>> be worked on immediately. Everything else please retarget to an
>>>> appropriate release.
>>>>
>>>> ==================
>>>> But my bug isn't fixed?
>>>> ==================
>>>> In order to make timely releases, we will typically not hold the
>>>> release unless the bug in question is a regression from the previous
>>>> release. That being said, if there is something which is a regression
>>>> that has not been correctly targeted please ping me or a committer to
>>>> help target the issue.
>>>>
>>>> Maxim Gekk
>>>>
>>>> Software Engineer
>>>>
>>>> Databricks, Inc.
>>>>
>>>
>>
>> --
>> Adam Binford
>>
>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Dongjoon Hyun <do...@gmail.com>.
Hi, Sean.
It's interesting. I didn't see those failures from my side.
Hi, Maxim.
In the following link, there are 17 in-progress and 6 to-do JIRA issues
which look irrelevant to this RC1 vote.
https://issues.apache.org/jira/projects/SPARK/versions/12350369
Since RC1 is started, could you move them out from the 3.3.0 milestone?
Otherwise, we cannot distinguish new real blocker issues from those
obsolete JIRA issues.
Thanks,
Dongjoon.
On Thu, May 5, 2022 at 11:46 AM Adam Binford <ad...@gmail.com> wrote:
> I looked back at the first one (SPARK-37618), it expects/assumes a 0022
> umask to correctly test the behavior. I'm not sure how to get that to not
> fail or be ignored with a more open umask.
>
> On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
>
>> I'm seeing test failures; is anyone seeing ones like this? This is Java 8
>> / Scala 2.12 / Ubuntu 22.04:
>>
>> - SPARK-37618: Sub dirs are group writable when removing from shuffle
>> service enabled *** FAILED ***
>> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
>> OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
>> (DiskBlockManagerSuite.scala:155)
>>
>> - Check schemas for expression examples *** FAILED ***
>> 396 did not equal 398 Expected 396 blocks in result file but got 398.
>> Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>>
>> Function 'bloom_filter_agg', Expression class
>> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
>> "" did not start with "
>> Examples:
>> " (ExpressionInfoSuite.scala:142)
>>
>> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
>> <ma...@databricks.com.invalid> wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark
>>> version 3.3.0.
>>>
>>> The vote is open until 11:59pm Pacific time May 10th and passes if a
>>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>>
>>> [ ] +1 Release this package as Apache Spark 3.3.0
>>> [ ] -1 Do not release this package because ...
>>>
>>> To learn more about Apache Spark, please see http://spark.apache.org/
>>>
>>> The tag to be voted on is v3.3.0-rc1 (commit
>>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>>
>>> Signatures used for Spark RCs can be found in this file:
>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>>
>>> The documentation corresponding to this release can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>>
>>> The list of bug fixes going into 3.3.0 can be found at the following
>>> URL:
>>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>>
>>> This release is using the release script of the tag v3.3.0-rc1.
>>>
>>>
>>> FAQ
>>>
>>> =========================
>>> How can I help test this release?
>>> =========================
>>> If you are a Spark user, you can help us test this release by taking
>>> an existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> If you're working in PySpark you can set up a virtual env and install
>>> the current RC and see if anything important breaks, in the Java/Scala
>>> you can add the staging repository to your projects resolvers and test
>>> with the RC (make sure to clean up the artifact cache before/after so
>>> you don't end up building with a out of date RC going forward).
>>>
>>> ===========================================
>>> What should happen to JIRA tickets still targeting 3.3.0?
>>> ===========================================
>>> The current list of open tickets targeted at 3.3.0 can be found at:
>>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>> Version/s" = 3.3.0
>>>
>>> Committers should look at those and triage. Extremely important bug
>>> fixes, documentation, and API tweaks that impact compatibility should
>>> be worked on immediately. Everything else please retarget to an
>>> appropriate release.
>>>
>>> ==================
>>> But my bug isn't fixed?
>>> ==================
>>> In order to make timely releases, we will typically not hold the
>>> release unless the bug in question is a regression from the previous
>>> release. That being said, if there is something which is a regression
>>> that has not been correctly targeted please ping me or a committer to
>>> help target the issue.
>>>
>>> Maxim Gekk
>>>
>>> Software Engineer
>>>
>>> Databricks, Inc.
>>>
>>
>
> --
> Adam Binford
>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Adam Binford <ad...@gmail.com>.
I looked back at the first one (SPARK-37618), it expects/assumes a 0022
umask to correctly test the behavior. I'm not sure how to get that to not
fail or be ignored with a more open umask.
On Thu, May 5, 2022 at 1:56 PM Sean Owen <sr...@gmail.com> wrote:
> I'm seeing test failures; is anyone seeing ones like this? This is Java 8
> / Scala 2.12 / Ubuntu 22.04:
>
> - SPARK-37618: Sub dirs are group writable when removing from shuffle
> service enabled *** FAILED ***
> [OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
> OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
> (DiskBlockManagerSuite.scala:155)
>
> - Check schemas for expression examples *** FAILED ***
> 396 did not equal 398 Expected 396 blocks in result file but got 398.
> Try regenerating the result files. (ExpressionsSchemaSuite.scala:161)
>
> Function 'bloom_filter_agg', Expression class
> 'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
> "" did not start with "
> Examples:
> " (ExpressionInfoSuite.scala:142)
>
> On Thu, May 5, 2022 at 6:01 AM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
>
>> Please vote on releasing the following candidate as Apache Spark version
>> 3.3.0.
>>
>> The vote is open until 11:59pm Pacific time May 10th and passes if a
>> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>
>> [ ] +1 Release this package as Apache Spark 3.3.0
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see http://spark.apache.org/
>>
>> The tag to be voted on is v3.3.0-rc1 (commit
>> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>
>> The release files, including signatures, digests, etc. can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>
>> Signatures used for Spark RCs can be found in this file:
>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>
>> The documentation corresponding to this release can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>
>> The list of bug fixes going into 3.3.0 can be found at the following URL:
>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>
>> This release is using the release script of the tag v3.3.0-rc1.
>>
>>
>> FAQ
>>
>> =========================
>> How can I help test this release?
>> =========================
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> If you're working in PySpark you can set up a virtual env and install
>> the current RC and see if anything important breaks, in the Java/Scala
>> you can add the staging repository to your projects resolvers and test
>> with the RC (make sure to clean up the artifact cache before/after so
>> you don't end up building with a out of date RC going forward).
>>
>> ===========================================
>> What should happen to JIRA tickets still targeting 3.3.0?
>> ===========================================
>> The current list of open tickets targeted at 3.3.0 can be found at:
>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>> Version/s" = 3.3.0
>>
>> Committers should look at those and triage. Extremely important bug
>> fixes, documentation, and API tweaks that impact compatibility should
>> be worked on immediately. Everything else please retarget to an
>> appropriate release.
>>
>> ==================
>> But my bug isn't fixed?
>> ==================
>> In order to make timely releases, we will typically not hold the
>> release unless the bug in question is a regression from the previous
>> release. That being said, if there is something which is a regression
>> that has not been correctly targeted please ping me or a committer to
>> help target the issue.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>
--
Adam Binford
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Sean Owen <sr...@gmail.com>.
I'm seeing test failures; is anyone seeing ones like this? This is Java 8 /
Scala 2.12 / Ubuntu 22.04:
- SPARK-37618: Sub dirs are group writable when removing from shuffle
service enabled *** FAILED ***
[OWNER_WRITE, GROUP_READ, GROUP_WRITE, GROUP_EXECUTE, OTHERS_READ,
OWNER_READ, OTHERS_EXECUTE, OWNER_EXECUTE] contained GROUP_WRITE
(DiskBlockManagerSuite.scala:155)
- Check schemas for expression examples *** FAILED ***
396 did not equal 398 Expected 396 blocks in result file but got 398. Try
regenerating the result files. (ExpressionsSchemaSuite.scala:161)
Function 'bloom_filter_agg', Expression class
'org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate'
"" did not start with "
Examples:
" (ExpressionInfoSuite.scala:142)
On Thu, May 5, 2022 at 6:01 AM Maxim Gekk <ma...@databricks.com.invalid>
wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 3.3.0.
>
> The vote is open until 11:59pm Pacific time May 10th and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.3.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> The tag to be voted on is v3.3.0-rc1 (commit
> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
> https://github.com/apache/spark/tree/v3.3.0-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1402
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>
> The list of bug fixes going into 3.3.0 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>
> This release is using the release script of the tag v3.3.0-rc1.
>
>
> FAQ
>
> =========================
> How can I help test this release?
> =========================
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with a out of date RC going forward).
>
> ===========================================
> What should happen to JIRA tickets still targeting 3.3.0?
> ===========================================
> The current list of open tickets targeted at 3.3.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.3.0
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==================
> But my bug isn't fixed?
> ==================
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Wenchen Fan <cl...@gmail.com>.
I'd like to see an RC2 as well. There is kind of a correctness bug fixed
after RC1 is cut: https://github.com/apache/spark/pull/36468
Users may hit this bug much more frequently if they enable ANSI mode. It's
not a regression so I'd vote -0.
On Wed, May 11, 2022 at 5:24 AM Thomas graves <tg...@apache.org> wrote:
> Is there going to be an rc2? I thought there were a couple of issue
> mentioned in the thread.
>
> On Tue, May 10, 2022 at 11:53 AM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
> >
> > Hi All,
> >
> > Today is the last day for voting. Please, test the RC1 and vote.
> >
> > Maxim Gekk
> >
> > Software Engineer
> >
> > Databricks, Inc.
> >
> >
> >
> > On Sat, May 7, 2022 at 10:58 AM beliefer <be...@163.com> wrote:
> >>
> >>
> >> @Maxim Gekk Glad to hear that!
> >>
> >> But there is a bug https://github.com/apache/spark/pull/36457
> >> I think we should merge it into 3.3.0
> >>
> >>
> >> At 2022-05-05 19:00:27, "Maxim Gekk" <ma...@databricks.com.INVALID>
> wrote:
> >>
> >> Please vote on releasing the following candidate as Apache Spark
> version 3.3.0.
> >>
> >> The vote is open until 11:59pm Pacific time May 10th and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
> >>
> >> [ ] +1 Release this package as Apache Spark 3.3.0
> >> [ ] -1 Do not release this package because ...
> >>
> >> To learn more about Apache Spark, please see http://spark.apache.org/
> >>
> >> The tag to be voted on is v3.3.0-rc1 (commit
> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
> >> https://github.com/apache/spark/tree/v3.3.0-rc1
> >>
> >> The release files, including signatures, digests, etc. can be found at:
> >> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
> >>
> >> Signatures used for Spark RCs can be found in this file:
> >> https://dist.apache.org/repos/dist/dev/spark/KEYS
> >>
> >> The staging repository for this release can be found at:
> >> https://repository.apache.org/content/repositories/orgapachespark-1402
> >>
> >> The documentation corresponding to this release can be found at:
> >> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
> >>
> >> The list of bug fixes going into 3.3.0 can be found at the following
> URL:
> >> https://issues.apache.org/jira/projects/SPARK/versions/12350369
> >>
> >> This release is using the release script of the tag v3.3.0-rc1.
> >>
> >>
> >> FAQ
> >>
> >> =========================
> >> How can I help test this release?
> >> =========================
> >> If you are a Spark user, you can help us test this release by taking
> >> an existing Spark workload and running on this release candidate, then
> >> reporting any regressions.
> >>
> >> If you're working in PySpark you can set up a virtual env and install
> >> the current RC and see if anything important breaks, in the Java/Scala
> >> you can add the staging repository to your projects resolvers and test
> >> with the RC (make sure to clean up the artifact cache before/after so
> >> you don't end up building with a out of date RC going forward).
> >>
> >> ===========================================
> >> What should happen to JIRA tickets still targeting 3.3.0?
> >> ===========================================
> >> The current list of open tickets targeted at 3.3.0 can be found at:
> >> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.3.0
> >>
> >> Committers should look at those and triage. Extremely important bug
> >> fixes, documentation, and API tweaks that impact compatibility should
> >> be worked on immediately. Everything else please retarget to an
> >> appropriate release.
> >>
> >> ==================
> >> But my bug isn't fixed?
> >> ==================
> >> In order to make timely releases, we will typically not hold the
> >> release unless the bug in question is a regression from the previous
> >> release. That being said, if there is something which is a regression
> >> that has not been correctly targeted please ping me or a committer to
> >> help target the issue.
> >>
> >> Maxim Gekk
> >>
> >> Software Engineer
> >>
> >> Databricks, Inc.
> >>
> >>
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Thomas graves <tg...@apache.org>.
Is there going to be an rc2? I thought there were a couple of issue
mentioned in the thread.
On Tue, May 10, 2022 at 11:53 AM Maxim Gekk
<ma...@databricks.com.invalid> wrote:
>
> Hi All,
>
> Today is the last day for voting. Please, test the RC1 and vote.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
>
>
> On Sat, May 7, 2022 at 10:58 AM beliefer <be...@163.com> wrote:
>>
>>
>> @Maxim Gekk Glad to hear that!
>>
>> But there is a bug https://github.com/apache/spark/pull/36457
>> I think we should merge it into 3.3.0
>>
>>
>> At 2022-05-05 19:00:27, "Maxim Gekk" <ma...@databricks.com.INVALID> wrote:
>>
>> Please vote on releasing the following candidate as Apache Spark version 3.3.0.
>>
>> The vote is open until 11:59pm Pacific time May 10th and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>>
>> [ ] +1 Release this package as Apache Spark 3.3.0
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see http://spark.apache.org/
>>
>> The tag to be voted on is v3.3.0-rc1 (commit 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
>> https://github.com/apache/spark/tree/v3.3.0-rc1
>>
>> The release files, including signatures, digests, etc. can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>>
>> Signatures used for Spark RCs can be found in this file:
>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1402
>>
>> The documentation corresponding to this release can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>>
>> The list of bug fixes going into 3.3.0 can be found at the following URL:
>> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>>
>> This release is using the release script of the tag v3.3.0-rc1.
>>
>>
>> FAQ
>>
>> =========================
>> How can I help test this release?
>> =========================
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> If you're working in PySpark you can set up a virtual env and install
>> the current RC and see if anything important breaks, in the Java/Scala
>> you can add the staging repository to your projects resolvers and test
>> with the RC (make sure to clean up the artifact cache before/after so
>> you don't end up building with a out of date RC going forward).
>>
>> ===========================================
>> What should happen to JIRA tickets still targeting 3.3.0?
>> ===========================================
>> The current list of open tickets targeted at 3.3.0 can be found at:
>> https://issues.apache.org/jira/projects/SPARK and search for "Target Version/s" = 3.3.0
>>
>> Committers should look at those and triage. Extremely important bug
>> fixes, documentation, and API tweaks that impact compatibility should
>> be worked on immediately. Everything else please retarget to an
>> appropriate release.
>>
>> ==================
>> But my bug isn't fixed?
>> ==================
>> In order to make timely releases, we will typically not hold the
>> release unless the bug in question is a regression from the previous
>> release. That being said, if there is something which is a regression
>> that has not been correctly targeted please ping me or a committer to
>> help target the issue.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>>
>>
>>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Tom Graves <tg...@yahoo.com.INVALID>.
Is there going to be an rc2? I thought there were a couple of issue mentioned in the thread.
Tom
On Tuesday, May 10, 2022, 11:53:36 AM CDT, Maxim Gekk <ma...@databricks.com.invalid> wrote:
Hi All,
Today is the last day for voting. Please, test the RC1 and vote.
Maxim Gekk
Software Engineer
Databricks, Inc.
On Sat, May 7, 2022 at 10:58 AM beliefer <be...@163.com> wrote:
@Maxim Gekk Glad to hear that!
But there is a bug https://github.com/apache/spark/pull/36457
I think we should merge it into 3.3.0
At 2022-05-05 19:00:27, "Maxim Gekk" <ma...@databricks.com.INVALID> wrote:
Please vote on releasing the following candidate as Apache Spark version 3.3.0.
The vote is open until 11:59pm Pacific time May 10th and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 3.3.0
[ ] -1 Do not release this package because ...
To learn more about Apache Spark, please see http://spark.apache.org/
The tag to be voted on is v3.3.0-rc1 (commit 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
https://github.com/apache/spark/tree/v3.3.0-rc1
The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS
The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1402
The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
The list of bug fixes going into 3.3.0 can be found at the following URL:
https://issues.apache.org/jira/projects/SPARK/versions/12350369
This release is using the release script of the tag v3.3.0-rc1.
FAQ
=========================
How can I help test this release?
=========================
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.
If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with a out of date RC going forward).
===========================================
What should happen to JIRA tickets still targeting 3.3.0?
===========================================
The current list of open tickets targeted at 3.3.0 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target Version/s" = 3.3.0
Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.
==================
But my bug isn't fixed?
==================
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.
Maxim Gekk
Software Engineer
Databricks, Inc.
Re: [VOTE] Release Spark 3.3.0 (RC1)
Posted by Maxim Gekk <ma...@databricks.com.INVALID>.
Hi All,
Today is the last day for voting. Please, *test* the RC1 and vote.
Maxim Gekk
Software Engineer
Databricks, Inc.
On Sat, May 7, 2022 at 10:58 AM beliefer <be...@163.com> wrote:
>
> @Maxim Gekk Glad to hear that!
> But there is a bug https://github.com/apache/spark/pull/36457
> I think we should merge it into 3.3.0
>
>
> At 2022-05-05 19:00:27, "Maxim Gekk" <ma...@databricks.com.INVALID>
> wrote:
>
> Please vote on releasing the following candidate as Apache Spark version
> 3.3.0.
>
> The vote is open until 11:59pm Pacific time May 10th and passes if a
> majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.3.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see http://spark.apache.org/
>
> The tag to be voted on is v3.3.0-rc1 (commit
> 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
> https://github.com/apache/spark/tree/v3.3.0-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1402
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
>
> The list of bug fixes going into 3.3.0 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12350369
>
> This release is using the release script of the tag v3.3.0-rc1.
>
>
> FAQ
>
> =========================
> How can I help test this release?
> =========================
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with a out of date RC going forward).
>
> ===========================================
> What should happen to JIRA tickets still targeting 3.3.0?
> ===========================================
> The current list of open tickets targeted at 3.3.0 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.3.0
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==================
> But my bug isn't fixed?
> ==================
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
>
>
>
>
Re:[VOTE] Release Spark 3.3.0 (RC1)
Posted by beliefer <be...@163.com>.
@Maxim Gekk Glad to hear that!
But there is a bug https://github.com/apache/spark/pull/36457
I think we should merge it into 3.3.0
At 2022-05-05 19:00:27, "Maxim Gekk" <ma...@databricks.com.INVALID> wrote:
Please vote on releasing the following candidate as Apache Spark version 3.3.0.
The vote is open until 11:59pm Pacific time May 10th and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 3.3.0
[ ] -1 Do not release this package because ...
To learn more about Apache Spark, please see http://spark.apache.org/
The tag to be voted on is v3.3.0-rc1 (commit 482b7d54b522c4d1e25f3e84eabbc78126f22a3d):
https://github.com/apache/spark/tree/v3.3.0-rc1
The release files, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-bin/
Signatures used for Spark RCs can be found in this file:
https://dist.apache.org/repos/dist/dev/spark/KEYS
The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1402
The documentation corresponding to this release can be found at:
https://dist.apache.org/repos/dist/dev/spark/v3.3.0-rc1-docs/
The list of bug fixes going into 3.3.0 can be found at the following URL:
https://issues.apache.org/jira/projects/SPARK/versions/12350369
This release is using the release script of the tag v3.3.0-rc1.
FAQ
=========================
How can I help test this release?
=========================
If you are a Spark user, you can help us test this release by taking
an existing Spark workload and running on this release candidate, then
reporting any regressions.
If you're working in PySpark you can set up a virtual env and install
the current RC and see if anything important breaks, in the Java/Scala
you can add the staging repository to your projects resolvers and test
with the RC (make sure to clean up the artifact cache before/after so
you don't end up building with a out of date RC going forward).
===========================================
What should happen to JIRA tickets still targeting 3.3.0?
===========================================
The current list of open tickets targeted at 3.3.0 can be found at:
https://issues.apache.org/jira/projects/SPARK and search for "Target Version/s" = 3.3.0
Committers should look at those and triage. Extremely important bug
fixes, documentation, and API tweaks that impact compatibility should
be worked on immediately. Everything else please retarget to an
appropriate release.
==================
But my bug isn't fixed?
==================
In order to make timely releases, we will typically not hold the
release unless the bug in question is a regression from the previous
release. That being said, if there is something which is a regression
that has not been correctly targeted please ping me or a committer to
help target the issue.
Maxim Gekk
Software Engineer
Databricks, Inc.