You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Dongjoon Hyun <do...@gmail.com> on 2022/07/06 20:39:07 UTC
Apache Spark 3.2.2 Release?
Hi, All.
Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
including 11 correctness patches arrived at branch-3.2.
Shall we make a new release, Apache Spark 3.2.2, as the third release
at 3.2 line? I'd like to volunteer as the release manager for Apache
Spark 3.2.2. I'm thinking about starting the first RC next week.
$ git log --oneline v3.2.1..HEAD | wc -l
197
# Correctness issues
SPARK-38075 Hive script transform with order by and limit will
return fake rows
SPARK-38204 All state operators are at a risk of inconsistency
between state partitioning and operator partitioning
SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
and shuffle total blocks metrics
SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
received inputs in the same microbatch
SPARK-38614 After Spark update, df.show() shows incorrect
F.percent_rank results
SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
row whose input is not null
SPARK-38684 Stream-stream outer join has a possible correctness
issue due to weakly read consistent on outer iterators
SPARK-39061 Incorrect results or NPE when using Inline function
against an array of dynamically created structs
SPARK-39107 Silent change in regexp_replace's handling of empty strings
SPARK-39259 Timestamps returned by now() and equivalent functions
are not consistent in subqueries
SPARK-39293 The accumulator of ArrayAggregate should copy the
intermediate result if string, struct, array, or map
Best,
Dongjoon.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Apache Spark 3.2.2 Release?
Posted by Dongjoon Hyun <do...@gmail.com>.
Thank you so much! :)
Dongjoon.
On Thu, Jul 7, 2022 at 6:51 PM Joshua Rosen <jo...@databricks.com> wrote:
>
> +1; thanks for coordinating this!
>
> I have a few more correctness bugs to add to the list in your original email (these were originally missing the 'correctness' JIRA label):
>
> - https://issues.apache.org/jira/browse/SPARK-37643 : when charVarcharAsString is true, char datatype partition table query incorrect
> - https://issues.apache.org/jira/browse/SPARK-37865 : Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns
> - https://issues.apache.org/jira/browse/SPARK-38787 : Possible correctness issue on stream-stream join when handling edge case
>
>
> On Thu, Jul 7, 2022 at 6:12 PM Dongjoon Hyun <do...@gmail.com> wrote:
>>
>> Thank you all.
>>
>> I'll check and prepare RC1 for next week.
>>
>> Dongjoon.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Apache Spark 3.2.2 Release?
Posted by Joshua Rosen <jo...@databricks.com.INVALID>.
+1; thanks for coordinating this!
I have a few more correctness bugs to add to the list in your original
email (these were originally missing the 'correctness' JIRA label):
- https://issues.apache.org/jira/browse/SPARK-37643 : when
charVarcharAsString is true, char datatype partition table query incorrect
- https://issues.apache.org/jira/browse/SPARK-37865 : Spark should not
dedup the groupingExpressions when the first child of Union has duplicate
columns
- https://issues.apache.org/jira/browse/SPARK-38787 : Possible correctness
issue on stream-stream join when handling edge case
On Thu, Jul 7, 2022 at 6:12 PM Dongjoon Hyun <do...@gmail.com>
wrote:
> Thank you all.
>
> I'll check and prepare RC1 for next week.
>
> Dongjoon.
>
Re: Apache Spark 3.2.2 Release?
Posted by Dongjoon Hyun <do...@gmail.com>.
Thank you all.
I'll check and prepare RC1 for next week.
Dongjoon.
Re: Apache Spark 3.2.2 Release?
Posted by Andrew Ray <ra...@gmail.com>.
+1 (non-binding) Thanks!
On Thu, Jul 7, 2022 at 7:00 AM Yang,Jie(INF) <ya...@baidu.com> wrote:
> +1 (non-binding) Thank you Dongjoon ~
>
>
>
> *发件人**: *Ruifeng Zheng <ru...@foxmail.com>
> *日期**: *2022年7月7日 星期四 16:28
> *收件人**: *dev <de...@spark.apache.org>
> *主题**: *Re: Apache Spark 3.2.2 Release?
>
>
>
> +1 thank you Dongjoon!
>
>
> ------------------------------
>
> [image: 图像已被发件人删除。]
>
> Ruifeng Zheng
>
> ruifengz@foxmail.com
>
>
>
>
>
>
>
> ------------------ Original ------------------
>
> *From:* "Yikun Jiang" <yi...@gmail.com>;
>
> *Date:* Thu, Jul 7, 2022 04:16 PM
>
> *To:* "Mridul Muralidharan"<mr...@gmail.com>;
>
> *Cc:* "Gengliang Wang"<lt...@gmail.com>;"Cheng Su"<sc...@gmail.com>;"Maxim
> Gekk"<ma...@databricks.com.invalid>;"Wenchen Fan"<cl...@gmail.com>;"Xiao
> Li"<ga...@gmail.com>;"Xinrong
> Meng"<xi...@databricks.com.invalid>;"Yuming Wang"<wgyumg@gmail.com
> >;"dev"<de...@spark.apache.org>;
>
> *Subject:* Re: Apache Spark 3.2.2 Release?
>
>
>
> +1 (non-binding)
>
>
>
> Thanks!
>
>
> Regards,
>
> Yikun
>
>
>
>
>
> On Thu, Jul 7, 2022 at 1:57 PM Mridul Muralidharan <mr...@gmail.com>
> wrote:
>
> +1
>
>
>
> Thanks for driving this Dongjoon !
>
>
>
> Regards,
>
> Mridul
>
>
>
> On Thu, Jul 7, 2022 at 12:36 AM Gengliang Wang <lt...@gmail.com> wrote:
>
> +1.
>
> Thank you, Dongjoon.
>
>
>
> On Wed, Jul 6, 2022 at 10:21 PM Wenchen Fan <cl...@gmail.com> wrote:
>
> +1
>
>
>
> On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng
> <xi...@databricks.com.invalid> wrote:
>
> +1
>
>
> Thanks!
>
>
>
> Xinrong Meng
>
> Software Engineer
>
> Databricks
>
>
>
>
>
> On Wed, Jul 6, 2022 at 7:25 PM Xiao Li <ga...@gmail.com> wrote:
>
> +1
>
>
>
> Xiao
>
>
>
> Cheng Su <sc...@gmail.com> 于2022年7月6日周三 19:16写道：
>
> +1 (non-binding)
>
>
>
> Thanks,
>
> Cheng Su
>
>
>
> On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com> wrote:
>
> +1
>
>
>
> On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
>
> +1
>
>
>
> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org> wrote:
>
> +1 Thanks for the effort!
>
>
>
> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <bj...@gmail.com>
> wrote:
>
> +1
>
>
>
> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>
> Yeah +1
>
>
>
> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
> Hi, All.
>
> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
> including 11 correctness patches arrived at branch-3.2.
>
> Shall we make a new release, Apache Spark 3.2.2, as the third release
> at 3.2 line? I'd like to volunteer as the release manager for Apache
> Spark 3.2.2. I'm thinking about starting the first RC next week.
>
> $ git log --oneline v3.2.1..HEAD | wc -l
> 197
>
> # Correctness issues
>
> SPARK-38075 Hive script transform with order by and limit will
> return fake rows
> SPARK-38204 All state operators are at a risk of inconsistency
> between state partitioning and operator partitioning
> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
> and shuffle total blocks metrics
> SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
> received inputs in the same microbatch
> SPARK-38614 After Spark update, df.show() shows incorrect
> F.percent_rank results
> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
> row whose input is not null
> SPARK-38684 Stream-stream outer join has a possible correctness
> issue due to weakly read consistent on outer iterators
> SPARK-39061 Incorrect results or NPE when using Inline function
> against an array of dynamically created structs
> SPARK-39107 Silent change in regexp_replace's handling of empty strings
> SPARK-39259 Timestamps returned by now() and equivalent functions
> are not consistent in subqueries
> SPARK-39293 The accumulator of ArrayAggregate should copy the
> intermediate result if string, struct, array, or map
>
> Best,
> Dongjoon.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
> --
>
> John Zhuge
>
>
Re: Apache Spark 3.2.2 Release?
Posted by "Yang,Jie(INF)" <ya...@baidu.com>.
+1 (non-binding) Thank you Dongjoon ~
发件人: Ruifeng Zheng <ru...@foxmail.com>
日期: 2022年7月7日 星期四 16:28
收件人: dev <de...@spark.apache.org>
主题: Re: Apache Spark 3.2.2 Release?
+1 thank you Dongjoon!
________________________________
[图像已被发件人删除。]
Ruifeng Zheng
ruifengz@foxmail.com
------------------ Original ------------------
From: "Yikun Jiang" <yi...@gmail.com>;
Date: Thu, Jul 7, 2022 04:16 PM
To: "Mridul Muralidharan"<mr...@gmail.com>;
Cc: "Gengliang Wang"<lt...@gmail.com>;"Cheng Su"<sc...@gmail.com>;"Maxim Gekk"<ma...@databricks.com.invalid>;"Wenchen Fan"<cl...@gmail.com>;"Xiao Li"<ga...@gmail.com>;"Xinrong Meng"<xi...@databricks.com.invalid>;"Yuming Wang"<wg...@spark.apache.org>;
Subject: Re: Apache Spark 3.2.2 Release?
+1 (non-binding)
Thanks!
Regards,
Yikun
On Thu, Jul 7, 2022 at 1:57 PM Mridul Muralidharan <mr...@gmail.com>> wrote:
+1
Thanks for driving this Dongjoon !
Regards,
Mridul
On Thu, Jul 7, 2022 at 12:36 AM Gengliang Wang <lt...@gmail.com>> wrote:
+1.
Thank you, Dongjoon.
On Wed, Jul 6, 2022 at 10:21 PM Wenchen Fan <cl...@gmail.com>> wrote:
+1
On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng <xi...@databricks.com.invalid> wrote:
+1
Thanks!
Xinrong Meng
Software Engineer
Databricks
On Wed, Jul 6, 2022 at 7:25 PM Xiao Li <ga...@gmail.com>> wrote:
+1
Xiao
Cheng Su <sc...@gmail.com>> 于2022年7月6日周三 19:16写道：
+1 (non-binding)
Thanks,
Cheng Su
On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com>> wrote:
+1
On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk <ma...@databricks.com.invalid> wrote:
+1
On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org>> wrote:
+1 Thanks for the effort!
On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <bj...@gmail.com>> wrote:
+1
ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>>:
Yeah +1
On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>> wrote:
Hi, All.
Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
including 11 correctness patches arrived at branch-3.2.
Shall we make a new release, Apache Spark 3.2.2, as the third release
at 3.2 line? I'd like to volunteer as the release manager for Apache
Spark 3.2.2. I'm thinking about starting the first RC next week.
$ git log --oneline v3.2.1..HEAD | wc -l
197
# Correctness issues
SPARK-38075 Hive script transform with order by and limit will
return fake rows
SPARK-38204 All state operators are at a risk of inconsistency
between state partitioning and operator partitioning
SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
and shuffle total blocks metrics
SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
received inputs in the same microbatch
SPARK-38614 After Spark update, df.show() shows incorrect
F.percent_rank results
SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
row whose input is not null
SPARK-38684 Stream-stream outer join has a possible correctness
issue due to weakly read consistent on outer iterators
SPARK-39061 Incorrect results or NPE when using Inline function
against an array of dynamically created structs
SPARK-39107 Silent change in regexp_replace's handling of empty strings
SPARK-39259 Timestamps returned by now() and equivalent functions
are not consistent in subqueries
SPARK-39293 The accumulator of ArrayAggregate should copy the
intermediate result if string, struct, array, or map
Best,
Dongjoon.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org<ma...@spark.apache.org>
--
John Zhuge
Re: Apache Spark 3.2.2 Release?
Posted by Ruifeng Zheng <ru...@foxmail.com>.
+1 thank you Dongjoon!
Ruifeng Zheng
ruifengz@foxmail.com
------------------ Original ------------------
From: "Yikun Jiang" <yikunkero@gmail.com>;
Date: Thu, Jul 7, 2022 04:16 PM
To: "Mridul Muralidharan"<mridul@gmail.com>;
Cc: "Gengliang Wang"<ltnwgl@gmail.com>;"Cheng Su"<scnju13@gmail.com>;"Maxim Gekk"<maxim.gekk@databricks.com.invalid>;"Wenchen Fan"<cloud0fan@gmail.com>;"Xiao Li"<gatorsmile@gmail.com>;"Xinrong Meng"<xinrong.meng@databricks.com.invalid>;"Yuming Wang"<wgyumg@gmail.com>;"dev"<dev@spark.apache.org>;
Subject: Re: Apache Spark 3.2.2 Release?
+1 (non-binding)
Thanks!
Regards,
Yikun
On Thu, Jul 7, 2022 at 1:57 PM Mridul Muralidharan <mridul@gmail.com> wrote:
+1
Thanks for driving this Dongjoon !
Regards,
Mridul
On Thu, Jul 7, 2022 at 12:36 AM Gengliang Wang <ltnwgl@gmail.com> wrote:
+1. Thank you, Dongjoon.
On Wed, Jul 6, 2022 at 10:21 PM Wenchen Fan <cloud0fan@gmail.com> wrote:
+1
On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng <xinrong.meng@databricks.com.invalid> wrote:
+1
Thanks!
Xinrong Meng
Software Engineer
Databricks
On Wed, Jul 6, 2022 at 7:25 PM Xiao Li <gatorsmile@gmail.com> wrote:
+1
Xiao
Cheng Su <scnju13@gmail.com> 于2022年7月6日周三 19:16写道：
+1 (non-binding)
Thanks,
Cheng Su
On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wgyumg@gmail.com> wrote:
+1
On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk <maxim.gekk@databricks.com.invalid> wrote:
+1
On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jzhuge@apache.org> wrote:
+1 Thanks for the effort!
On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <bjornjorgensen@gmail.com> wrote:
+1
ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gurwls223@gmail.com>:
Yeah +1
On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <dongjoon.hyun@gmail.com> wrote:
Hi, All.
Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
including 11 correctness patches arrived at branch-3.2.
Shall we make a new release, Apache Spark 3.2.2, as the third release
at 3.2 line? I'd like to volunteer as the release manager for Apache
Spark 3.2.2. I'm thinking about starting the first RC next week.
$ git log --oneline v3.2.1..HEAD | wc -l
197
# Correctness issues
SPARK-38075 Hive script transform with order by and limit will
return fake rows
SPARK-38204 All state operators are at a risk of inconsistency
between state partitioning and operator partitioning
SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
and shuffle total blocks metrics
SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
received inputs in the same microbatch
SPARK-38614 After Spark update, df.show() shows incorrect
F.percent_rank results
SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
row whose input is not null
SPARK-38684 Stream-stream outer join has a possible correctness
issue due to weakly read consistent on outer iterators
SPARK-39061 Incorrect results or NPE when using Inline function
against an array of dynamically created structs
SPARK-39107 Silent change in regexp_replace's handling of empty strings
SPARK-39259 Timestamps returned by now() and equivalent functions
are not consistent in subqueries
SPARK-39293 The accumulator of ArrayAggregate should copy the
intermediate result if string, struct, array, or map
Best,
Dongjoon.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
--
John Zhuge
Re: Apache Spark 3.2.2 Release?
Posted by Yikun Jiang <yi...@gmail.com>.
+1 (non-binding)
Thanks!
Regards,
Yikun
On Thu, Jul 7, 2022 at 1:57 PM Mridul Muralidharan <mr...@gmail.com> wrote:
> +1
>
> Thanks for driving this Dongjoon !
>
> Regards,
> Mridul
>
> On Thu, Jul 7, 2022 at 12:36 AM Gengliang Wang <lt...@gmail.com> wrote:
>
>> +1.
>> Thank you, Dongjoon.
>>
>> On Wed, Jul 6, 2022 at 10:21 PM Wenchen Fan <cl...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng
>>> <xi...@databricks.com.invalid> wrote:
>>>
>>>> +1
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> Xinrong Meng
>>>>
>>>> Software Engineer
>>>>
>>>> Databricks
>>>>
>>>>
>>>> On Wed, Jul 6, 2022 at 7:25 PM Xiao Li <ga...@gmail.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Xiao
>>>>>
>>>>> Cheng Su <sc...@gmail.com> 于2022年7月6日周三 19:16写道：
>>>>>
>>>>>> +1 (non-binding)
>>>>>>
>>>>>> Thanks,
>>>>>> Cheng Su
>>>>>>
>>>>>> On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com> wrote:
>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk
>>>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>>>
>>>>>>>> +1
>>>>>>>>
>>>>>>>> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1 Thanks for the effort!
>>>>>>>>>
>>>>>>>>> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <
>>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> +1
>>>>>>>>>>
>>>>>>>>>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gurwls223@gmail.com
>>>>>>>>>> >:
>>>>>>>>>>
>>>>>>>>>>> Yeah +1
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <
>>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi, All.
>>>>>>>>>>>>
>>>>>>>>>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>>>>>>>>>> including 11 correctness patches arrived at branch-3.2.
>>>>>>>>>>>>
>>>>>>>>>>>> Shall we make a new release, Apache Spark 3.2.2, as the third
>>>>>>>>>>>> release
>>>>>>>>>>>> at 3.2 line? I'd like to volunteer as the release manager for
>>>>>>>>>>>> Apache
>>>>>>>>>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>>>>>>>>>
>>>>>>>>>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>>>>>>>>>> 197
>>>>>>>>>>>>
>>>>>>>>>>>> # Correctness issues
>>>>>>>>>>>>
>>>>>>>>>>>> SPARK-38075 Hive script transform with order by and limit
>>>>>>>>>>>> will
>>>>>>>>>>>> return fake rows
>>>>>>>>>>>> SPARK-38204 All state operators are at a risk of
>>>>>>>>>>>> inconsistency
>>>>>>>>>>>> between state partitioning and operator partitioning
>>>>>>>>>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read
>>>>>>>>>>>> bytes
>>>>>>>>>>>> and shuffle total blocks metrics
>>>>>>>>>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups
>>>>>>>>>>>> which just
>>>>>>>>>>>> received inputs in the same microbatch
>>>>>>>>>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>>>>>>>>>> F.percent_rank results
>>>>>>>>>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the
>>>>>>>>>>>> offset
>>>>>>>>>>>> row whose input is not null
>>>>>>>>>>>> SPARK-38684 Stream-stream outer join has a possible
>>>>>>>>>>>> correctness
>>>>>>>>>>>> issue due to weakly read consistent on outer iterators
>>>>>>>>>>>> SPARK-39061 Incorrect results or NPE when using Inline
>>>>>>>>>>>> function
>>>>>>>>>>>> against an array of dynamically created structs
>>>>>>>>>>>> SPARK-39107 Silent change in regexp_replace's handling of
>>>>>>>>>>>> empty strings
>>>>>>>>>>>> SPARK-39259 Timestamps returned by now() and equivalent
>>>>>>>>>>>> functions
>>>>>>>>>>>> are not consistent in subqueries
>>>>>>>>>>>> SPARK-39293 The accumulator of ArrayAggregate should copy
>>>>>>>>>>>> the
>>>>>>>>>>>> intermediate result if string, struct, array, or map
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>> John Zhuge
>>>>>>>>>
>>>>>>>>
Re: Apache Spark 3.2.2 Release?
Posted by Mridul Muralidharan <mr...@gmail.com>.
+1
Thanks for driving this Dongjoon !
Regards,
Mridul
On Thu, Jul 7, 2022 at 12:36 AM Gengliang Wang <lt...@gmail.com> wrote:
> +1.
> Thank you, Dongjoon.
>
> On Wed, Jul 6, 2022 at 10:21 PM Wenchen Fan <cl...@gmail.com> wrote:
>
>> +1
>>
>> On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng
>> <xi...@databricks.com.invalid> wrote:
>>
>>> +1
>>>
>>> Thanks!
>>>
>>>
>>> Xinrong Meng
>>>
>>> Software Engineer
>>>
>>> Databricks
>>>
>>>
>>> On Wed, Jul 6, 2022 at 7:25 PM Xiao Li <ga...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> Xiao
>>>>
>>>> Cheng Su <sc...@gmail.com> 于2022年7月6日周三 19:16写道：
>>>>
>>>>> +1 (non-binding)
>>>>>
>>>>> Thanks,
>>>>> Cheng Su
>>>>>
>>>>> On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk
>>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 Thanks for the effort!
>>>>>>>>
>>>>>>>> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <
>>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> +1
>>>>>>>>>
>>>>>>>>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Yeah +1
>>>>>>>>>>
>>>>>>>>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <
>>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi, All.
>>>>>>>>>>>
>>>>>>>>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>>>>>>>>> including 11 correctness patches arrived at branch-3.2.
>>>>>>>>>>>
>>>>>>>>>>> Shall we make a new release, Apache Spark 3.2.2, as the third
>>>>>>>>>>> release
>>>>>>>>>>> at 3.2 line? I'd like to volunteer as the release manager for
>>>>>>>>>>> Apache
>>>>>>>>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>>>>>>>>
>>>>>>>>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>>>>>>>>> 197
>>>>>>>>>>>
>>>>>>>>>>> # Correctness issues
>>>>>>>>>>>
>>>>>>>>>>> SPARK-38075 Hive script transform with order by and limit
>>>>>>>>>>> will
>>>>>>>>>>> return fake rows
>>>>>>>>>>> SPARK-38204 All state operators are at a risk of
>>>>>>>>>>> inconsistency
>>>>>>>>>>> between state partitioning and operator partitioning
>>>>>>>>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read
>>>>>>>>>>> bytes
>>>>>>>>>>> and shuffle total blocks metrics
>>>>>>>>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups
>>>>>>>>>>> which just
>>>>>>>>>>> received inputs in the same microbatch
>>>>>>>>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>>>>>>>>> F.percent_rank results
>>>>>>>>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the
>>>>>>>>>>> offset
>>>>>>>>>>> row whose input is not null
>>>>>>>>>>> SPARK-38684 Stream-stream outer join has a possible
>>>>>>>>>>> correctness
>>>>>>>>>>> issue due to weakly read consistent on outer iterators
>>>>>>>>>>> SPARK-39061 Incorrect results or NPE when using Inline
>>>>>>>>>>> function
>>>>>>>>>>> against an array of dynamically created structs
>>>>>>>>>>> SPARK-39107 Silent change in regexp_replace's handling of
>>>>>>>>>>> empty strings
>>>>>>>>>>> SPARK-39259 Timestamps returned by now() and equivalent
>>>>>>>>>>> functions
>>>>>>>>>>> are not consistent in subqueries
>>>>>>>>>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>>>>>>>>>> intermediate result if string, struct, array, or map
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>> John Zhuge
>>>>>>>>
>>>>>>>
Re: Apache Spark 3.2.2 Release?
Posted by Gengliang Wang <lt...@gmail.com>.
+1.
Thank you, Dongjoon.
On Wed, Jul 6, 2022 at 10:21 PM Wenchen Fan <cl...@gmail.com> wrote:
> +1
>
> On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng
> <xi...@databricks.com.invalid> wrote:
>
>> +1
>>
>> Thanks!
>>
>>
>> Xinrong Meng
>>
>> Software Engineer
>>
>> Databricks
>>
>>
>> On Wed, Jul 6, 2022 at 7:25 PM Xiao Li <ga...@gmail.com> wrote:
>>
>>> +1
>>>
>>> Xiao
>>>
>>> Cheng Su <sc...@gmail.com> 于2022年7月6日周三 19:16写道：
>>>
>>>> +1 (non-binding)
>>>>
>>>> Thanks,
>>>> Cheng Su
>>>>
>>>> On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk
>>>>> <ma...@databricks.com.invalid> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org> wrote:
>>>>>>
>>>>>>> +1 Thanks for the effort!
>>>>>>>
>>>>>>> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <
>>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>>
>>>>>>>> +1
>>>>>>>>
>>>>>>>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>>>>>>>>
>>>>>>>>> Yeah +1
>>>>>>>>>
>>>>>>>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <
>>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi, All.
>>>>>>>>>>
>>>>>>>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>>>>>>>> including 11 correctness patches arrived at branch-3.2.
>>>>>>>>>>
>>>>>>>>>> Shall we make a new release, Apache Spark 3.2.2, as the third
>>>>>>>>>> release
>>>>>>>>>> at 3.2 line? I'd like to volunteer as the release manager for
>>>>>>>>>> Apache
>>>>>>>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>>>>>>>
>>>>>>>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>>>>>>>> 197
>>>>>>>>>>
>>>>>>>>>> # Correctness issues
>>>>>>>>>>
>>>>>>>>>> SPARK-38075 Hive script transform with order by and limit will
>>>>>>>>>> return fake rows
>>>>>>>>>> SPARK-38204 All state operators are at a risk of inconsistency
>>>>>>>>>> between state partitioning and operator partitioning
>>>>>>>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read
>>>>>>>>>> bytes
>>>>>>>>>> and shuffle total blocks metrics
>>>>>>>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which
>>>>>>>>>> just
>>>>>>>>>> received inputs in the same microbatch
>>>>>>>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>>>>>>>> F.percent_rank results
>>>>>>>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the
>>>>>>>>>> offset
>>>>>>>>>> row whose input is not null
>>>>>>>>>> SPARK-38684 Stream-stream outer join has a possible
>>>>>>>>>> correctness
>>>>>>>>>> issue due to weakly read consistent on outer iterators
>>>>>>>>>> SPARK-39061 Incorrect results or NPE when using Inline
>>>>>>>>>> function
>>>>>>>>>> against an array of dynamically created structs
>>>>>>>>>> SPARK-39107 Silent change in regexp_replace's handling of
>>>>>>>>>> empty strings
>>>>>>>>>> SPARK-39259 Timestamps returned by now() and equivalent
>>>>>>>>>> functions
>>>>>>>>>> are not consistent in subqueries
>>>>>>>>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>>>>>>>>> intermediate result if string, struct, array, or map
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Dongjoon.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>> John Zhuge
>>>>>>>
>>>>>>
Re: Apache Spark 3.2.2 Release?
Posted by Wenchen Fan <cl...@gmail.com>.
+1
On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng
<xi...@databricks.com.invalid> wrote:
> +1
>
> Thanks!
>
>
> Xinrong Meng
>
> Software Engineer
>
> Databricks
>
>
> On Wed, Jul 6, 2022 at 7:25 PM Xiao Li <ga...@gmail.com> wrote:
>
>> +1
>>
>> Xiao
>>
>> Cheng Su <sc...@gmail.com> 于2022年7月6日周三 19:16写道：
>>
>>> +1 (non-binding)
>>>
>>> Thanks,
>>> Cheng Su
>>>
>>> On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk
>>>> <ma...@databricks.com.invalid> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org> wrote:
>>>>>
>>>>>> +1 Thanks for the effort!
>>>>>>
>>>>>> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <
>>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>>>>>>>
>>>>>>>> Yeah +1
>>>>>>>>
>>>>>>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <
>>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi, All.
>>>>>>>>>
>>>>>>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>>>>>>> including 11 correctness patches arrived at branch-3.2.
>>>>>>>>>
>>>>>>>>> Shall we make a new release, Apache Spark 3.2.2, as the third
>>>>>>>>> release
>>>>>>>>> at 3.2 line? I'd like to volunteer as the release manager for
>>>>>>>>> Apache
>>>>>>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>>>>>>
>>>>>>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>>>>>>> 197
>>>>>>>>>
>>>>>>>>> # Correctness issues
>>>>>>>>>
>>>>>>>>> SPARK-38075 Hive script transform with order by and limit will
>>>>>>>>> return fake rows
>>>>>>>>> SPARK-38204 All state operators are at a risk of inconsistency
>>>>>>>>> between state partitioning and operator partitioning
>>>>>>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read
>>>>>>>>> bytes
>>>>>>>>> and shuffle total blocks metrics
>>>>>>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which
>>>>>>>>> just
>>>>>>>>> received inputs in the same microbatch
>>>>>>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>>>>>>> F.percent_rank results
>>>>>>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the
>>>>>>>>> offset
>>>>>>>>> row whose input is not null
>>>>>>>>> SPARK-38684 Stream-stream outer join has a possible correctness
>>>>>>>>> issue due to weakly read consistent on outer iterators
>>>>>>>>> SPARK-39061 Incorrect results or NPE when using Inline function
>>>>>>>>> against an array of dynamically created structs
>>>>>>>>> SPARK-39107 Silent change in regexp_replace's handling of
>>>>>>>>> empty strings
>>>>>>>>> SPARK-39259 Timestamps returned by now() and equivalent
>>>>>>>>> functions
>>>>>>>>> are not consistent in subqueries
>>>>>>>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>>>>>>>> intermediate result if string, struct, array, or map
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Dongjoon.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>>>
>>>>>>>>> --
>>>>>> John Zhuge
>>>>>>
>>>>>
Re: Apache Spark 3.2.2 Release?
Posted by Xinrong Meng <xi...@databricks.com.INVALID>.
+1
Thanks!
Xinrong Meng
Software Engineer
Databricks
On Wed, Jul 6, 2022 at 7:25 PM Xiao Li <ga...@gmail.com> wrote:
> +1
>
> Xiao
>
> Cheng Su <sc...@gmail.com> 于2022年7月6日周三 19:16写道：
>
>> +1 (non-binding)
>>
>> Thanks,
>> Cheng Su
>>
>> On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk
>>> <ma...@databricks.com.invalid> wrote:
>>>
>>>> +1
>>>>
>>>> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org> wrote:
>>>>
>>>>> +1 Thanks for the effort!
>>>>>
>>>>> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <
>>>>> bjornjorgensen@gmail.com> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>>>>>>
>>>>>>> Yeah +1
>>>>>>>
>>>>>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <
>>>>>>> dongjoon.hyun@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi, All.
>>>>>>>>
>>>>>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>>>>>> including 11 correctness patches arrived at branch-3.2.
>>>>>>>>
>>>>>>>> Shall we make a new release, Apache Spark 3.2.2, as the third
>>>>>>>> release
>>>>>>>> at 3.2 line? I'd like to volunteer as the release manager for Apache
>>>>>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>>>>>
>>>>>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>>>>>> 197
>>>>>>>>
>>>>>>>> # Correctness issues
>>>>>>>>
>>>>>>>> SPARK-38075 Hive script transform with order by and limit will
>>>>>>>> return fake rows
>>>>>>>> SPARK-38204 All state operators are at a risk of inconsistency
>>>>>>>> between state partitioning and operator partitioning
>>>>>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
>>>>>>>> and shuffle total blocks metrics
>>>>>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which
>>>>>>>> just
>>>>>>>> received inputs in the same microbatch
>>>>>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>>>>>> F.percent_rank results
>>>>>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
>>>>>>>> row whose input is not null
>>>>>>>> SPARK-38684 Stream-stream outer join has a possible correctness
>>>>>>>> issue due to weakly read consistent on outer iterators
>>>>>>>> SPARK-39061 Incorrect results or NPE when using Inline function
>>>>>>>> against an array of dynamically created structs
>>>>>>>> SPARK-39107 Silent change in regexp_replace's handling of empty
>>>>>>>> strings
>>>>>>>> SPARK-39259 Timestamps returned by now() and equivalent
>>>>>>>> functions
>>>>>>>> are not consistent in subqueries
>>>>>>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>>>>>>> intermediate result if string, struct, array, or map
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>>
>>>>>>>> --
>>>>> John Zhuge
>>>>>
>>>>
Re: Apache Spark 3.2.2 Release?
Posted by Xiao Li <ga...@gmail.com>.
+1
Xiao
Cheng Su <sc...@gmail.com> 于2022年7月6日周三 19:16写道：
> +1 (non-binding)
>
> Thanks,
> Cheng Su
>
> On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com> wrote:
>
>> +1
>>
>> On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk
>> <ma...@databricks.com.invalid> wrote:
>>
>>> +1
>>>
>>> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org> wrote:
>>>
>>>> +1 Thanks for the effort!
>>>>
>>>> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <
>>>> bjornjorgensen@gmail.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>>>>>
>>>>>> Yeah +1
>>>>>>
>>>>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi, All.
>>>>>>>
>>>>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>>>>> including 11 correctness patches arrived at branch-3.2.
>>>>>>>
>>>>>>> Shall we make a new release, Apache Spark 3.2.2, as the third release
>>>>>>> at 3.2 line? I'd like to volunteer as the release manager for Apache
>>>>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>>>>
>>>>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>>>>> 197
>>>>>>>
>>>>>>> # Correctness issues
>>>>>>>
>>>>>>> SPARK-38075 Hive script transform with order by and limit will
>>>>>>> return fake rows
>>>>>>> SPARK-38204 All state operators are at a risk of inconsistency
>>>>>>> between state partitioning and operator partitioning
>>>>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
>>>>>>> and shuffle total blocks metrics
>>>>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which
>>>>>>> just
>>>>>>> received inputs in the same microbatch
>>>>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>>>>> F.percent_rank results
>>>>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
>>>>>>> row whose input is not null
>>>>>>> SPARK-38684 Stream-stream outer join has a possible correctness
>>>>>>> issue due to weakly read consistent on outer iterators
>>>>>>> SPARK-39061 Incorrect results or NPE when using Inline function
>>>>>>> against an array of dynamically created structs
>>>>>>> SPARK-39107 Silent change in regexp_replace's handling of empty
>>>>>>> strings
>>>>>>> SPARK-39259 Timestamps returned by now() and equivalent functions
>>>>>>> are not consistent in subqueries
>>>>>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>>>>>> intermediate result if string, struct, array, or map
>>>>>>>
>>>>>>> Best,
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>
>>>>>>> --
>>>> John Zhuge
>>>>
>>>
Re: Apache Spark 3.2.2 Release?
Posted by Cheng Su <sc...@gmail.com>.
+1 (non-binding)
Thanks,
Cheng Su
On Wed, Jul 6, 2022 at 6:01 PM Yuming Wang <wg...@gmail.com> wrote:
> +1
>
> On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
>
>> +1
>>
>> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org> wrote:
>>
>>> +1 Thanks for the effort!
>>>
>>> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <bj...@gmail.com>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>>>>
>>>>> Yeah +1
>>>>>
>>>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi, All.
>>>>>>
>>>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>>>> including 11 correctness patches arrived at branch-3.2.
>>>>>>
>>>>>> Shall we make a new release, Apache Spark 3.2.2, as the third release
>>>>>> at 3.2 line? I'd like to volunteer as the release manager for Apache
>>>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>>>
>>>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>>>> 197
>>>>>>
>>>>>> # Correctness issues
>>>>>>
>>>>>> SPARK-38075 Hive script transform with order by and limit will
>>>>>> return fake rows
>>>>>> SPARK-38204 All state operators are at a risk of inconsistency
>>>>>> between state partitioning and operator partitioning
>>>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
>>>>>> and shuffle total blocks metrics
>>>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
>>>>>> received inputs in the same microbatch
>>>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>>>> F.percent_rank results
>>>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
>>>>>> row whose input is not null
>>>>>> SPARK-38684 Stream-stream outer join has a possible correctness
>>>>>> issue due to weakly read consistent on outer iterators
>>>>>> SPARK-39061 Incorrect results or NPE when using Inline function
>>>>>> against an array of dynamically created structs
>>>>>> SPARK-39107 Silent change in regexp_replace's handling of empty
>>>>>> strings
>>>>>> SPARK-39259 Timestamps returned by now() and equivalent functions
>>>>>> are not consistent in subqueries
>>>>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>>>>> intermediate result if string, struct, array, or map
>>>>>>
>>>>>> Best,
>>>>>> Dongjoon.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>
>>>>>> --
>>> John Zhuge
>>>
>>
Re: Apache Spark 3.2.2 Release?
Posted by Yuming Wang <wg...@gmail.com>.
+1
On Thu, Jul 7, 2022 at 5:53 AM Maxim Gekk <ma...@databricks.com.invalid>
wrote:
> +1
>
> On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org> wrote:
>
>> +1 Thanks for the effort!
>>
>> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <bj...@gmail.com>
>> wrote:
>>
>>> +1
>>>
>>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>>>
>>>> Yeah +1
>>>>
>>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi, All.
>>>>>
>>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>>> including 11 correctness patches arrived at branch-3.2.
>>>>>
>>>>> Shall we make a new release, Apache Spark 3.2.2, as the third release
>>>>> at 3.2 line? I'd like to volunteer as the release manager for Apache
>>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>>
>>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>>> 197
>>>>>
>>>>> # Correctness issues
>>>>>
>>>>> SPARK-38075 Hive script transform with order by and limit will
>>>>> return fake rows
>>>>> SPARK-38204 All state operators are at a risk of inconsistency
>>>>> between state partitioning and operator partitioning
>>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
>>>>> and shuffle total blocks metrics
>>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
>>>>> received inputs in the same microbatch
>>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>>> F.percent_rank results
>>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
>>>>> row whose input is not null
>>>>> SPARK-38684 Stream-stream outer join has a possible correctness
>>>>> issue due to weakly read consistent on outer iterators
>>>>> SPARK-39061 Incorrect results or NPE when using Inline function
>>>>> against an array of dynamically created structs
>>>>> SPARK-39107 Silent change in regexp_replace's handling of empty
>>>>> strings
>>>>> SPARK-39259 Timestamps returned by now() and equivalent functions
>>>>> are not consistent in subqueries
>>>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>>>> intermediate result if string, struct, array, or map
>>>>>
>>>>> Best,
>>>>> Dongjoon.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>
>>>>> --
>> John Zhuge
>>
>
Re: Apache Spark 3.2.2 Release?
Posted by Maxim Gekk <ma...@databricks.com.INVALID>.
+1
On Thu, Jul 7, 2022 at 12:26 AM John Zhuge <jz...@apache.org> wrote:
> +1 Thanks for the effort!
>
> On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <bj...@gmail.com>
> wrote:
>
>> +1
>>
>> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>>
>>> Yeah +1
>>>
>>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>
>>> wrote:
>>>
>>>> Hi, All.
>>>>
>>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>>> including 11 correctness patches arrived at branch-3.2.
>>>>
>>>> Shall we make a new release, Apache Spark 3.2.2, as the third release
>>>> at 3.2 line? I'd like to volunteer as the release manager for Apache
>>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>>
>>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>>> 197
>>>>
>>>> # Correctness issues
>>>>
>>>> SPARK-38075 Hive script transform with order by and limit will
>>>> return fake rows
>>>> SPARK-38204 All state operators are at a risk of inconsistency
>>>> between state partitioning and operator partitioning
>>>> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
>>>> and shuffle total blocks metrics
>>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
>>>> received inputs in the same microbatch
>>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>>> F.percent_rank results
>>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
>>>> row whose input is not null
>>>> SPARK-38684 Stream-stream outer join has a possible correctness
>>>> issue due to weakly read consistent on outer iterators
>>>> SPARK-39061 Incorrect results or NPE when using Inline function
>>>> against an array of dynamically created structs
>>>> SPARK-39107 Silent change in regexp_replace's handling of empty
>>>> strings
>>>> SPARK-39259 Timestamps returned by now() and equivalent functions
>>>> are not consistent in subqueries
>>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>>> intermediate result if string, struct, array, or map
>>>>
>>>> Best,
>>>> Dongjoon.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>
>>>> --
> John Zhuge
>
Re: Apache Spark 3.2.2 Release?
Posted by John Zhuge <jz...@apache.org>.
+1 Thanks for the effort!
On Wed, Jul 6, 2022 at 2:23 PM Bjørn Jørgensen <bj...@gmail.com>
wrote:
> +1
>
> ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
>
>> Yeah +1
>>
>> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>
>> wrote:
>>
>>> Hi, All.
>>>
>>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>>> including 11 correctness patches arrived at branch-3.2.
>>>
>>> Shall we make a new release, Apache Spark 3.2.2, as the third release
>>> at 3.2 line? I'd like to volunteer as the release manager for Apache
>>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>>
>>> $ git log --oneline v3.2.1..HEAD | wc -l
>>> 197
>>>
>>> # Correctness issues
>>>
>>> SPARK-38075 Hive script transform with order by and limit will
>>> return fake rows
>>> SPARK-38204 All state operators are at a risk of inconsistency
>>> between state partitioning and operator partitioning
>>> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
>>> and shuffle total blocks metrics
>>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
>>> received inputs in the same microbatch
>>> SPARK-38614 After Spark update, df.show() shows incorrect
>>> F.percent_rank results
>>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
>>> row whose input is not null
>>> SPARK-38684 Stream-stream outer join has a possible correctness
>>> issue due to weakly read consistent on outer iterators
>>> SPARK-39061 Incorrect results or NPE when using Inline function
>>> against an array of dynamically created structs
>>> SPARK-39107 Silent change in regexp_replace's handling of empty
>>> strings
>>> SPARK-39259 Timestamps returned by now() and equivalent functions
>>> are not consistent in subqueries
>>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>>> intermediate result if string, struct, array, or map
>>>
>>> Best,
>>> Dongjoon.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>> --
John Zhuge
Re: Apache Spark 3.2.2 Release?
Posted by Bjørn Jørgensen <bj...@gmail.com>.
+1
ons. 6. jul. 2022, 23:05 skrev Hyukjin Kwon <gu...@gmail.com>:
> Yeah +1
>
> On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
>> Hi, All.
>>
>> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
>> including 11 correctness patches arrived at branch-3.2.
>>
>> Shall we make a new release, Apache Spark 3.2.2, as the third release
>> at 3.2 line? I'd like to volunteer as the release manager for Apache
>> Spark 3.2.2. I'm thinking about starting the first RC next week.
>>
>> $ git log --oneline v3.2.1..HEAD | wc -l
>> 197
>>
>> # Correctness issues
>>
>> SPARK-38075 Hive script transform with order by and limit will
>> return fake rows
>> SPARK-38204 All state operators are at a risk of inconsistency
>> between state partitioning and operator partitioning
>> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
>> and shuffle total blocks metrics
>> SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
>> received inputs in the same microbatch
>> SPARK-38614 After Spark update, df.show() shows incorrect
>> F.percent_rank results
>> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
>> row whose input is not null
>> SPARK-38684 Stream-stream outer join has a possible correctness
>> issue due to weakly read consistent on outer iterators
>> SPARK-39061 Incorrect results or NPE when using Inline function
>> against an array of dynamically created structs
>> SPARK-39107 Silent change in regexp_replace's handling of empty
>> strings
>> SPARK-39259 Timestamps returned by now() and equivalent functions
>> are not consistent in subqueries
>> SPARK-39293 The accumulator of ArrayAggregate should copy the
>> intermediate result if string, struct, array, or map
>>
>> Best,
>> Dongjoon.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
Re: Apache Spark 3.2.2 Release?
Posted by Hyukjin Kwon <gu...@gmail.com>.
Yeah +1
On Thu, Jul 7, 2022 at 5:40 AM Dongjoon Hyun <do...@gmail.com>
wrote:
> Hi, All.
>
> Since Apache Spark 3.2.1 tag creation (Jan 19), new 197 patches
> including 11 correctness patches arrived at branch-3.2.
>
> Shall we make a new release, Apache Spark 3.2.2, as the third release
> at 3.2 line? I'd like to volunteer as the release manager for Apache
> Spark 3.2.2. I'm thinking about starting the first RC next week.
>
> $ git log --oneline v3.2.1..HEAD | wc -l
> 197
>
> # Correctness issues
>
> SPARK-38075 Hive script transform with order by and limit will
> return fake rows
> SPARK-38204 All state operators are at a risk of inconsistency
> between state partitioning and operator partitioning
> SPARK-38309 SHS has incorrect percentiles for shuffle read bytes
> and shuffle total blocks metrics
> SPARK-38320 (flat)MapGroupsWithState can timeout groups which just
> received inputs in the same microbatch
> SPARK-38614 After Spark update, df.show() shows incorrect
> F.percent_rank results
> SPARK-38655 OffsetWindowFunctionFrameBase cannot find the offset
> row whose input is not null
> SPARK-38684 Stream-stream outer join has a possible correctness
> issue due to weakly read consistent on outer iterators
> SPARK-39061 Incorrect results or NPE when using Inline function
> against an array of dynamically created structs
> SPARK-39107 Silent change in regexp_replace's handling of empty strings
> SPARK-39259 Timestamps returned by now() and equivalent functions
> are not consistent in subqueries
> SPARK-39293 The accumulator of ArrayAggregate should copy the
> intermediate result if string, struct, array, or map
>
> Best,
> Dongjoon.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>