You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Yi Wu <yi...@databricks.com> on 2021/06/09 03:05:56 UTC
Apache Spark 3.0.3 Release?
Hi, All.
Since Apache Spark 3.0.2 tag creation (Feb 16),
new 119 patches (92 issues
<https://issues.apache.org/jira/projects/SPARK/versions/12349723> resolved)
arrived at branch-3.0.
Shall we make a new release, Apache Spark 3.0.3, as the 3rd release at the
3.0 line?
I'd like to volunteer as the release manager for Apache Spark 3.0.3.
I'm thinking about starting the first RC at the end of this week.
$ git log --oneline v3.0.2..HEAD | wc -l
119
# Known correctness issues
SPARK-34534 <https://issues.apache.org/jira/browse/SPARK-34534> New
protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or
correctness
SPARK-34545 <https://issues.apache.org/jira/browse/SPARK-34545> PySpark
Python UDF return inconsistent results when applying 2 UDFs with different
return type to 2 columns together
SPARK-34719 <https://issues.apache.org/jira/browse/SPARK-34719> fail if
the view query has duplicated column names
SPARK-34794 <https://issues.apache.org/jira/browse/SPARK-34794> Nested
higher-order functions broken in DSL
# Notable user-facing changes
SPARK-32924 <https://issues.apache.org/jira/browse/SPARK-32924> Web UI
sort on duration is wrong
SPARK-35405 <https://issues.apache.org/jira/browse/SPARK-35405>
Submitting Applications documentation has outdated information about K8s
client mode support
Thanks,
Yi
Re: Apache Spark 3.0.3 Release?
Posted by Liang-Chi Hsieh <vi...@gmail.com>.
+1. Thank you!
Liang-Chi
Dongjoon Hyun-2 wrote
> +1, Thank you! :)
>
> Bests,
> Dongjoon.
>
> On Tue, Jun 8, 2021 at 9:05 PM Kent Yao <
> yaooqinn@
> > wrote:
>
>> +1. Thanks, Yi ~
>>
>> Bests,
>> *Kent Yao *
>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>> *a spark enthusiast*
>> *kyuubi <https://github.com/yaooqinn/kyuubi>is a unified
>> multi-tenant JDBC
>> interface for large-scale data processing and analytics, built on top
>> of Apache Spark <http://spark.apache.org/>.*
>> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A
>> Spark
>> SQL extension which provides SQL Standard Authorization for **Apache
>> Spark <http://spark.apache.org/>.*
>> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A
>> library for
>> reading data from and transferring data to Postgres / Greenplum with
>> Spark
>> SQL and DataFrames, 10~100x faster.*
>> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library
>> t**hat
>> brings useful functions from various modern database management systems
>> to **Apache
>> Spark <http://spark.apache.org/>.*
>>
>>
>>
>> On 06/9/2021 11:54，Takeshi Yamamuro<
> linguin.m.s@
> >
>> <
> linguin.m.s@
> > wrote：
>>
>> +1. Thank you, Yi ~
>>
>> Bests,
>> Takeshi
>>
>> On Wed, Jun 9, 2021 at 12:18 PM Mridul Muralidharan <
> mridul@
> >
>> wrote:
>>
>>>
>>> +1
>>>
>>> Regards,
>>> Mridul
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Apache Spark 3.0.3 Release?
Posted by Dongjoon Hyun <do...@gmail.com>.
+1, Thank you! :)
Bests,
Dongjoon.
On Tue, Jun 8, 2021 at 9:05 PM Kent Yao <ya...@gmail.com> wrote:
> +1. Thanks, Yi ~
>
> Bests,
> *Kent Yao *
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> *a spark enthusiast*
> *kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC
> interface for large-scale data processing and analytics, built on top
> of Apache Spark <http://spark.apache.org/>.*
> *spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
> SQL extension which provides SQL Standard Authorization for **Apache
> Spark <http://spark.apache.org/>.*
> *spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for
> reading data from and transferring data to Postgres / Greenplum with Spark
> SQL and DataFrames, 10~100x faster.*
> *itatchi <https://github.com/yaooqinn/spark-func-extras>A** library t**hat
> brings useful functions from various modern database management systems to **Apache
> Spark <http://spark.apache.org/>.*
>
>
>
> On 06/9/2021 11:54，Takeshi Yamamuro<li...@gmail.com>
> <li...@gmail.com> wrote：
>
> +1. Thank you, Yi ~
>
> Bests,
> Takeshi
>
> On Wed, Jun 9, 2021 at 12:18 PM Mridul Muralidharan <mr...@gmail.com>
> wrote:
>
>>
>> +1
>>
>> Regards,
>> Mridul
>>
>> On Tue, Jun 8, 2021 at 10:11 PM Hyukjin Kwon <gu...@gmail.com> wrote:
>>
>>> Yeah, +1
>>>
>>> 2021년 6월 9일 (수) 오후 12:06, Yi Wu <yi...@databricks.com>님이 작성:
>>>
>>>> Hi, All.
>>>>
>>>> Since Apache Spark 3.0.2 tag creation (Feb 16),
>>>> new 119 patches (92 issues
>>>> <https://issues.apache.org/jira/projects/SPARK/versions/12349723>
>>>> resolved) arrived at branch-3.0.
>>>>
>>>> Shall we make a new release, Apache Spark 3.0.3, as the 3rd release at
>>>> the 3.0 line?
>>>> I'd like to volunteer as the release manager for Apache Spark 3.0.3.
>>>> I'm thinking about starting the first RC at the end of this week.
>>>>
>>>> $ git log --oneline v3.0.2..HEAD | wc -l
>>>> 119
>>>>
>>>> # Known correctness issues
>>>> SPARK-34534 <https://issues.apache.org/jira/browse/SPARK-34534>
>>>> New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss
>>>> or correctness
>>>> SPARK-34545 <https://issues.apache.org/jira/browse/SPARK-34545>
>>>> PySpark Python UDF return inconsistent results when applying 2 UDFs with
>>>> different return type to 2 columns together
>>>> SPARK-34719 <https://issues.apache.org/jira/browse/SPARK-34719>
>>>> fail if the view query has duplicated column names
>>>> SPARK-34794 <https://issues.apache.org/jira/browse/SPARK-34794>
>>>> Nested higher-order functions broken in DSL
>>>>
>>>> # Notable user-facing changes
>>>> SPARK-32924 <https://issues.apache.org/jira/browse/SPARK-32924>
>>>> Web UI sort on duration is wrong
>>>> SPARK-35405 <https://issues.apache.org/jira/browse/SPARK-35405>
>>>> Submitting Applications documentation has outdated information about K8s
>>>> client mode support
>>>>
>>>> Thanks,
>>>> Yi
>>>>
>>>
>
> --
> ---
> Takeshi Yamamuro
>
> --------------------------------------------------------------------- To
> unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Apache Spark 3.0.3 Release?
Posted by Kent Yao <ya...@gmail.com>.
+1. Thanks, Yi ~
Bests,
**Kent Yao **
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
** _a s_ _park enthusiast_**
_[kyuubi](https://github.com/yaooqinn/kyuubi) is a unified multi-tenant JDBC
interface for large-scale data processing and analytics, built on top of
[Apache Spark](http://spark.apache.org/)._
#
_[spark-authorizer](https://github.com/yaooqinn/spark-authorizer) A Spark SQL
extension which provides SQL Standard Authorization for _ _[Apache
Spark](http://spark.apache.org/)._
_ _[spark-postgres](https://github.com/yaooqinn/spark-postgres) A library
for reading data from and transferring data to Postgres / Greenplum with Spark
SQL and DataFrames, 10~100x faster.__
_ _[itatchi](https://github.com/yaooqinn/spark-func-extras) A__ _library t_
_hat brings useful functions from various modern database management systems
to _ _ _[Apache Spark](http://spark.apache.org/).__
#
On 06/9/2021 11:54，[Takeshi
Yamamuro<li...@gmail.com>](mailto:linguin.m.s@gmail.com) wrote：
> +1. Thank you, Yi ~
>
>
>
>
>
> Bests,
>
> Takeshi
>
>
>
>
> On Wed, Jun 9, 2021 at 12:18 PM Mridul Muralidharan
<[mridul@gmail.com](mailto:mridul@gmail.com)> wrote:
>
>
>>
>
>>
>> +1
>>
>>
>
>>
>> Regards,
>>
>> Mridul
>>
>>
>
>>
>> On Tue, Jun 8, 2021 at 10:11 PM Hyukjin Kwon
<[gurwls223@gmail.com](mailto:gurwls223@gmail.com)> wrote:
>
>>
>>> Yeah, +1
>>>
>>>
>
>>>
>>> 2021년 6월 9일 (수) 오후 12:06, Yi Wu
<[yi.wu@databricks.com](mailto:yi.wu@databricks.com)>님이 작성:
>
>>>
>>>> Hi, All.
>
> Since Apache Spark 3.0.2 tag creation (Feb 16),
> new 119 patches ([92
issues](https://issues.apache.org/jira/projects/SPARK/versions/12349723)
resolved) arrived at branch-3.0.
>
> Shall we make a new release, Apache Spark 3.0.3, as the 3rd release at the
3.0 line?
> I'd like to volunteer as the release manager for Apache Spark 3.0.3.
> I'm thinking about starting the first RC at the end of this week.
>
> $ git log --oneline v3.0.2..HEAD | wc -l
> 119
>
> # Known correctness issues
> [SPARK-34534](https://issues.apache.org/jira/browse/SPARK-34534) New
protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or
correctness
> [SPARK-34545 ](https://issues.apache.org/jira/browse/SPARK-34545) PySpark
Python UDF return inconsistent results when applying 2 UDFs with different
return type to 2 columns together
> [SPARK-34719](https://issues.apache.org/jira/browse/SPARK-34719) fail if
the view query has duplicated column names
> [SPARK-34794 ](https://issues.apache.org/jira/browse/SPARK-34794) Nested
higher-order functions broken in DSL
>
> # Notable user-facing changes
>
>>>>
>>>> [SPARK-32924](https://issues.apache.org/jira/browse/SPARK-32924) Web
UI sort on duration is wrong
>
>>>>
>>>> [SPARK-35405](https://issues.apache.org/jira/browse/SPARK-35405)
Submitting Applications documentation has outdated information about K8s
client mode support
>
>>>>
>>>>
>
>>>>
>>>> Thanks,
>>>>
>>>> Yi
>
>
>
>
>
>
>
> \--
>
>
> \---
> Takeshi Yamamuro
>
\--------------------------------------------------------------------- To
unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Apache Spark 3.0.3 Release?
Posted by Takeshi Yamamuro <li...@gmail.com>.
+1. Thank you, Yi ~
Bests,
Takeshi
On Wed, Jun 9, 2021 at 12:18 PM Mridul Muralidharan <mr...@gmail.com>
wrote:
>
> +1
>
> Regards,
> Mridul
>
> On Tue, Jun 8, 2021 at 10:11 PM Hyukjin Kwon <gu...@gmail.com> wrote:
>
>> Yeah, +1
>>
>> 2021년 6월 9일 (수) 오후 12:06, Yi Wu <yi...@databricks.com>님이 작성:
>>
>>> Hi, All.
>>>
>>> Since Apache Spark 3.0.2 tag creation (Feb 16),
>>> new 119 patches (92 issues
>>> <https://issues.apache.org/jira/projects/SPARK/versions/12349723>
>>> resolved) arrived at branch-3.0.
>>>
>>> Shall we make a new release, Apache Spark 3.0.3, as the 3rd release at
>>> the 3.0 line?
>>> I'd like to volunteer as the release manager for Apache Spark 3.0.3.
>>> I'm thinking about starting the first RC at the end of this week.
>>>
>>> $ git log --oneline v3.0.2..HEAD | wc -l
>>> 119
>>>
>>> # Known correctness issues
>>> SPARK-34534 <https://issues.apache.org/jira/browse/SPARK-34534> New
>>> protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or
>>> correctness
>>> SPARK-34545 <https://issues.apache.org/jira/browse/SPARK-34545>
>>> PySpark Python UDF return inconsistent results when applying 2 UDFs with
>>> different return type to 2 columns together
>>> SPARK-34719 <https://issues.apache.org/jira/browse/SPARK-34719>
>>> fail if the view query has duplicated column names
>>> SPARK-34794 <https://issues.apache.org/jira/browse/SPARK-34794>
>>> Nested higher-order functions broken in DSL
>>>
>>> # Notable user-facing changes
>>> SPARK-32924 <https://issues.apache.org/jira/browse/SPARK-32924> Web
>>> UI sort on duration is wrong
>>> SPARK-35405 <https://issues.apache.org/jira/browse/SPARK-35405>
>>> Submitting Applications documentation has outdated information about K8s
>>> client mode support
>>>
>>> Thanks,
>>> Yi
>>>
>>
--
---
Takeshi Yamamuro
Re: Apache Spark 3.0.3 Release?
Posted by Jungtaek Lim <ka...@gmail.com>.
Late +1 Thanks!
On Thu, Jun 10, 2021 at 12:06 PM Yi Wu <yi...@databricks.com> wrote:
> Thanks all, I'll start the RC soon.
>
> On Wed, Jun 9, 2021 at 7:07 PM Gengliang Wang <lt...@gmail.com> wrote:
>
>> +1, thanks Yi
>>
>> Gengliang Wang
>>
>>
>>
>>
>> On Jun 9, 2021, at 6:03 PM, 郑瑞峰 <ru...@foxmail.com> wrote:
>>
>> +1, thanks Yi
>>
>>
>>
Re: Apache Spark 3.0.3 Release?
Posted by Yi Wu <yi...@databricks.com>.
Thanks all, I'll start the RC soon.
On Wed, Jun 9, 2021 at 7:07 PM Gengliang Wang <lt...@gmail.com> wrote:
> +1, thanks Yi
>
> Gengliang Wang
>
>
>
>
> On Jun 9, 2021, at 6:03 PM, 郑瑞峰 <ru...@foxmail.com> wrote:
>
> +1, thanks Yi
>
>
>
Re: Apache Spark 3.0.3 Release?
Posted by Gengliang Wang <lt...@gmail.com>.
+1, thanks Yi
Gengliang Wang
> On Jun 9, 2021, at 6:03 PM, 郑瑞峰 <ru...@foxmail.com> wrote:
>
> +1, thanks Yi
回复： Apache Spark 3.0.3 Release?
Posted by 郑瑞峰 <ru...@foxmail.com>.
+1, thanks Yi
------------------ 原始邮件 ------------------
发件人: "Mridul Muralidharan" <mridul@gmail.com>;
发送时间: 2021年6月9日(星期三) 中午11:17
收件人: "Hyukjin Kwon"<gurwls223@gmail.com>;
抄送: "Yi Wu"<yi.wu@databricks.com>;"dev"<dev@spark.apache.org>;
主题: Re: Apache Spark 3.0.3 Release?
+1
Regards,
Mridul
On Tue, Jun 8, 2021 at 10:11 PM Hyukjin Kwon <gurwls223@gmail.com> wrote:
Yeah, +1
2021년 6월 9일 (수) 오후 12:06, Yi Wu <yi.wu@databricks.com>님이 작성:
Hi, All.
Since Apache Spark 3.0.2 tag creation (Feb 16),
new 119 patches (92 issues resolved) arrived at branch-3.0.
Shall we make a new release, Apache Spark 3.0.3, as the 3rd release at the 3.0 line?
I'd like to volunteer as the release manager for Apache Spark 3.0.3.
I'm thinking about starting the first RC at the end of this week.
$ git log --oneline v3.0.2..HEAD | wc -l
119
# Known correctness issues
SPARK-34534 New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness
SPARK-34545 PySpark Python UDF return inconsistent results when applying 2 UDFs with different return type to 2 columns together
SPARK-34719 fail if the view query has duplicated column names
SPARK-34794 Nested higher-order functions broken in DSL
# Notable user-facing changes
SPARK-32924 Web UI sort on duration is wrong
SPARK-35405 Submitting Applications documentation has outdated information about K8s client mode support
Thanks,
Yi
Re: Apache Spark 3.0.3 Release?
Posted by Mridul Muralidharan <mr...@gmail.com>.
+1
Regards,
Mridul
On Tue, Jun 8, 2021 at 10:11 PM Hyukjin Kwon <gu...@gmail.com> wrote:
> Yeah, +1
>
> 2021년 6월 9일 (수) 오후 12:06, Yi Wu <yi...@databricks.com>님이 작성:
>
>> Hi, All.
>>
>> Since Apache Spark 3.0.2 tag creation (Feb 16),
>> new 119 patches (92 issues
>> <https://issues.apache.org/jira/projects/SPARK/versions/12349723>
>> resolved) arrived at branch-3.0.
>>
>> Shall we make a new release, Apache Spark 3.0.3, as the 3rd release at
>> the 3.0 line?
>> I'd like to volunteer as the release manager for Apache Spark 3.0.3.
>> I'm thinking about starting the first RC at the end of this week.
>>
>> $ git log --oneline v3.0.2..HEAD | wc -l
>> 119
>>
>> # Known correctness issues
>> SPARK-34534 <https://issues.apache.org/jira/browse/SPARK-34534> New
>> protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or
>> correctness
>> SPARK-34545 <https://issues.apache.org/jira/browse/SPARK-34545>
>> PySpark Python UDF return inconsistent results when applying 2 UDFs with
>> different return type to 2 columns together
>> SPARK-34719 <https://issues.apache.org/jira/browse/SPARK-34719> fail
>> if the view query has duplicated column names
>> SPARK-34794 <https://issues.apache.org/jira/browse/SPARK-34794>
>> Nested higher-order functions broken in DSL
>>
>> # Notable user-facing changes
>> SPARK-32924 <https://issues.apache.org/jira/browse/SPARK-32924> Web
>> UI sort on duration is wrong
>> SPARK-35405 <https://issues.apache.org/jira/browse/SPARK-35405>
>> Submitting Applications documentation has outdated information about K8s
>> client mode support
>>
>> Thanks,
>> Yi
>>
>
Re: Apache Spark 3.0.3 Release?
Posted by Hyukjin Kwon <gu...@gmail.com>.
Yeah, +1
2021년 6월 9일 (수) 오후 12:06, Yi Wu <yi...@databricks.com>님이 작성:
> Hi, All.
>
> Since Apache Spark 3.0.2 tag creation (Feb 16),
> new 119 patches (92 issues
> <https://issues.apache.org/jira/projects/SPARK/versions/12349723>
> resolved) arrived at branch-3.0.
>
> Shall we make a new release, Apache Spark 3.0.3, as the 3rd release at
> the 3.0 line?
> I'd like to volunteer as the release manager for Apache Spark 3.0.3.
> I'm thinking about starting the first RC at the end of this week.
>
> $ git log --oneline v3.0.2..HEAD | wc -l
> 119
>
> # Known correctness issues
> SPARK-34534 <https://issues.apache.org/jira/browse/SPARK-34534> New
> protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or
> correctness
> SPARK-34545 <https://issues.apache.org/jira/browse/SPARK-34545>
> PySpark Python UDF return inconsistent results when applying 2 UDFs with
> different return type to 2 columns together
> SPARK-34719 <https://issues.apache.org/jira/browse/SPARK-34719> fail
> if the view query has duplicated column names
> SPARK-34794 <https://issues.apache.org/jira/browse/SPARK-34794> Nested
> higher-order functions broken in DSL
>
> # Notable user-facing changes
> SPARK-32924 <https://issues.apache.org/jira/browse/SPARK-32924> Web
> UI sort on duration is wrong
> SPARK-35405 <https://issues.apache.org/jira/browse/SPARK-35405>
> Submitting Applications documentation has outdated information about K8s
> client mode support
>
> Thanks,
> Yi
>