You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Maxim Gekk <ma...@databricks.com.INVALID> on 2022/06/03 08:24:30 UTC
The draft of the Spark 3.3.0 release notes
Hi All,
I am preparing the release notes of Spark 3.3.0. Here is a draft document:
https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
Please take a look and let me know if I missed any major changes or
something.
Maxim Gekk
Software Engineer
Databricks, Inc.
Re: The draft of the Spark 3.3.0 release notes
Posted by Adam Binford <ad...@gmail.com>.
It's more of a missing feature than a bug, but I guess it's all subjective.
Adam
On Fri, Jun 3, 2022 at 10:24 AM Maxim Gekk <ma...@databricks.com>
wrote:
> Hi Adam,
>
> The release notes focus on new features and user-facing improvements
> mostly. SPARK-37618 <https://issues.apache.org/jira/browse/SPARK-37618> seems
> like a bug fix, that's why I didn't put it in the doc.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
>
> On Fri, Jun 3, 2022 at 2:20 PM Adam Binford <ad...@gmail.com> wrote:
>
>> I don't think I see https://issues.apache.org/jira/browse/SPARK-37618
>> which got merged post branch cut.
>>
>> Adam
>>
>> On Fri, Jun 3, 2022 at 4:25 AM Maxim Gekk
>> <ma...@databricks.com.invalid> wrote:
>>
>>> Hi All,
>>>
>>> I am preparing the release notes of Spark 3.3.0. Here is a draft
>>> document:
>>>
>>> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>>>
>>> Please take a look and let me know if I missed any major changes or
>>> something.
>>>
>>> Maxim Gekk
>>>
>>> Software Engineer
>>>
>>> Databricks, Inc.
>>>
>>
>>
>> --
>> Adam Binford
>>
>
--
Adam Binford
Re: The draft of the Spark 3.3.0 release notes
Posted by Maxim Gekk <ma...@databricks.com.INVALID>.
Hi Adam,
The release notes focus on new features and user-facing improvements
mostly. SPARK-37618 <https://issues.apache.org/jira/browse/SPARK-37618> seems
like a bug fix, that's why I didn't put it in the doc.
Maxim Gekk
Software Engineer
Databricks, Inc.
On Fri, Jun 3, 2022 at 2:20 PM Adam Binford <ad...@gmail.com> wrote:
> I don't think I see https://issues.apache.org/jira/browse/SPARK-37618
> which got merged post branch cut.
>
> Adam
>
> On Fri, Jun 3, 2022 at 4:25 AM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
>
>> Hi All,
>>
>> I am preparing the release notes of Spark 3.3.0. Here is a draft document:
>>
>> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>>
>> Please take a look and let me know if I missed any major changes or
>> something.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>
>
> --
> Adam Binford
>
Re: The draft of the Spark 3.3.0 release notes
Posted by Adam Binford <ad...@gmail.com>.
I don't think I see https://issues.apache.org/jira/browse/SPARK-37618 which
got merged post branch cut.
Adam
On Fri, Jun 3, 2022 at 4:25 AM Maxim Gekk <ma...@databricks.com.invalid>
wrote:
> Hi All,
>
> I am preparing the release notes of Spark 3.3.0. Here is a draft document:
>
> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>
> Please take a look and let me know if I missed any major changes or
> something.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
--
Adam Binford
Re: The draft of the Spark 3.3.0 release notes
Posted by Bjørn Jørgensen <bj...@gmail.com>.
-
Support lambda `column` parameter of `DataFrame.rename`(SPARK-38763
<https://issues.apache.org/jira/browse/SPARK-38763>)
This did work before 18.jan.2022, see JIRA post for more information.
I think we can remove this one from the list.
fre. 3. jun. 2022 kl. 20:42 skrev Dongjoon Hyun <do...@gmail.com>:
> You are right.
>
> After SPARK-36837, we tried to ship Apache Spark 3.3.0 with Apache Kafka
> 3.1.1 via the following PR.
>
> https://github.com/apache/spark/pull/36135
> [WIP][SPARK-38850][BUILD] Upgrade Kafka to 3.1.1
>
> However, the final decision was to revert it from `branch-3.3` and move
> directly to Apache Kafka 3.2.0 at `master` branch. We need to remove it
> from the 3.3.0 release note.
>
>
> On Fri, Jun 3, 2022 at 9:54 AM Koert Kuipers <ko...@tresata.com> wrote:
>
>> i thought SPARK-36837 didnt make it in? i see it in notes
>>
>> On Fri, Jun 3, 2022 at 4:31 AM Maxim Gekk
>> <ma...@databricks.com.invalid> wrote:
>>
>>> Hi All,
>>>
>>> I am preparing the release notes of Spark 3.3.0. Here is a draft
>>> document:
>>>
>>> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>>>
>>> Please take a look and let me know if I missed any major changes or
>>> something.
>>>
>>> Maxim Gekk
>>>
>>> Software Engineer
>>>
>>> Databricks, Inc.
>>>
>>
>> CONFIDENTIALITY NOTICE: This electronic communication and any files
>> transmitted with it are confidential, privileged and intended solely for
>> the use of the individual or entity to whom they are addressed. If you are
>> not the intended recipient, you are hereby notified that any disclosure,
>> copying, distribution (electronic or otherwise) or forwarding of, or the
>> taking of any action in reliance on the contents of this transmission is
>> strictly prohibited. Please notify the sender immediately by e-mail if you
>> have received this email by mistake and delete this email from your system.
>>
>> Is it necessary to print this email? If you care about the environment
>> like we do, please refrain from printing emails. It helps to keep the
>> environment forested and litter-free.
>
>
--
Bjørn Jørgensen
Vestre Aspehaug 4, 6010 Ålesund
Norge
+47 480 94 297
Re: The draft of the Spark 3.3.0 release notes
Posted by Dongjoon Hyun <do...@gmail.com>.
You are right.
After SPARK-36837, we tried to ship Apache Spark 3.3.0 with Apache Kafka
3.1.1 via the following PR.
https://github.com/apache/spark/pull/36135
[WIP][SPARK-38850][BUILD] Upgrade Kafka to 3.1.1
However, the final decision was to revert it from `branch-3.3` and move
directly to Apache Kafka 3.2.0 at `master` branch. We need to remove it
from the 3.3.0 release note.
On Fri, Jun 3, 2022 at 9:54 AM Koert Kuipers <ko...@tresata.com> wrote:
> i thought SPARK-36837 didnt make it in? i see it in notes
>
> On Fri, Jun 3, 2022 at 4:31 AM Maxim Gekk
> <ma...@databricks.com.invalid> wrote:
>
>> Hi All,
>>
>> I am preparing the release notes of Spark 3.3.0. Here is a draft document:
>>
>> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>>
>> Please take a look and let me know if I missed any major changes or
>> something.
>>
>> Maxim Gekk
>>
>> Software Engineer
>>
>> Databricks, Inc.
>>
>
> CONFIDENTIALITY NOTICE: This electronic communication and any files
> transmitted with it are confidential, privileged and intended solely for
> the use of the individual or entity to whom they are addressed. If you are
> not the intended recipient, you are hereby notified that any disclosure,
> copying, distribution (electronic or otherwise) or forwarding of, or the
> taking of any action in reliance on the contents of this transmission is
> strictly prohibited. Please notify the sender immediately by e-mail if you
> have received this email by mistake and delete this email from your system.
>
> Is it necessary to print this email? If you care about the environment
> like we do, please refrain from printing emails. It helps to keep the
> environment forested and litter-free.
Re: The draft of the Spark 3.3.0 release notes
Posted by Koert Kuipers <ko...@tresata.com>.
i thought SPARK-36837 didnt make it in? i see it in notes
On Fri, Jun 3, 2022 at 4:31 AM Maxim Gekk <ma...@databricks.com.invalid>
wrote:
> Hi All,
>
> I am preparing the release notes of Spark 3.3.0. Here is a draft document:
>
> https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing
>
> Please take a look and let me know if I missed any major changes or
> something.
>
> Maxim Gekk
>
> Software Engineer
>
> Databricks, Inc.
>
--
CONFIDENTIALITY NOTICE: This electronic communication and any files
transmitted with it are confidential, privileged and intended solely for
the use of the individual or entity to whom they are addressed. If you are
not the intended recipient, you are hereby notified that any disclosure,
copying, distribution (electronic or otherwise) or forwarding of, or the
taking of any action in reliance on the contents of this transmission is
strictly prohibited. Please notify the sender immediately by e-mail if you
have received this email by mistake and delete this email from your system.
Is it necessary to print this email? If you care about the environment
like we do, please refrain from printing emails. It helps to keep the
environment forested and litter-free.