You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Dongjoon Hyun <do...@apache.org> on 2023/06/20 20:41:54 UTC
Apache Spark 4.0.0 Dev Item Planning (SPARK-44111)
Hi, All.
As a continuation of our previous discussion, the official Apache Spark 4.0
Plan JIRA is created today in order to collect the community dev items.
Feel free to add your work items, ideas, suggestions, aspirations and
interests. We will moderate together.
https://issues.apache.org/jira/browse/SPARK-44111
Prepare Apache Spark 4.0.0
In addition, we are going to include all left-over items which Apache Spark
3.5 cannot include on July 16th (Feature Freeze,
https://spark.apache.org/versioning-policy.html)
======= PREVIOUS THREADS =======
2023-05-28 Apache Spark 3.5.0 Expectations (?)
https://lists.apache.org/thread/3x6dh17bmy20n3frtt3crgxjydnxh2o0
2023-05-30 Apache Spark 4.0 Timeframe?
https://lists.apache.org/thread/xhkgj60j361gdpywoxxz7qspp2w80ry6
2023-06-05 ASF policy violation and Scala version issues
https://lists.apache.org/thread/k7gr65wt0fwtldc7hp7bd0vkg1k93rrb
2023-06-12 [VOTE] Release Plan for Apache Spark 4.0.0 (June 2024)
https://lists.apache.org/thread/r0zn6rd8y25yn2dg59ktw3ttrwxzqrfb
2023-06-16 [VOTE][RESULT] Release Plan for Apache Spark 4.0.0 (June 2024)
https://lists.apache.org/thread/5vfof0nm82gt5b2k2o0ws944hofz232g
Re: Apache Spark 4.0.0 Dev Item Planning (SPARK-44111)
Posted by Dongjoon Hyun <do...@gmail.com>.
Thank you, Yuming.
Please update SPARK-44111 by adding links to those JIRA for visibility.
Otherwise, we may miss them during the up-coming discussion.
Dongjoon.
On Tue, Jun 20, 2023 at 6:40 PM Yuming Wang <yu...@apache.org> wrote:
> Thank you Dongjoon. I'd like to add these items.
>
> *Support for more SQL syntax*
> SPARK-31561 <https://issues.apache.org/jira/browse/SPARK-31561> Add
> QUALIFY clause
> SPARK-24497 <https://issues.apache.org/jira/browse/SPARK-24497> Support
> recursive SQL
> SPARK-32064 <https://issues.apache.org/jira/browse/SPARK-32064> Support
> temporary table
>
> *Improve Query performance in specific scenarios*
> SPARK-8682 <https://issues.apache.org/jira/browse/SPARK-8682> Range Join
> for Spark SQL. We have a blog in Chinese
> <https://mp.weixin.qq.com/s/BK1dHMe91tIl85JfxXNP7g> about this
> optimization.
> SPARK-38506 <https://issues.apache.org/jira/browse/SPARK-38506> Push
> partial aggregation through join
>
>
> On Wed, Jun 21, 2023 at 4:42 AM Dongjoon Hyun <do...@apache.org> wrote:
>
>> Hi, All.
>>
>> As a continuation of our previous discussion, the official Apache Spark
>> 4.0 Plan JIRA is created today in order to collect the community dev items.
>> Feel free to add your work items, ideas, suggestions, aspirations and
>> interests. We will moderate together.
>>
>> https://issues.apache.org/jira/browse/SPARK-44111
>> Prepare Apache Spark 4.0.0
>>
>> In addition, we are going to include all left-over items which Apache
>> Spark 3.5 cannot include on July 16th (Feature Freeze,
>> https://spark.apache.org/versioning-policy.html)
>>
>>
>>
>> ======= PREVIOUS THREADS =======
>>
>> 2023-05-28 Apache Spark 3.5.0 Expectations (?)
>> https://lists.apache.org/thread/3x6dh17bmy20n3frtt3crgxjydnxh2o0
>>
>> 2023-05-30 Apache Spark 4.0 Timeframe?
>> https://lists.apache.org/thread/xhkgj60j361gdpywoxxz7qspp2w80ry6
>>
>> 2023-06-05 ASF policy violation and Scala version issues
>> https://lists.apache.org/thread/k7gr65wt0fwtldc7hp7bd0vkg1k93rrb
>>
>> 2023-06-12 [VOTE] Release Plan for Apache Spark 4.0.0 (June 2024)
>> https://lists.apache.org/thread/r0zn6rd8y25yn2dg59ktw3ttrwxzqrfb
>>
>> 2023-06-16 [VOTE][RESULT] Release Plan for Apache Spark 4.0.0 (June 2024)
>> https://lists.apache.org/thread/5vfof0nm82gt5b2k2o0ws944hofz232g
>>
>
Re: Apache Spark 4.0.0 Dev Item Planning (SPARK-44111)
Posted by Yuming Wang <yu...@apache.org>.
Thank you Dongjoon. I'd like to add these items.
*Support for more SQL syntax*
SPARK-31561 <https://issues.apache.org/jira/browse/SPARK-31561> Add QUALIFY
clause
SPARK-24497 <https://issues.apache.org/jira/browse/SPARK-24497> Support
recursive SQL
SPARK-32064 <https://issues.apache.org/jira/browse/SPARK-32064> Support
temporary table
*Improve Query performance in specific scenarios*
SPARK-8682 <https://issues.apache.org/jira/browse/SPARK-8682> Range Join
for Spark SQL. We have a blog in Chinese
<https://mp.weixin.qq.com/s/BK1dHMe91tIl85JfxXNP7g> about this optimization.
SPARK-38506 <https://issues.apache.org/jira/browse/SPARK-38506> Push
partial aggregation through join
On Wed, Jun 21, 2023 at 4:42 AM Dongjoon Hyun <do...@apache.org> wrote:
> Hi, All.
>
> As a continuation of our previous discussion, the official Apache Spark
> 4.0 Plan JIRA is created today in order to collect the community dev items.
> Feel free to add your work items, ideas, suggestions, aspirations and
> interests. We will moderate together.
>
> https://issues.apache.org/jira/browse/SPARK-44111
> Prepare Apache Spark 4.0.0
>
> In addition, we are going to include all left-over items which Apache
> Spark 3.5 cannot include on July 16th (Feature Freeze,
> https://spark.apache.org/versioning-policy.html)
>
>
>
> ======= PREVIOUS THREADS =======
>
> 2023-05-28 Apache Spark 3.5.0 Expectations (?)
> https://lists.apache.org/thread/3x6dh17bmy20n3frtt3crgxjydnxh2o0
>
> 2023-05-30 Apache Spark 4.0 Timeframe?
> https://lists.apache.org/thread/xhkgj60j361gdpywoxxz7qspp2w80ry6
>
> 2023-06-05 ASF policy violation and Scala version issues
> https://lists.apache.org/thread/k7gr65wt0fwtldc7hp7bd0vkg1k93rrb
>
> 2023-06-12 [VOTE] Release Plan for Apache Spark 4.0.0 (June 2024)
> https://lists.apache.org/thread/r0zn6rd8y25yn2dg59ktw3ttrwxzqrfb
>
> 2023-06-16 [VOTE][RESULT] Release Plan for Apache Spark 4.0.0 (June 2024)
> https://lists.apache.org/thread/5vfof0nm82gt5b2k2o0ws944hofz232g
>