You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Dongjoon Hyun <do...@gmail.com> on 2025/02/11 23:00:46 UTC
SPARK-51166: Prepare Apache Spark 4.1.0
Hi, All.
While the whole community is very busy for preparing Apache Spark 4.0.0 RC,
SPARK-51166 is created as a new umbrella JIRA issue to embrace
- all re-targeted JIRA issues
- up-coming feature links
For now, it includes the following and I hope this could be another helpful
resource
for you to easily identify what is not a part of Apache Spark 4.0 but will
arrive soon in 2025.
SPARK-51167 Build and Run Spark on Java 25
SPARK-51169 Support Python 3.14
SPARK-51168 Upgrade Hadoop to 3.4.2
SPARK-51064 Enable `spark.sql.sources.v2.bucketing.enabled` by default
SPARK-51165 Enable `spark.master.rest.enabled` by default
SPARK-51073 Remove `Unstable` from `SparkSessionExtensionsProvider`
SPARK-51155 Make SparkContext show total runtime after stopping
SPARK-51140 Sort the params before saving
SPARK-51148 Upgrade `zstd-jni` to 1.5.6-10
SPARK-51133 Upgrade Apache `commons-pool2` to 2.12.1
SPARK-51048 Support stop java spark context with exit code
SPARK-51023 log remote address on RPC exception
SPARK-50582 Add quote builtin function
According to the Apache Spark versioning policy, Apache Spark 4.1.0 is
scheduled to occur after 6 months from Apache Spark 4.0.0 release.
https://spark.apache.org/versioning-policy.html
Please include all missed items into this umbrella JIRA issue
in order not to forget them.
Best Regards,
Dongjoon.
Re: SPARK-51166: Prepare Apache Spark 4.1.0
Posted by Dongjoon Hyun <do...@apache.org>.
Here is an update.
The following are moved to 4.1.0 (SPARK-51166) from 4.0.0 (SPARK-44111) for now
because those issues are not ready with proper PRs.
SPARK-47110 Reenble AmmoniteTest tests in Maven builds
SPARK-48139 Re-enable `SparkSessionE2ESuite.interrupt tag`
SPARK-48163 Fix Flaky Test: `SparkConnectServiceSuite.SPARK-43923: ...`
SPARK-49586 Add addArtifact API to PySpark
SPARK-50205 Re-enable `SparkSessionJobTaggingAndCancellationSuite.Cancellation APIs...`
SPARK-50748 Fix a flaky test: `SparkSessionE2ESuite.interrupt all - background queries, ...`
SPARK-50771 Fix a flaky test: BlockInfoManagerSuite.SPARK-38675 - concurrent unlock ...
SPARK-50888 Fix Flaky Test: `SparkConnectServiceSuite.SPARK-44776: LocalTableScanExe`
SPARK-50889 Fix Flaky Test: `SparkSessionE2ESuite.interrupt operation`
SPARK-51019 Fix Flaky Test: `SPARK-47148: AQE should avoid to submit shuffle job on cancellation`
SPARK-51046 `SubExprEliminationBenchmark` fails at `CodeGenerator`
Dongjoon
On 2025/02/11 23:00:46 Dongjoon Hyun wrote:
> Hi, All.
>
> While the whole community is very busy for preparing Apache Spark 4.0.0 RC,
> SPARK-51166 is created as a new umbrella JIRA issue to embrace
> - all re-targeted JIRA issues
> - up-coming feature links
>
> For now, it includes the following and I hope this could be another helpful
> resource
> for you to easily identify what is not a part of Apache Spark 4.0 but will
> arrive soon in 2025.
>
> SPARK-51167 Build and Run Spark on Java 25
> SPARK-51169 Support Python 3.14
> SPARK-51168 Upgrade Hadoop to 3.4.2
> SPARK-51064 Enable `spark.sql.sources.v2.bucketing.enabled` by default
> SPARK-51165 Enable `spark.master.rest.enabled` by default
> SPARK-51073 Remove `Unstable` from `SparkSessionExtensionsProvider`
> SPARK-51155 Make SparkContext show total runtime after stopping
> SPARK-51140 Sort the params before saving
> SPARK-51148 Upgrade `zstd-jni` to 1.5.6-10
> SPARK-51133 Upgrade Apache `commons-pool2` to 2.12.1
> SPARK-51048 Support stop java spark context with exit code
> SPARK-51023 log remote address on RPC exception
> SPARK-50582 Add quote builtin function
>
> According to the Apache Spark versioning policy, Apache Spark 4.1.0 is
> scheduled to occur after 6 months from Apache Spark 4.0.0 release.
>
> https://spark.apache.org/versioning-policy.html
>
> Please include all missed items into this umbrella JIRA issue
> in order not to forget them.
>
> Best Regards,
> Dongjoon.
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org