You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by ka...@apache.org on 2025/03/31 22:05:59 UTC
(datafusion-comet) branch main updated: docs: Update supported Spark versions (#1580)
This is an automated email from the ASF dual-hosted git repository.
kazuyukitanimura pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 68199c2dc docs: Update supported Spark versions (#1580)
68199c2dc is described below
commit 68199c2dc09600fdb0f92b702c8a81ed04355105
Author: Andy Grove <ag...@apache.org>
AuthorDate: Mon Mar 31 16:05:54 2025 -0600
docs: Update supported Spark versions (#1580)
## Which issue does this PR close?
Part of https://github.com/apache/datafusion-comet/issues/1579
Follows on from https://github.com/apache/datafusion-comet/pull/1573
---
docs/source/user-guide/installation.md | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/docs/source/user-guide/installation.md b/docs/source/user-guide/installation.md
index 88b651131..6e57407d9 100644
--- a/docs/source/user-guide/installation.md
+++ b/docs/source/user-guide/installation.md
@@ -30,22 +30,32 @@ Make sure the following requirements are met and software installed on your mach
### Supported Spark Versions
-Comet currently supports the following versions of Apache Spark:
+Comet currently supports the following versions of Apache Spark. Note that this documentation is for the main
+branch. Please refer to the version of this documentation in the appropriate GitHub release branch for released
+Comet versions.
-- 3.4.x (Java 8/11/17, Scala 2.12/2.13)
-- 3.5.x (Java 8/11/17, Scala 2.12/2.13)
+We recommend only using Comet with Spark versions where we currently have both Comet and Spark tests enabled in CI.
+Other versions may work well enough for development and evaluation purposes.
+
+| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark Tests in CI |
+| ------------- | ------------ | ------------- | ----------------- | ----------------- |
+| 3.4.3 | 8/11/17 | 2.12/2.13 | Yes | Yes |
+| 3.5.4 | 8/11/17 | 2.12/2.13 | No | Yes |
+| 3.5.5 | 8/11/17 | 2.12/2.13 | Yes | Yes |
Experimental support is provided for the following versions of Apache Spark and is intended for development/testing
use only and should not be used in production yet.
-- 4.0.0-preview1 (Java 17/21, Scala 2.13)
+| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark Tests in CI |
+| -------------- | ------------ | ------------- | ----------------- | ----------------- |
+| 4.0.0-preview1 | 17 | 2.13 | Yes | Yes |
Note that Comet may not fully work with proprietary forks of Apache Spark such as the Spark versions offered by
Cloud Service Providers.
## Using a Published JAR File
-Comet jar files are available in [Maven Central](https://central.sonatype.com/namespace/org.apache.datafusion) for amd64 and arm64 architectures for Linux. For Apple OSX, it
+Comet jar files are available in [Maven Central](https://central.sonatype.com/namespace/org.apache.datafusion) for amd64 and arm64 architectures for Linux. For Apple OSX, it
is currently necessary to build from source.
Here are the direct links for downloading the Comet 0.7.0 jar file.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org