You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by ka...@apache.org on 2025/03/31 22:05:59 UTC

(datafusion-comet) branch main updated: docs: Update supported Spark versions (#1580)

This is an automated email from the ASF dual-hosted git repository.

kazuyukitanimura pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git


The following commit(s) were added to refs/heads/main by this push:
     new 68199c2dc docs: Update supported Spark versions (#1580)
68199c2dc is described below

commit 68199c2dc09600fdb0f92b702c8a81ed04355105
Author: Andy Grove <ag...@apache.org>
AuthorDate: Mon Mar 31 16:05:54 2025 -0600

    docs: Update supported Spark versions (#1580)
    
    ## Which issue does this PR close?
    
    Part of https://github.com/apache/datafusion-comet/issues/1579
    
    Follows on from https://github.com/apache/datafusion-comet/pull/1573
---
 docs/source/user-guide/installation.md | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/docs/source/user-guide/installation.md b/docs/source/user-guide/installation.md
index 88b651131..6e57407d9 100644
--- a/docs/source/user-guide/installation.md
+++ b/docs/source/user-guide/installation.md
@@ -30,22 +30,32 @@ Make sure the following requirements are met and software installed on your mach
 
 ### Supported Spark Versions
 
-Comet currently supports the following versions of Apache Spark:
+Comet currently supports the following versions of Apache Spark. Note that this documentation is for the main
+branch. Please refer to the version of this documentation in the appropriate GitHub release branch for released
+Comet versions.
 
-- 3.4.x (Java 8/11/17, Scala 2.12/2.13)
-- 3.5.x (Java 8/11/17, Scala 2.12/2.13)
+We recommend only using Comet with Spark versions where we currently have both Comet and Spark tests enabled in CI.
+Other versions may work well enough for development and evaluation purposes.
+
+| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark Tests in CI |
+| ------------- | ------------ | ------------- | ----------------- | ----------------- |
+| 3.4.3         | 8/11/17      | 2.12/2.13     | Yes               | Yes               |
+| 3.5.4         | 8/11/17      | 2.12/2.13     | No                | Yes               |
+| 3.5.5         | 8/11/17      | 2.12/2.13     | Yes               | Yes               |
 
 Experimental support is provided for the following versions of Apache Spark and is intended for development/testing
 use only and should not be used in production yet.
 
-- 4.0.0-preview1 (Java 17/21, Scala 2.13)
+| Spark Version  | Java Version | Scala Version | Comet Tests in CI | Spark Tests in CI |
+| -------------- | ------------ | ------------- | ----------------- | ----------------- |
+| 4.0.0-preview1 | 17           | 2.13          | Yes               | Yes               |
 
 Note that Comet may not fully work with proprietary forks of Apache Spark such as the Spark versions offered by
 Cloud Service Providers.
 
 ## Using a Published JAR File
 
-Comet jar files are available in [Maven Central](https://central.sonatype.com/namespace/org.apache.datafusion) for amd64 and arm64 architectures for Linux. For Apple OSX, it 
+Comet jar files are available in [Maven Central](https://central.sonatype.com/namespace/org.apache.datafusion) for amd64 and arm64 architectures for Linux. For Apple OSX, it
 is currently necessary to build from source.
 
 Here are the direct links for downloading the Comet 0.7.0 jar file.


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org