You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2020/04/19 17:53:00 UTC
[spark] branch master updated: [SPARK-31464][BUILD][SS] Upgrade
Kafka to 2.5.0
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c6e39df [SPARK-31464][BUILD][SS] Upgrade Kafka to 2.5.0
c6e39df is described below
commit c6e39dffd663ef978972307052de5a38605ee9c8
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sun Apr 19 10:51:09 2020 -0700
[SPARK-31464][BUILD][SS] Upgrade Kafka to 2.5.0
### What changes were proposed in this pull request?
This PR aims to upgrade Kafka library to 2.5.0 for Apache Spark 3.1.0.
### Why are the changes needed?
Apache Kafka 2.5.0 client has improvements and bug fixes like [KAFKA-9241](https://issues.apache.org/jira/browse/KAFKA-9241)
- https://downloads.apache.org/kafka/2.5.0/RELEASE_NOTES.html
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Pass the Jenkins with the existing tests.
- [x] SBT https://github.com/apache/spark/pull/28235#issuecomment-615936382
- [x] Maven https://github.com/apache/spark/pull/28235#issuecomment-616138840 (All Scala/Java/Python/R UT tests passed. It's timeout during R installation testing which is already covered by SBT.)
Closes #28235 from dongjoon-hyun/SPARK-KAFKA-2.5.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
external/kafka-0-10-sql/pom.xml | 4 ++--
external/kafka-0-10/pom.xml | 4 ++--
pom.xml | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/external/kafka-0-10-sql/pom.xml b/external/kafka-0-10-sql/pom.xml
index 26707e1..95a99ac 100644
--- a/external/kafka-0-10-sql/pom.xml
+++ b/external/kafka-0-10-sql/pom.xml
@@ -108,11 +108,11 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minikdc</artifactId>
</dependency>
- <!-- Kafka embedded server uses Zookeeper 3.4.7 API -->
+ <!-- Kafka embedded server uses Zookeeper 3.5.7 API -->
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
- <version>3.4.7</version>
+ <version>3.5.7</version>
<scope>test</scope>
</dependency>
<dependency>
diff --git a/external/kafka-0-10/pom.xml b/external/kafka-0-10/pom.xml
index 666d941..024fdb2 100644
--- a/external/kafka-0-10/pom.xml
+++ b/external/kafka-0-10/pom.xml
@@ -84,11 +84,11 @@
</exclusion>
</exclusions>
</dependency>
- <!-- Kafka embedded server uses Zookeeper 3.4.7 API -->
+ <!-- Kafka embedded server uses Zookeeper 3.5.7 API -->
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
- <version>3.4.7</version>
+ <version>3.5.7</version>
<scope>test</scope>
</dependency>
<dependency>
diff --git a/pom.xml b/pom.xml
index cd85db6..470edfd 100644
--- a/pom.xml
+++ b/pom.xml
@@ -132,7 +132,7 @@
<!-- Version used for internal directory structure -->
<hive.version.short>2.3</hive.version.short>
<!-- note that this should be compatible with Kafka brokers version 0.10 and up -->
- <kafka.version>2.4.1</kafka.version>
+ <kafka.version>2.5.0</kafka.version>
<derby.version>10.12.1.1</derby.version>
<parquet.version>1.10.1</parquet.version>
<orc.version>1.5.9</orc.version>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org