You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2019/12/15 23:25:00 UTC
[jira] [Comment Edited] (SPARK-29294) Update Kafka to a verison
that supports Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-29294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16996892#comment-16996892 ]
Jungtaek Lim edited comment on SPARK-29294 at 12/15/19 11:24 PM:
-----------------------------------------------------------------
1. SPARK-28367 (replace poll method from long param to Duration, which Gabor submitted a patch) requires Kafka 2.4.0 as well. If Kafka 2.4.0 still keeps poll(long) then ZkUtils is the only blocker to upgrade to Kafka 2.4.0, otherwise we should do ZkUtils replacement and SPARK-28367 altogether.
2. Gabor already filed SPARK-28144 to replace ZkUtil with something, but at least the description and comment in the issue Gabor left, it seems to require KAFKA-8468 which is still not addressed. So there might not be many options but inlining the ZkUtils code. (With proper comment that it's ideal to replace it with something when Kafka has one.)
Btw, just FYI, Gabor may be available in next month.
was (Author: kabhwan):
1. SPARK-28367 (replace poll method from long param to Duration, which Gabor submitted a patch) requires Kafka 2.4.0 as well. If Kafka 2.4.0 still keeps poll(long) then ZkUtils is the only blocker to upgrade to Kafka 2.4.0, otherwise we should do ZkUtils replacement and SPARK-28367 altogether.
2. Gabor already filed SPARK-28144 to replace ZkUtil with something, but at least the description and comment in the issue Gabor left, it seems to require KAFKA-8468 which is still not addressed.
Btw, just FYI, Gabor may be available in next month.
> Update Kafka to a verison that supports Scala 2.13
> --------------------------------------------------
>
> Key: SPARK-29294
> URL: https://issues.apache.org/jira/browse/SPARK-29294
> Project: Spark
> Issue Type: Sub-task
> Components: Structured Streaming
> Affects Versions: 3.0.0
> Reporter: Sean R. Owen
> Priority: Major
>
> Kafka 2.3.0 doesn't seem to support 2.13 yet. We'll need to update when one is published, to support 2.13.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org