You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sönke Liebau (JIRA)" <ji...@apache.org> on 2018/02/21 10:11:00 UTC
[jira] [Comment Edited] (SPARK-18057) Update structured streaming
kafka from 10.0.1 to 10.2.0
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16371174#comment-16371174 ]
Sönke Liebau edited comment on SPARK-18057 at 2/21/18 10:10 AM:
----------------------------------------------------------------
I agree that we should not rename the existing kafka_10 package, as that would probably cause people loads of pain.
It is however tempting to have "clean" naming - would it be an option to add a package simply called kafka and update the kafka version in that? We could keep the kafka_10 package around for now but deprecate it at some point in time.
I am a bit on the fence about this, as in principle the current Kafka story is: "Any client with a version of 0.10.2.0 or later will support brokers of version 0.10.x or later, so simply upgrading the kafka version in the existing package should not break anything, as 0.9.x is not currently supported anyway.
However there is a caveat:
{code}
If the burden of backwards compatibility becomes too large, at some point we may need to break it.
{code}
So there is the possibility of the kafka_10 package becoming relevant again if later Kafka versions stop supporting 0.10.x brokers.
was (Author: sliebau):
I agree that we should not rename the existing kafka_10 package, as that would probably cause people loads of pain.
It is however tempting to have "clean" naming - would it be an option to add a package simply called kafka and update the kafka version in that? We could keep the kafka_10 package around for now but deprecate it at some point in time.
I am a bit on the fence about this, as in principle the current Kafka story is: "Any client with a version of 0.10.2.0 or later will support brokers of version 0.10.x or later, so simply upgrading the kafka version in the existing package should not break anything, as 0.9.x is not currently supported anyway.
However there is a caveat:
{code:java}
If the burden of backwards compatibility becomes too large, at some point we may need to break it.
{code}
So there is the possibility of the kafka_10 package becoming relevant again if later Kafka versions stop supporting 0.10.x brokers.
> Update structured streaming kafka from 10.0.1 to 10.2.0
> -------------------------------------------------------
>
> Key: SPARK-18057
> URL: https://issues.apache.org/jira/browse/SPARK-18057
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Reporter: Cody Koeninger
> Priority: Major
>
> There are a couple of relevant KIPs here, https://archive.apache.org/dist/kafka/0.10.1.0/RELEASE_NOTES.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org