You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2020/06/10 21:38:00 UTC
[jira] [Comment Edited] (SPARK-31961) Add a class in spark with all
Kafka configuration key available as string
[ https://issues.apache.org/jira/browse/SPARK-31961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132736#comment-17132736 ]
Jungtaek Lim edited comment on SPARK-31961 at 6/10/20, 9:37 PM:
----------------------------------------------------------------
My point is that given they're Kafka config why you're requiring them from Spark.
If Kafka provides such constant then you can just use that. If not, how could you tolerate the fact Kafka users also have to write down config key as string? In either way, Kafka should provide it under such rationalization, not Spark.
was (Author: kabhwan):
My point is that given they're Kafka config why you're requiring they from Spark.
If Kafka provides such constant then you can just use that. If not, how could you tolerate the fact Kafka users also have to write down config key as string? In either way, Kafka should provide it under such rationalization, not Spark.
> Add a class in spark with all Kafka configuration key available as string
> -------------------------------------------------------------------------
>
> Key: SPARK-31961
> URL: https://issues.apache.org/jira/browse/SPARK-31961
> Project: Spark
> Issue Type: New Feature
> Components: SQL, Structured Streaming
> Affects Versions: 2.4.6
> Reporter: Gunjan Kumar
> Priority: Minor
> Labels: kafka, sql, structured-streaming
>
> Add a class in spark with all Kafka configuration key available as string.
> see the highligted class which i want.
> eg:-
> Current code:-
> val df_cluster1 = spark
> .read
> .format("kafka")
> .option("kafka.bootstrap.servers","cluster1_host:cluster1_port)
> .option("subscribe", "topic1")
> Expected code:-
> val df_cluster1 = spark
> .read
> .format("kafka")
> .option(*KafkaConstantClass*.KAFKA_BOOTSTRAP_SERVERS,"cluster1_host:cluster1_port)
> .option(*KafkaConstantClass*.subscribe, "topic1")
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org