You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by karthikjay <as...@gmail.com> on 2018/04/20 23:49:49 UTC
[Structured Streaming] [Kafka] How to repartition the data and
distribute the processing among worker nodes
Any help appreciated. please find the question in the link:
https://stackoverflow.com/questions/49951022/spark-structured-streaming-with-kafka-how-to-repartition-the-data-and-distribu
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: [Structured Streaming] [Kafka] How to repartition the data and
distribute the processing among worker nodes
Posted by "Bowden, Chris" <ch...@microfocus.com>.
The primary role of a sink is storing output tuples. Consider groupByKey and map/flatMapGroupsWithState instead.
-Chris
________________________________
From: karthikjay <as...@gmail.com>
Sent: Friday, April 20, 2018 4:49:49 PM
To: user@spark.apache.org
Subject: [Structured Streaming] [Kafka] How to repartition the data and distribute the processing among worker nodes
Any help appreciated. please find the question in the link:
https://stackoverflow.com/questions/49951022/spark-structured-streaming-with-kafka-how-to-repartition-the-data-and-distribu
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org