You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jeff Nadler (JIRA)" <ji...@apache.org> on 2016/09/12 19:21:20 UTC
[jira] [Created] (SPARK-17510) Set Streaming MaxRate Independently
For Multiple Streams
Jeff Nadler created SPARK-17510:
-----------------------------------
Summary: Set Streaming MaxRate Independently For Multiple Streams
Key: SPARK-17510
URL: https://issues.apache.org/jira/browse/SPARK-17510
Project: Spark
Issue Type: Improvement
Components: Streaming
Affects Versions: 2.0.0
Reporter: Jeff Nadler
We use multiple DStreams coming from different Kafka topics in a Streaming application.
Some settings like maxrate and backpressure enabled/disabled would be better passed as config to KafkaUtils.createStream and KafkaUtils.createDirectStream, instead of setting them in SparkConf.
Being able to set a different maxrate for different streams is an important requirement for us; we currently work-around the problem by using one receiver-based stream and one direct stream.
We would like to be able to turn on backpressure for only one of the streams as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org