You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tathagata Das (JIRA)" <ji...@apache.org> on 2015/05/19 08:40:02 UTC
[jira] [Commented] (SPARK-7661) Support for dynamic allocation of
resources in Kinesis Spark Streaming
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549911#comment-14549911 ]
Tathagata Das commented on SPARK-7661:
--------------------------------------
Currently there is not way to scale up the receivers. Consider shutting
down the existing streaming context gracefully, and then restart the
context with more receivers.
On Mon, May 18, 2015 at 12:35 AM, Murtaza Kanchwala (JIRA) <ji...@apache.org>
> Support for dynamic allocation of resources in Kinesis Spark Streaming
> ----------------------------------------------------------------------
>
> Key: SPARK-7661
> URL: https://issues.apache.org/jira/browse/SPARK-7661
> Project: Spark
> Issue Type: New Feature
> Components: Streaming
> Affects Versions: 1.3.1
> Environment: AWS-EMR
> Reporter: Murtaza Kanchwala
>
> Currently the no. of cores is (N + 1), where N is no. of shards in a Kinesis Stream.
> My Requirement is that if I use this Resharding util for Amazon Kinesis :
> Amazon Kinesis Resharding : https://github.com/awslabs/amazon-kinesis-scaling-utils
> Then there should be some way to allocate executors on the basis of no. of shards directly (for Spark Streaming only).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org