You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:34:03 UTC
[jira] [Resolved] (SPARK-7167) Receivers are not distributed
efficiently when starting from checkpoint
[ https://issues.apache.org/jira/browse/SPARK-7167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-7167.
---------------------------------
Resolution: Incomplete
> Receivers are not distributed efficiently when starting from checkpoint
> -----------------------------------------------------------------------
>
> Key: SPARK-7167
> URL: https://issues.apache.org/jira/browse/SPARK-7167
> Project: Spark
> Issue Type: Bug
> Components: DStreams
> Affects Versions: 1.2.1, 1.2.2
> Reporter: Marius Soutier
> Priority: Minor
> Labels: bulk-closed
> Attachments: Screen Shot 2015-04-27 at 14.10.05.jpg
>
>
> Bug report: I'm seeing an issue where after starting a streaming application from a checkpoint, the network receivers are distributed such that not all nodes are used.
> For example, I have five nodes:
> node0 - 1 receiver
> node1 - 2 receivers
> node2 - 0 receivers
> node3 - 2 receivers
> node4 - 0 receivers
> This slows down the job, waiting batches pile up, and I have to kill and restart it, hoping that next time it will be distributed in a sensible fashion.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org