You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Burak Yavuz (JIRA)" <ji...@apache.org> on 2017/03/06 18:44:32 UTC
[jira] [Assigned] (SPARK-19304) Kinesis checkpoint recovery is 10x
slow
[ https://issues.apache.org/jira/browse/SPARK-19304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Burak Yavuz reassigned SPARK-19304:
-----------------------------------
Assignee: Gaurav Shah
> Kinesis checkpoint recovery is 10x slow
> ---------------------------------------
>
> Key: SPARK-19304
> URL: https://issues.apache.org/jira/browse/SPARK-19304
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.0.0
> Environment: using s3 for checkpoints using 1 executor, with 19g mem & 3 cores per executor
> Reporter: Gaurav Shah
> Assignee: Gaurav Shah
> Labels: kinesis
>
> Application runs fine initially, running batches of 1hour and the processing time is less than 30 minutes on average. For some reason lets say the application crashes, and we try to restart from checkpoint. The processing now takes forever and does not move forward. We tried to test out the same thing at batch interval of 1 minute, the processing runs fine and takes 1.2 minutes for batch to finish. When we recover from checkpoint it takes about 15 minutes for each batch. Post the recovery the batches again process at normal speed
> I suspect the KinesisBackedBlockRDD used for recovery is causing the slowdown.
> Stackoverflow post with more details: http://stackoverflow.com/questions/38390567/spark-streaming-checkpoint-recovery-is-very-very-slow
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org