You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2015/08/17 14:46:46 UTC

[jira] [Commented] (SPARK-10052) KafKaDirectDstream should filter empty partition task or rdd

    [ https://issues.apache.org/jira/browse/SPARK-10052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699482#comment-14699482 ] 

Sean Owen commented on SPARK-10052:
-----------------------------------

You can't be sure the stage doesn't have side-effects, even if the input size is 0. I believe we've had similar discussions before. You still need to launch a job, but, the job can be a no-op.

> KafKaDirectDstream should filter empty partition task or rdd
> ------------------------------------------------------------
>
>                 Key: SPARK-10052
>                 URL: https://issues.apache.org/jira/browse/SPARK-10052
>             Project: Spark
>          Issue Type: Improvement
>          Components: Streaming
>    Affects Versions: 1.4.1
>            Reporter: SuYan
>
> We run spark 1.4.0 spark direct streaming, found it submit stages and tasks for input event = 0 events.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org