You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Tathagata Das (JIRA)" <ji...@apache.org> on 2015/08/05 04:27:04 UTC

[jira] [Updated] (SPARK-9619) BlockGenerator should not receive data even after it has been stopped

     [ https://issues.apache.org/jira/browse/SPARK-9619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tathagata Das updated SPARK-9619:
---------------------------------
    Description:     (was: The internal default block generator that is used by receivers gets reused across receiver restarts. This can lead to duplicate data. This is sort-of-okay as receivers really provide at-least once guarantee at best. Furthermore Reliable receivers like the ReliableKafkaReceiver, did not reuse BlockGenerator objects hence did not have the problem.

The solution is to ensure that the internal buffer of the BlockGenerator is cleared every time it is started.)

> BlockGenerator should not receive data even after it has been stopped
> ---------------------------------------------------------------------
>
>                 Key: SPARK-9619
>                 URL: https://issues.apache.org/jira/browse/SPARK-9619
>             Project: Spark
>          Issue Type: Bug
>          Components: Streaming
>            Reporter: Tathagata Das
>            Assignee: Tathagata Das
>            Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org