You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:20:13 UTC

[jira] [Updated] (SPARK-16160) Last remembered metadata window per dstream is not cleared upon context graceful stop

     [ https://issues.apache.org/jira/browse/SPARK-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon updated SPARK-16160:
---------------------------------
    Labels: bulk-closed  (was: )

> Last remembered metadata window per dstream is not cleared upon context graceful stop
> -------------------------------------------------------------------------------------
>
>                 Key: SPARK-16160
>                 URL: https://issues.apache.org/jira/browse/SPARK-16160
>             Project: Spark
>          Issue Type: Bug
>          Components: DStreams
>            Reporter: Ahmed Mahran
>            Priority: Minor
>              Labels: bulk-closed
>
> When stopping a streaming context gracefully, the last remembered window for each dstream is not cleared. If the streaming context is stopped without stopping the spark context, the persisted garbage rdds would still be there especially when there is a relatively large remember duration in case of windowing or checkpointing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org