You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Shixiong Zhu (JIRA)" <ji...@apache.org> on 2016/12/28 18:10:58 UTC
[jira] [Updated] (SPARK-18153) CLONE - Ability to remove old
metadata for structure streaming MetadataLog
[ https://issues.apache.org/jira/browse/SPARK-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shixiong Zhu updated SPARK-18153:
---------------------------------
Component/s: (was: DStreams)
(was: SQL)
Structured Streaming
> CLONE - Ability to remove old metadata for structure streaming MetadataLog
> --------------------------------------------------------------------------
>
> Key: SPARK-18153
> URL: https://issues.apache.org/jira/browse/SPARK-18153
> Project: Spark
> Issue Type: Sub-task
> Components: Structured Streaming
> Reporter: Sunil Kumar
> Assignee: Saisai Shao
> Fix For: 2.0.1, 2.1.0
>
>
> Current MetadataLog lacks the ability to remove old checkpoint file, we'd better add this functionality to the MetadataLog and honor it in the place where MetadataLog is used, that will reduce unnecessary small files in the long running scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org