[jira] [Updated] (HDFS-2016) 1073: remove/archive unneeded/old storage files

["Todd Lipcon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon updated HDFS-2016:
------------------------------

    Attachment: hdfs-2016.txt

Updated patch.

I also discovered a potential bug, since the "inspectStorageDirs" had an unintended side effect of calling sd.read() in all of the configured storage directories. I don't know that it would have caused a problem, but it certainly wasn't intended. This patch splits up the storage dir inspection into two parts, so the unnecessary read()s don't happen in the archival code.

> 1073: remove/archive unneeded/old storage files
> -----------------------------------------------
>
>                 Key: HDFS-2016
>                 URL: https://issues.apache.org/jira/browse/HDFS-2016
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: Edit log branch (HDFS-1073)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: Edit log branch (HDFS-1073)
>
>         Attachments: hdfs-2016.txt, hdfs-2016.txt
>
>
> This JIRA is for the infrastructure that periodically scans the storage directories for files that are no longer necessary and "archives" them. The default archival is just to delete them, but we may implement something fancier in the future (eg move to SAN or HDFS)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira