[jira] Updated: (HADOOP-4404) saveFSImage() should remove files from a storage directory that do not correspond to its type.

["Konstantin Shvachko (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-4404:
----------------------------------------

    Issue Type: Bug  (was: Improvement)

After successfully splitting directory {{name}}, which contained both {{fsimage}} and {{edits}}, into image only directory  {{name}} and edits only {{editsDir}} I see that {{name}} still contains the edits file. The same happens if I split out the image directory. The old one will still contain {{fsimage}}.

Files that do not correspond to the current directory type should be removed be saveFSImage().

I initially thought it is not a bug. But in fact it is.
Suppose I have an image-only directory {{imageDir}}, which is used to be image+edits directory and therefore contains an old file "edits". I also have an edits-only directory {{editsDir}}, which contains the up to date journal.
Then I by mistake configure  {{imageDir}} to be combined image+edits directory. Since {{fstime}} is the same in {{imageDir}} and {{editsDir}} any of the two edits files can be chosen by the name-node during startup. And if it chooses the old one from {{imageDir}} latest mutations to the name space will be lost.

> saveFSImage() should remove files from a storage directory that do not correspond to its type.
> ----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4404
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4404
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.19.0
>            Reporter: Konstantin Shvachko
>            Priority: Blocker
>             Fix For: 0.19.0
>
>
> NameNode's image and edits directories can be separated (HADOOP-3948).
> So the name-node can have 3 types of storage directories. 
> And each directory should contain only those files that correspond to its type:
> # combined image and edits directory: should contain {{fsimage}} and {{edits}}
> # image only directory should contain only {{fsimage}}
> # edits only directory should contain only {{edits}}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.