[jira] Commented: (HADOOP-5099) DataNode should warn about unknown files in storage

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707965#action_12707965 ] 

Steve Loughran commented on HADOOP-5099:
----------------------------------------

-there could be a lot of unknown files -may be best to list a warning on the count, and give the first two or three.
-you may want to have the option to completely halt the DN if there are unknowns, this could be handy to ensure the node hasnt been brought up pointing at the wrong part of the filesystem

> DataNode should warn about unknown files in storage
> ---------------------------------------------------
>
>                 Key: HADOOP-5099
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5099
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Raghu Angadi
>            Assignee: Jakob Homan
>
> DataNode currently just ignores the files it does not know about. There could be a lot of files left in DataNode's storage that never get noticed or deleted. These files could be left because of bugs or by a misconfiguration. E.g. while upgrading from 0.17, DN left a lot of metada files that were not named in correct format for 0.18 (HADOOP-4663).
> The proposal here is simply to make DN print a warning for each of the unknown files at the start up. This at least gives a way to list all the unknown files and  (equally importantly) forces a notion of "known" and "unknown" files in the storage.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.