You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Brian Bockelman (JIRA)" <ji...@apache.org> on 2009/03/22 17:47:50 UTC
[jira] Created: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Namenode permits directory destruction on overwrite
---------------------------------------------------
Key: HADOOP-5551
URL: https://issues.apache.org/jira/browse/HADOOP-5551
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.19.1
Reporter: Brian Bockelman
Priority: Critical
The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688383#action_12688383 ]
Konstantin Shvachko commented on HADOOP-5551:
---------------------------------------------
Looks like your solution will work only if permission is enabled. If not it will still replace the directory by the file.
Could you please include a test for that. You can just add a test case to TestFileCreation in order to avoid cluster startup / shutdown just for this one operation.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Attachments: HADOOP-5551.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-5551:
-------------------------------------------
Component/s: dfs
Assignee: Brian Bockelman
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695428#action_12695428 ]
Hudson commented on HADOOP-5551:
--------------------------------
Integrated in Hadoop-trunk #796 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/796/])
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch, HADOOP-5551-v4.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Status: Patch Available (was: Open)
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch, HADOOP-5551-v4.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Attachment: HADOOP-5551-v2.patch
This is an improved patch - fixing the obvious thing Konstantin pointed out and contains a test case.
Will submit it as a patch.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12693563#action_12693563 ]
Brian Bockelman commented on HADOOP-5551:
-----------------------------------------
Test failures appear to be unrelated?
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch, HADOOP-5551-v4.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12693497#action_12693497 ]
Hadoop QA commented on HADOOP-5551:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12403752/HADOOP-5551-v3.patch
against trunk revision 759398.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/75/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/75/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/75/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/75/console
This message is automatically generated.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Status: Open (was: Patch Available)
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689684#action_12689684 ]
Konstantin Shvachko commented on HADOOP-5551:
---------------------------------------------
+1 for {{FSNamesystem}} changes.
{{TestCreateFile}} could be simplified:
- you can assert right after creating the file, because the creation should not be successful. Listing the directory does not really add any value.
- if by any chance the file is created you should close the output stream.
Something like this:
{code}
try {
FSDataOutputStream out = fs.create(dir1, true);
out.close();
assertTrue("Did not prevent directory from being overwritten.", false);
} catch (IOException ie) {
if (!ie.getMessage().contains("already exists as a directory.")) {
throw ie;
}
}
{code}
- also you got some indentation problems in the test code (should be 2 spaces)
- and please make sure that lines do not exceed 80 symbols.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-5551:
----------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thank you Brian.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch, HADOOP-5551-v4.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-5551:
----------------------------------
Fix Version/s: (was: 0.19.1)
0.19.2
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Status: In Progress (was: Patch Available)
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Status: Patch Available (was: In Progress)
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Attachment: HADOOP-5551-v4.patch
Shoot - accidentally negated the test condition after testing the patch locally, causing the hudson failure.
Here's the fixed test case; I did check both testFileCreation and testFileCreationSimulated locally prior to submitting. Passes locally.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch, HADOOP-5551-v4.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Fix Version/s: 0.19.1
0.20.0
Status: Patch Available (was: Open)
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Fix For: 0.20.0, 0.19.1
>
> Attachments: HADOOP-5551-v2.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Attachment: HADOOP-5551.patch
Suggested patch attached.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Attachments: HADOOP-5551.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Attachment: (was: HADOOP-5551.patch)
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Attachments: HADOOP-5551-v2.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Brian Bockelman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Bockelman updated HADOOP-5551:
------------------------------------
Attachment: HADOOP-5551-v3.patch
This patch takes into consideration Konstantin's comments. Made the test case play nicer with Hadoop; same patch for FSNamesystem.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12693514#action_12693514 ]
Hadoop QA commented on HADOOP-5551:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12404070/HADOOP-5551-v4.patch
against trunk revision 759398.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/76/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/76/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/76/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/76/console
This message is automatically generated.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch, HADOOP-5551-v4.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689780#action_12689780 ]
Konstantin Shvachko commented on HADOOP-5551:
---------------------------------------------
+1
Could you please resubmit the patch so that Hudson could pick it up.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Assignee: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch, HADOOP-5551-v3.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5551) Namenode permits directory
destruction on overwrite
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689066#action_12689066 ]
Hadoop QA commented on HADOOP-5551:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12403562/HADOOP-5551-v2.patch
against trunk revision 758156.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/135/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/135/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/135/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/135/console
This message is automatically generated.
> Namenode permits directory destruction on overwrite
> ---------------------------------------------------
>
> Key: HADOOP-5551
> URL: https://issues.apache.org/jira/browse/HADOOP-5551
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.19.1
> Reporter: Brian Bockelman
> Priority: Critical
> Fix For: 0.19.2, 0.20.0
>
> Attachments: HADOOP-5551-v2.patch
>
>
> The FSNamesystem's startFileInternal allows overwriting of directories. That is, if you have a directory named /foo/bar and you try to write a file named /foo/bar, the file is written and the directory disappears.
> This is most apparent for folks using libhdfs directly, as overwriting is always turned on. Therefore, if libhdfs applications do not check the existence of a directory first, then they will permit new files to destroy directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.