You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/05/20 01:56:47 UTC
[jira] [Created] (HDFS-1967) TestHDFSTrash failing on trunk and 22
TestHDFSTrash failing on trunk and 22
-------------------------------------
Key: HDFS-1967
URL: https://issues.apache.org/jira/browse/HDFS-1967
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.22.0
Reporter: Todd Lipcon
Fix For: 0.22.0
Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109208#comment-13109208 ]
Mayank Bansal commented on HDFS-1967:
-------------------------------------
Adding the test-patch output
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec] +1 system test framework. The patch passed system test framework compile.
[exec]
[exec]
[exec]
[exec]
[exec] ======================================================================
[exec] ======================================================================
[exec] Finished build.
[exec] ======================================================================
[exec] ======================================================================
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: patch-HDFS-1967.patch
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-1967) TestHDFSTrash failing on trunk and 22
Posted by "Konstantin Shvachko (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko resolved HDFS-1967.
---------------------------------------
Resolution: Duplicate
Fixed by HADOOP-7663
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: patch-HDFS-1967.patch
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109216#comment-13109216 ]
Konstantin Shvachko commented on HDFS-1967:
-------------------------------------------
Patch looks good to me. But it is a common fix, not HDFS. So the patch needs to be under a HADOOP jira.
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: patch-HDFS-1967.patch
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038816#comment-13038816 ]
Matt Foley commented on HDFS-1967:
----------------------------------
Todd commented in another thread:
bq. It's still not clear why TestHDFSTrash was putting stuff in the local filesystem's .Trash folder. ... TestHDFSTrash is supposed to test HDFS, not local.
Completely agree. If you can help me understand what TestHDFSTrash thinks it's doing, and whether it accomplishes it, I'd appreciate it. I think it's trying to do exactly equivalent tests on Local FileSystem with TestTrash and HDFS FileSystem with TestHDFSTrash, but it doesn't seem to be succeeding.
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109224#comment-13109224 ]
Mayank Bansal commented on HDFS-1967:
-------------------------------------
opened the jira HADOOP-7663
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: patch-HDFS-1967.patch
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038886#comment-13038886 ]
Matt Foley commented on HDFS-1967:
----------------------------------
Suresh suggested that when running TestHDFSTrash on HDFS, the unit test should clear the .Trash subdirectory before running. This will help avoid issues with what is effectively a shared Trash directory, although simultaneous execution of two builds on the same server may still have problems. (As pointed out in HADOOP-7326, we can't clear the Trash directory when running on Local, as this may be your personal Trash folder!)
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1967) TestHDFSTrash failing on trunk and 22
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mayank Bansal updated HDFS-1967:
--------------------------------
Attachment: patch-HDFS-1967.patch
Initial Patch
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: patch-HDFS-1967.patch
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Mayank Bansal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109206#comment-13109206 ]
Mayank Bansal commented on HDFS-1967:
-------------------------------------
The problem was Trash emptier was running on the hdfs however files were getting deleted on the local file system. So that's the reason this was failing.
Adding both to local file system fixed the issue.
Submitting the patch.
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: patch-HDFS-1967.patch
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080592#comment-13080592 ]
Konstantin Boudnik commented on HDFS-1967:
------------------------------------------
Oh, I love messages like that whenever tests fail:
{noformat}
FAILED
null
junit.framework.AssertionFailedError: null
at org.apache.hadoop.fs.TestTrash.testTrashEmptier(TestTrash.java:473)
{format}
they are so highly informative and mnemonic!
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1967) TestHDFSTrash failing on trunk and 22
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-1967:
--------------------------------------
Priority: Blocker (was: Major)
This is consistently failing on 0.22 with the following diagnostics:
{code}
------------- Standard Error -----------------
Problem with Trash.. Consider using -skipTrash option
rmr: Cannot move "hdfs://my-0052.host.com:43098/user/xxx" to the trash, as it contains the trash
------------- ---------------- ---------------
Testcase: testTrash took 0.867 sec
Testcase: testNonDefaultFS took 0.011 sec
Testcase: testTrashEmptier took 0.037 sec
FAILED
null
junit.framework.AssertionFailedError: null
at org.apache.hadoop.fs.TestTrash.testTrashEmptier(TestTrash.java:473)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
at junit.extensions.TestSetup.run(TestSetup.java:27)
{code}
Making it a blocker for 0.22.
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Priority: Blocker
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036613#comment-13036613 ]
Todd Lipcon commented on HDFS-1967:
-----------------------------------
best I can tell, when TestHDFSTrash runs, it's removing things on HDFS and then looking at the state of the trash dir on local filesystem to verify things... debugging it cross-project is nearly impossible, though.
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1967) TestHDFSTrash failing on trunk and 22
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HDFS-1967:
-----------------------------
Issue Type: Sub-task (was: Bug)
Parent: HDFS-1852
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038276#comment-13038276 ]
Matt Foley commented on HDFS-1967:
----------------------------------
TestHDFSTrash.testTrashEmptier() was failing on almost every Hudson build through 605. However, has not failed for the last four auto-test builds. Watch-listing for trunk.
However, we'd like to understand what fixed it (if it is fixed) so we can apply the patch to v22 and yahoo-merge branches.
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038296#comment-13038296 ]
Todd Lipcon commented on HDFS-1967:
-----------------------------------
I think someone must have cleared out ~/.Trash on the hudson build machines.
This test is a bit of a mess - would be great if someone could clean it up. I think cross-project test inheritance should be made illegal - makes it very hard to debug test failures like this.
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038894#comment-13038894 ]
Todd Lipcon commented on HDFS-1967:
-----------------------------------
When running on HDFS, it should have its own MiniDFSCluster anyway -- which means that it would be very surprising to see the .Trash directory exist at all! Somehow TestHDFSTrash just isn't looking at HDFS, best I can tell.
Let's pretend these tests never existed, scrap them, and rewrite them....
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1967) TestHDFSTrash failing on trunk and
22
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036603#comment-13036603 ]
Todd Lipcon commented on HDFS-1967:
-----------------------------------
This seems to be an artifact of the current contents of the ~/.Trash directory on the build machines:
{code}
$ ls ~/.Trash/
110519062812 110519062818 110519062842 Current
{code}
On my local machine if I make ~/.Trash look like this, the test fails too.
> TestHDFSTrash failing on trunk and 22
> -------------------------------------
>
> Key: HDFS-1967
> URL: https://issues.apache.org/jira/browse/HDFS-1967
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Fix For: 0.22.0
>
>
> Seems to have started failing recently in many commit builds as well as the last two nightly builds of 22:
> https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/51/testReport/org.apache.hadoop.hdfs/TestHDFSTrash/testTrashEmptier/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira