You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Mukund Madhugiri (JIRA)" <ji...@apache.org> on 2007/09/21 01:45:50 UTC
[jira] Created: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Test dfs.TestFileCreation.testFileCreation failed on Windows
------------------------------------------------------------
Key: HADOOP-1932
URL: https://issues.apache.org/jira/browse/HADOOP-1932
Project: Hadoop
Issue Type: Bug
Components: test
Affects Versions: 0.14.2
Environment: Windows
Reporter: Mukund Madhugiri
Fails with this assert error:
junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1932:
-------------------------------------
Priority: Blocker (was: Major)
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1932:
-------------------------------------
Resolution: Fixed
Fix Version/s: 0.15.0
Status: Resolved (was: Patch Available)
I just committed this.
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530009 ]
Nigel Daley commented on HADOOP-1932:
-------------------------------------
Also occurs on Solaris:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/817/testReport/org.apache.hadoop.dfs/TestFileCreation/testFileCreation/
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529935 ]
Raghu Angadi commented on HADOOP-1932:
--------------------------------------
Isn't it true that no blocks are guaranteed to be present unless the file is closed? Adding one more byte, only makes the issue less probable, right?
Flusing to datanodes is guaranteed (by implementation, not by contract) even while writing exactly two blocks. What is not guaranteed is datanodes reporting the block to Namenode.
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1932:
-------------------------------------
Attachment: fileCloseTest.patch
The test was creating a file with two blocks. After writing to the two blocks (and before closing the file), it was verfying that the file size was 2 blocks. This wasn't always true because the client might not have flushed/closed the last block.
The fix is to make the test create a file that is slightly larger than 2 blocks. When the client finished writing to the whole file, it is guaranteed that the first two blocks have been flushed to the datanodes.
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1932:
-------------------------------------
Status: Patch Available (was: Open)
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529956 ]
Raghu Angadi commented on HADOOP-1932:
--------------------------------------
Thats right. Thanks.
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1932:
-------------------------------------
Affects Version/s: (was: 0.14.2)
0.15.0
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530241 ]
Hudson commented on HADOOP-1932:
--------------------------------
Integrated in Hadoop-Nightly #250 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/250/])
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.15.0
>
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530037 ]
Hadoop QA commented on HADOOP-1932:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12366478/fileCloseTest.patch
against trunk revision r579005.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/820/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/820/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/820/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/820/console
This message is automatically generated.
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur reassigned HADOOP-1932:
----------------------------------------
Assignee: dhruba borthakur
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1932) Test
dfs.TestFileCreation.testFileCreation failed on Windows
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529941 ]
dhruba borthakur commented on HADOOP-1932:
------------------------------------------
It is *guaranteed* that the namenode will allocate a new block for the file only if the penultimate block has been confirmed by at least the min number of datanodes. That shud take care of the issue that you mention.
> Test dfs.TestFileCreation.testFileCreation failed on Windows
> ------------------------------------------------------------
>
> Key: HADOOP-1932
> URL: https://issues.apache.org/jira/browse/HADOOP-1932
> Project: Hadoop
> Issue Type: Bug
> Components: test
> Affects Versions: 0.15.0
> Environment: Windows
> Reporter: Mukund Madhugiri
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: fileCloseTest.patch
>
>
> Fails with this assert error:
> junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
> at org.apache.hadoop.dfs.TestFileCreation.testFileCreation(TestFileCreation.java:137)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.