You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Raghu Angadi (JIRA)" <ji...@apache.org> on 2008/09/30 20:55:44 UTC
[jira] Created: (HADOOP-4314) TestReplication fails quite often
TestReplication fails quite often
---------------------------------
Key: HADOOP-4314
URL: https://issues.apache.org/jira/browse/HADOOP-4314
Project: Hadoop Core
Issue Type: Bug
Components: dfs, test
Affects Versions: 0.18.0
Reporter: Raghu Angadi
Priority: Blocker
Fix For: 0.18.2
TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4314) TestReplication fails quite often
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636153#action_12636153 ]
Hadoop QA commented on HADOOP-4314:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12391252/HADOOP-4314.patch
against trunk revision 700628.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/console
This message is automatically generated.
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635911#action_12635911 ]
Raghu Angadi commented on HADOOP-4314:
--------------------------------------
This affects only simulated datanodes.
When a SimulatedDataNode sends block report, it includes the blocks that are currently being written. The reported size of such blocks is the default blocks size (64MB). NameNode assumes it is a real block. When the datanodes report the block when the writing finishes, NameNode rejects those blocks unless the size is same as 64MB. This bug always existed but only since 0.18, NameNode started rejecting blocks that are smaller than largest size reported by any datanode. Even replication attempts fail.
The fix is to exclude blocks that are not "finalized" at the datanode in the block report from simulated datanodes. I will attach a patch.
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Hudson test failure is a known intermittent failure and not related to this patch.
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4314) TestReplication fails quite often
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636652#action_12636652 ]
Hudson commented on HADOOP-4314:
--------------------------------
Integrated in Hadoop-trunk #622 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/622/])
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18-java5-fix.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Attachment: HADOOP-4314-branch-18.patch
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Attachment: HADOOP-4314-branch-18.patch
That is better. Thanks Nicholas. Updated patch is attached.
The change in TestReplication.java is strictly not required but it just removes some code replication.
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Attachment: HADOOP-4314-branch-18-java5-fix.patch
Java 5 does not have {{Arrays.copyOf()}}. Attached patch will be committed to 0.18 in addition to the earlier patch.
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18-java5-fix.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Release Note: Simulated Datanodes should not include blocks that are still being written in their block report.
Status: Patch Available (was: Open)
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-4314:
------------------------------------
Assignee: Raghu Angadi
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4314:
-------------------------------------------
Description: TestReplication times out once every 5-6 attempts. I will attach log from a failed test. (was:
TestReplication times out once every 5-6 attempts. I will attach log from a failed test.)
Hadoop Flags: [Reviewed]
+1 patch looks good
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Attachment: TestReplication.txt
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Attachment: HADOOP-4314-branch-18.patch
update : trim the size of the array returned if required.
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4314) TestReplication fails quite often
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635915#action_12635915 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4314:
------------------------------------------------
I think it is better to use blockMap.values() in the loop since only the values are needed.
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Attachment: HADOOP-4314.patch
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4314) TestReplication fails quite often
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4314:
---------------------------------
Attachment: HADOOP-4314-branch-18.patch
Updated patch for branch 0.18 == (earlier branch-18.patch + java5-fix.patch).
> TestReplication fails quite often
> ---------------------------------
>
> Key: HADOOP-4314
> URL: https://issues.apache.org/jira/browse/HADOOP-4314
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.18.2
>
> Attachments: HADOOP-4314-branch-18-java5-fix.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314-branch-18.patch, HADOOP-4314.patch, TestReplication.txt
>
>
> TestReplication times out once every 5-6 attempts. I will attach log from a failed test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.