You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Patrick Hunt (Created) (JIRA)" <ji...@apache.org> on 2012/02/17 00:21:00 UTC
[jira] [Created] (MAPREDUCE-3872) event handling races in
ContainerLauncherImpl and TestContainerLauncher
event handling races in ContainerLauncherImpl and TestContainerLauncher
-----------------------------------------------------------------------
Key: MAPREDUCE-3872
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 0.23.1
Reporter: Patrick Hunt
TestContainerLauncher is failing intermittently for me.
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3872) event handling races in
ContainerLauncherImpl and TestContainerLauncher
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214987#comment-13214987 ]
Hadoop QA commented on MAPREDUCE-3872:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515773/MAPREDUCE-3872.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1915//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1915//console
This message is automatically generated.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
> Affects Versions: 0.23.1
> Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch, MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: <null> but was: Expected 22 but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in
ContainerLauncherImpl and TestContainerLauncher
Posted by "Patrick Hunt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated MAPREDUCE-3872:
------------------------------------
Attachment: MAPREDUCE-3872.patch
This patch fixes three issues:
1) ContainerLauncherImpl had a race in updating "allNodes". This might result in the pool size being incorrectly calculated.
2) test had an issues where waitForEvent could run forever
3) there's a race btw eventHandlingThread in ContainerLauncherImpl and the test code. In some cases the event would be processed, and the pool size increased, before all events had been processed (and allNodes updated with all new hosts). I believe this is the result of the test failure I documented in the description.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
> Affects Versions: 0.23.1
> Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: <null> but was: Expected 22 but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in
ContainerLauncherImpl and TestContainerLauncher
Posted by "Patrick Hunt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated MAPREDUCE-3872:
------------------------------------
Description:
TestContainerLauncher is failing intermittently for me.
{noformat}
junit.framework.AssertionFailedError: Expected: <null> but was: Expected 22 but found 21
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertNull(Assert.java:233)
at junit.framework.Assert.assertNull(Assert.java:226)
at org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
{noformat}
Patch momentarily.
was:
TestContainerLauncher is failing intermittently for me.
{noformat}
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
> Affects Versions: 0.23.1
> Reporter: Patrick Hunt
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: <null> but was: Expected 22 but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in
ContainerLauncherImpl and TestContainerLauncher
Posted by "Patrick Hunt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated MAPREDUCE-3872:
------------------------------------
Status: Patch Available (was: Open)
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
> Affects Versions: 0.23.1
> Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: <null> but was: Expected 22 but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in
ContainerLauncherImpl and TestContainerLauncher
Posted by "Patrick Hunt (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated MAPREDUCE-3872:
------------------------------------
Attachment: MAPREDUCE-3872.patch
Refreshing the patch. Looks like MAPREDUCE-3634 fixed a number of the issues I had originally seen/fixed in this patch.
The latest version of this patch fixes the obvious concurrency bug in updating allNodes. This patch is currently tested by the unit tests, I don't see a way to trigger the bad case given it's non-deterministic. However by inspection you can see the obvious concurrency bug that exists in the current code.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
> Affects Versions: 0.23.1
> Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch, MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: <null> but was: Expected 22 but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in
ContainerLauncherImpl and TestContainerLauncher
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3872:
-------------------------------------
Component/s: mrv2
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
> Affects Versions: 0.23.1
> Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: <null> but was: Expected 22 but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3872) event handling races in
ContainerLauncherImpl and TestContainerLauncher
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209917#comment-13209917 ]
Hadoop QA commented on MAPREDUCE-3872:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12514910/MAPREDUCE-3872.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1883//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1883//console
This message is automatically generated.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
> Affects Versions: 0.23.1
> Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: <null> but was: Expected 22 but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira