You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2008/10/21 03:24:44 UTC
[jira] Created: (HADOOP-4470) Tests fail on windows due to
BindException.
Tests fail on windows due to BindException.
-------------------------------------------
Key: HADOOP-4470
URL: https://issues.apache.org/jira/browse/HADOOP-4470
Project: Hadoop Core
Issue Type: Bug
Components: dfs, test
Affects Versions: 0.19.0
Environment: Windows, cygwin.
Reporter: Konstantin Shvachko
Fix For: 0.19.0
On Windows a series of tests fail starting from {{TestAbandonBlock}}.
The reason is that name-nodes or data-nodes cannot start because of the "BindException: Address already in use: connect".
Seems like some of the tests that run before {{TestAbandonBlock}} are not releasing resources or not closing sockets.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4470) Tests fail on windows due to
BindException.
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641349#action_12641349 ]
Steve Loughran commented on HADOOP-4470:
----------------------------------------
That's interesting. Certainly I can create similar scenarios in my tests, but that's because I'm doing my test runs all in the same process, so until I have shutdown working properly across all services (and their wrapper classes) I can see similar events. But in the Hadoop core junit classes, every test class is running in its own process. If the port is staying open, it means that the test case isn't stopping.
1. You can use netstat -a -p to list the processes that have open ports; link that using jps -v to the java process at fault.
2. do you have test.junit.fork.mode set to something other than perTest ?
> Tests fail on windows due to BindException.
> -------------------------------------------
>
> Key: HADOOP-4470
> URL: https://issues.apache.org/jira/browse/HADOOP-4470
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Environment: Windows, cygwin.
> Reporter: Konstantin Shvachko
> Fix For: 0.19.0
>
>
> On Windows a series of tests fail starting from {{TestAbandonBlock}}.
> The reason is that name-nodes or data-nodes cannot start because of the "BindException: Address already in use: connect".
> Seems like some of the tests that run before {{TestAbandonBlock}} are not releasing resources or not closing sockets.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4470) Tests fail on windows due to
BindException.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641246#action_12641246 ]
Konstantin Shvachko commented on HADOOP-4470:
---------------------------------------------
I narrowed down the number of test preceding TestAbandonBlock that lead to massive failures to 2 tests:
- {{TestFTPFileSystem}} and
- {{TestLoadGenerator}}
Removing any of the two stops failures. Here is the log when both tests present:
{code}
test-core:
[mkdir] Created dir: c:\Work\Projects\Hadoop\build\test\data
[mkdir] Created dir: c:\Work\Projects\Hadoop\build\test\logs
[junit] Running org.apache.hadoop.fs.ftp.TestFTPFileSystem
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 10.594 sec
[junit] Running org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 37.643 sec
[junit] Running org.apache.hadoop.hdfs.TestAbandonBlock
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 1.718 sec
[junit] Test org.apache.hadoop.hdfs.TestAbandonBlock FAILED
[junit] Running org.apache.hadoop.hdfs.TestBlocksScheduledCounter
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.89 sec
[junit] Test org.apache.hadoop.hdfs.TestBlocksScheduledCounter FAILED
[junit] Running org.apache.hadoop.hdfs.TestCrcCorruption
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 3.36 sec
[junit] Test org.apache.hadoop.hdfs.TestCrcCorruption FAILED
[junit] Running org.apache.hadoop.hdfs.TestDFSClientRetries
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 1.016 sec
[junit] Test org.apache.hadoop.hdfs.TestDFSClientRetries FAILED
[junit] Running org.apache.hadoop.hdfs.TestDFSFinalize
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.766 sec
[junit] Test org.apache.hadoop.hdfs.TestDFSFinalize FAILED
[junit] Running org.apache.hadoop.hdfs.TestDFSMkdirs
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.922 sec
[junit] Test org.apache.hadoop.hdfs.TestDFSMkdirs FAILED
[junit] Running org.apache.hadoop.hdfs.TestDFSPermission
[junit] Tests run: 3, Failures: 0, Errors: 3, Time elapsed: 3.328 sec
[junit] Test org.apache.hadoop.hdfs.TestDFSPermission FAILED
[junit] Running org.apache.hadoop.hdfs.TestDFSRename
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 3.188 sec
[junit] Test org.apache.hadoop.hdfs.TestDFSRename FAILED
[junit] Running org.apache.hadoop.hdfs.TestDFSRollback
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0.75 sec
[junit] Test org.apache.hadoop.hdfs.TestDFSRollback FAILED
[junit] Running org.apache.hadoop.hdfs.TestDFSShell
[junit] Tests run: 13, Failures: 0, Errors: 13, Time elapsed: 9.141 sec
[junit] Test org.apache.hadoop.hdfs.TestDFSShell FAILED
{code}
The tests fail with on of the following exceptions
{code}
Testcase: testAbandonBlock took 4.736 sec
Caused an ERROR
Unable to establish loopback connection
java.io.IOException: Unable to establish loopback connection
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:106)
at java.security.AccessController.doPrivileged(Native Method)
at sun.nio.ch.PipeImpl.<init>(PipeImpl.java:122)
at sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:27)
at java.nio.channels.Pipe.open(Pipe.java:133)
at sun.nio.ch.WindowsSelectorImpl.<init>(WindowsSelectorImpl.java:105)
at sun.nio.ch.WindowsSelectorProvider.openSelector(WindowsSelectorProvider.java:26)
at java.nio.channels.Selector.open(Selector.java:209)
at org.apache.hadoop.ipc.Server$Responder.<init>(Server.java:442)
at org.apache.hadoop.ipc.Server.<init>(Server.java:961)
at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:436)
at org.apache.hadoop.ipc.RPC.getServer(RPC.java:398)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:373)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:205)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1199)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1154)
at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:414)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:278)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:119)
at org.apache.hadoop.hdfs.TestAbandonBlock.testAbandonBlock(TestAbandonBlock.java:38)
Caused by: java.net.BindException: Address already in use: connect
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:78)
{code}
or
{code}
Testcase: testBlocksScheduledCounter took 0.906 sec
Caused an ERROR
Unable to establish loopback connection
java.io.IOException: Unable to establish loopback connection
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:106)
at java.security.AccessController.doPrivileged(Native Method)
at sun.nio.ch.PipeImpl.<init>(PipeImpl.java:122)
at sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:27)
at java.nio.channels.Pipe.open(Pipe.java:133)
at sun.nio.ch.WindowsSelectorImpl.<init>(WindowsSelectorImpl.java:105)
at sun.nio.ch.WindowsSelectorProvider.openSelector(WindowsSelectorProvider.java:26)
at java.nio.channels.Selector.open(Selector.java:209)
at org.apache.hadoop.ipc.Server$Responder.<init>(Server.java:442)
at org.apache.hadoop.ipc.Server.<init>(Server.java:961)
at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:436)
at org.apache.hadoop.ipc.RPC.getServer(RPC.java:398)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:153)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:208)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:194)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:859)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:275)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:119)
at org.apache.hadoop.hdfs.TestBlocksScheduledCounter.testBlocksScheduledCounter(TestBlocksScheduledCounter.java:41)
Caused by: java.net.BindException: Address already in use: connect
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:78)
{code}
> Tests fail on windows due to BindException.
> -------------------------------------------
>
> Key: HADOOP-4470
> URL: https://issues.apache.org/jira/browse/HADOOP-4470
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Environment: Windows, cygwin.
> Reporter: Konstantin Shvachko
> Fix For: 0.19.0
>
>
> On Windows a series of tests fail starting from {{TestAbandonBlock}}.
> The reason is that name-nodes or data-nodes cannot start because of the "BindException: Address already in use: connect".
> Seems like some of the tests that run before {{TestAbandonBlock}} are not releasing resources or not closing sockets.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4470) Tests fail on windows due to
BindException.
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-4470:
--------------------------------
Fix Version/s: (was: 0.19.0)
Not a blocker for 0.19.0
> Tests fail on windows due to BindException.
> -------------------------------------------
>
> Key: HADOOP-4470
> URL: https://issues.apache.org/jira/browse/HADOOP-4470
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.19.0
> Environment: Windows, cygwin.
> Reporter: Konstantin Shvachko
>
> On Windows a series of tests fail starting from {{TestAbandonBlock}}.
> The reason is that name-nodes or data-nodes cannot start because of the "BindException: Address already in use: connect".
> Seems like some of the tests that run before {{TestAbandonBlock}} are not releasing resources or not closing sockets.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.