You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Vinayakumar B (JIRA)" <ji...@apache.org> on 2016/01/25 14:22:39 UTC

[jira] [Comment Edited] (HDFS-9689) Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently

    [ https://issues.apache.org/jira/browse/HDFS-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115072#comment-15115072 ] 

Vinayakumar B edited comment on HDFS-9689 at 1/25/16 1:22 PM:
--------------------------------------------------------------

One possible solution to Namenode restart tests, is to implement {{BPOfferService#refreshNNList()}} in way to support in between changing of namenode addresses, and issue, DN#refreshNamenodes(Conf) call to all DNs after restart in MiniDfsCluster.
This way all tests which fails intermittently with this kind of problems would be solved.

After analyzing, current MiniDfsCluster, is not having option to restartNamenode in another optional ports. 
If this support is required to be added, there are some points needs to be checked.
1. FileSystem's URI will change in case of NonHA, So all FileSystem's instances should be refreshed in tests after restart of Namenode.

2. Current restartDatanode(..) have keepPort default as false, but this will work only if {{setupHostsFile}} is false. Otherwise, DN will try to restart in same port. This could result in some occasional test failures.

3. In case of restartDatanodes() or restartNamenodes(),, assertions of URIs or DN names, should be checked, as these may change with changed port post restart.

So, to completely resolve issues raising due to Port Bind issues, from restart (Name|Data)nodes needs some effort :)

What you say, [~liuml07] ?


was (Author: vinayrpet):
One possible solution to Namenode restart tests, is to implement {{BPOfferService#refreshNNList()}} in way to support in between changing of namenode addresses, and issue, DN#refreshNamenodes(Conf) call to all DNs after restart in MiniDfsCluster.
This way all tests which fails intermittently with this kind of problems would be solved.

What you say, [~liuml07] ?

> Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently 
> ---------------------------------------------------------
>
>                 Key: HDFS-9689
>                 URL: https://issues.apache.org/jira/browse/HDFS-9689
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 3.0.0, 2.8.0
>            Reporter: Mingliang Liu
>            Assignee: Mingliang Liu
>         Attachments: HDFS-9689.000.patch
>
>
> The test fails in recent builds, e.g.
> https://builds.apache.org/job/PreCommit-HDFS-Build/14063/testReport/org.apache.hadoop.hdfs/TestRenameWhileOpen/
> and
> https://builds.apache.org/job/PreCommit-HDFS-Build/14212/testReport/org.apache.hadoop.hdfs/TestRenameWhileOpen/testWhileOpenRenameToNonExistentDirectory/
> The *Error Message* is like:
> {code}
> Problem binding to [localhost:60690] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
> {code}
> and *Stacktrace* is:
> {code}
> java.net.BindException: Problem binding to [localhost:60690] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
> 	at sun.nio.ch.Net.bind0(Native Method)
> 	at sun.nio.ch.Net.bind(Net.java:463)
> 	at sun.nio.ch.Net.bind(Net.java:455)
> 	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
> 	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> 	at org.apache.hadoop.ipc.Server.bind(Server.java:469)
> 	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:695)
> 	at org.apache.hadoop.ipc.Server.<init>(Server.java:2464)
> 	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:958)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:535)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:510)
> 	at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:800)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.<init>(NameNodeRpcServer.java:392)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.createRpcServer(NameNode.java:743)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:685)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:884)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:863)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1581)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:1247)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1016)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:891)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:823)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:482)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:441)
> 	at org.apache.hadoop.hdfs.TestRenameWhileOpen.testWhileOpenRenameToNonExistentDirectory(TestRenameWhileOpen.java:332)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)