[jira] [Created] (HDFS-14180) Suspect there is a bug in NetworkTopology.java chooseRandom function.

["Sihai Ke (JIRA)" <ji...@apache.org>]

Sihai Ke created HDFS-14180:
-------------------------------

             Summary: Suspect there is a bug in NetworkTopology.java chooseRandom function.
                 Key: HDFS-14180
                 URL: https://issues.apache.org/jira/browse/HDFS-14180
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: hdfs, namenode
    Affects Versions: 2.9.2
            Reporter: Sihai Ke


During reading the hadoop NetworkTopology.java, I suspect there is a bug in function 

chooseRandom (line 498, hadoop version 2.9.2-RC0), 

 I think there is a bug in{color:#f79232} code,{color:#333333} ~excludedScope doesn't mean availableNodes under Scope node, and I also add unit test for this and get an exception.{color}{color}
{quote} 
protected Node chooseRandom(final String scope, String excludedScope,
 final Collection<Node> excludedNodes) {
 if (excludedScope != null) {
 if (scope.startsWith(excludedScope)) {
 return null;
 }
 if (!excludedScope.startsWith(scope)) {
 excludedScope = null;
 }
 }
 Node node = getNode(scope);
 if (!(node instanceof InnerNode)) {
 return excludedNodes != null && excludedNodes.contains(node) ?
 null : node;
 }
 InnerNode innerNode = (InnerNode)node;
 int numOfDatanodes = innerNode.getNumOfLeaves();
 if (excludedScope == null) {
 node = null;
 } else {
 node = getNode(excludedScope);
 if (!(node instanceof InnerNode)) {
 numOfDatanodes -= 1;
 } else {
 numOfDatanodes -= ((InnerNode)node).getNumOfLeaves();
 }
 }
 if (numOfDatanodes <= 0) {
 LOG.debug("Failed to find datanode (scope=\"{}\" excludedScope=\"{}\")."
 + " numOfDatanodes={}",
 scope, excludedScope, numOfDatanodes);
 return null;
 }
 final int availableNodes;
 if (excludedScope == null) {
 availableNodes = countNumOfAvailableNodes(scope, excludedNodes);
 } else {
 availableNodes =
 {color:#f79232}countNumOfAvailableNodes("~" + excludedScope, excludedNodes);{color}
 }
 LOG.debug("Choosing random from {} available nodes on node {},"
 + " scope={}, excludedScope={}, excludeNodes={}. numOfDatanodes={}.",
 availableNodes, innerNode, scope, excludedScope, excludedNodes,
 numOfDatanodes);
 Node ret = null;
 if (availableNodes > 0) {
 ret = chooseRandom(innerNode, node, excludedNodes, numOfDatanodes,
 availableNodes);
 }
 LOG.debug("chooseRandom returning {}", ret);
 return ret;
}{quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org