You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Junping Du (JIRA)" <ji...@apache.org> on 2012/06/04 10:26:23 UTC
[jira] [Updated] (HADOOP-8470) Implementation of 4-layer subclass
of NetworkTopology (NetworkTopologyWithNodeGroup)
[ https://issues.apache.org/jira/browse/HADOOP-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8470:
-------------------------------
Attachment: HADOOP-8470-NetworkTopology-new-impl.patch
This patch will depends on the checkin of
https://issues.apache.org/jira/browse/HADOOP-8469
> Implementation of 4-layer subclass of NetworkTopology (NetworkTopologyWithNodeGroup)
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-8470
> URL: https://issues.apache.org/jira/browse/HADOOP-8470
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0, 2.0.0-alpha
> Reporter: Junping Du
> Assignee: Junping Du
> Attachments: HADOOP-8470-NetworkTopology-new-impl.patch
>
>
> To support the four-layer hierarchical topology shown in attached figure as a subclass of NetworkTopology, NetworkTopologyWithNodeGroup was developed along with unit tests. NetworkTopologyWithNodeGroup overriding the methods add, remove, and pseudoSortByDistance were the most relevant to support the four-layer topology. The method seudoSortByDistance selects the nodes to use for reading data and sorts the nodes in sequence of node-local, nodegroup-local, rack- local, rack–off. Another slightly change to seudoSortByDistance is to support cases of separation data node and node manager: if the reader cannot be found in NetworkTopology tree (formed by data nodes only), then it will try to sort according to reader's sibling node in the tree.
> The distance calculation changes the weights from 0 (local), 2 (rack- local), 4 (rack-off) to: 0 (local), 2 (nodegroup-local), 4 (rack-local), 6 (rack-off).
> The additional node group layer should be specified in the topology script or table mapping, e.g. input 10.1.1.1, output: /rack1/nodegroup1
> A subclass on InnerNode, InnerNodeWithNodeGroup, was also needed to support NetworkTopologyWithNodeGroup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira