You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Junping Du (JIRA)" <ji...@apache.org> on 2012/09/16 17:31:07 UTC
[jira] [Created] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Junping Du created HADOOP-8817:
----------------------------------
Summary: Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
Key: HADOOP-8817
URL: https://issues.apache.org/jira/browse/HADOOP-8817
Project: Hadoop Common
Issue Type: Sub-task
Affects Versions: 1.0.0
Reporter: Junping Du
Assignee: Junping Du
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8817) Backport Network Topology
Extension for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456889#comment-13456889 ]
Junping Du commented on HADOOP-8817:
------------------------------------
Thanks Suresh for the correction here. So branch-1.2 is next feature branch (1.1 is next bug-fix branch). Isn't it?
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8817:
-------------------------------
Attachment: HADOOP-8817-v2.patch
Updated patch with rebase to recent commits on branch-1 and bug fix.
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch, HADOOP-8817-v2.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8817) Backport Network Topology
Extension for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495869#comment-13495869 ]
Junping Du commented on HADOOP-8817:
------------------------------------
I meet some issues in running ant test on local environment that even branch-1 without patch cannot pass all tests. I run related unit tests which is good and did lots of tests on real cluster for this patch (reliability and performance) which you can find in attachment in HADOOP-8848. Anyone can help to verify this patch to pass "ant test"?
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch, HADOOP-8817-v2.patch, HADOOP-8817-v3.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8817) Backport Network Topology
Extension for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489624#comment-13489624 ]
Junping Du commented on HADOOP-8817:
------------------------------------
Luke. Thanks for the review and comments. I will attach these results soon.
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch, HADOOP-8817-v2.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8817:
-------------------------------
Attachment: HADOOP-8817-v3.patch
Update patch slightly due to Nicholas's comments in HADOOP-8820
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch, HADOOP-8817-v2.patch, HADOOP-8817-v3.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8817) Backport Network Topology
Extension for Virtualization (HADOOP-8468) to branch-1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489615#comment-13489615 ]
Luke Lu commented on HADOOP-8817:
---------------------------------
The changes lgtm. Please post ant test and test-patch results. Thanks Junping.
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch, HADOOP-8817-v2.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8817:
-------------------------------
Attachment: HADOOP-8817-v4.patch
Address Jing's comments with addressing one NPE issue and remove some redundant code.
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch, HADOOP-8817-v2.patch, HADOOP-8817-v3.patch, HADOOP-8817-v4.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8817:
-------------------------------
Description:
HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
3. Update balancer policy:HDFS-3495
4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
This JIRA will address the backport work which will be divided into 4 issues/patches in related jiras.
was:
HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
3. Update balancer policy:HDFS-3495
4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality.
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work which will be divided into 4 issues/patches in related jiras.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Suresh Srinivas updated HADOOP-8817:
------------------------------------
Target Version/s: 1.2.0 (was: 1.0.3)
Target version cannot be 1.0.3 since it is already released.
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8817:
-------------------------------
Description:
HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
3. Update balancer policy:HDFS-3495
4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
was:
HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
3. Update balancer policy:HDFS-3495
4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
This JIRA will address the backport work which will be divided into 4 issues/patches in related jiras.
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8817:
-------------------------------
Attachment: HADOOP-8817.patch
Attach the whole patch as an example and will divided into 4 patches later.
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
> Attachments: HADOOP-8817.patch
>
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality: YARN-18, YARN-19.
> This JIRA will address the backport work on branch-1 which will be divided into 4 issues/patches in related jira issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8817) Backport Network Topology Extension
for Virtualization (HADOOP-8468) to branch-1
Posted by "Junping Du (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Junping Du updated HADOOP-8817:
-------------------------------
Description:
HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
3. Update balancer policy:HDFS-3495
4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality.
Target Version/s: 1.0.3
Labels: features (was: )
> Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1
> --------------------------------------------------------------------------------
>
> Key: HADOOP-8817
> URL: https://issues.apache.org/jira/browse/HADOOP-8817
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Junping Du
> Assignee: Junping Du
> Labels: features
>
> HADOOP-8468 propose network topology changes for running on virtualized infrastructure, which includes:
> 1. Add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup): HADOOP-8469, HADOOP-8470
> 2. Update Replica Placement/Removal Policy to reflect new topology layer: HDFS-3498, HDFS-3601
> 3. Update balancer policy:HDFS-3495
> 4. Update Task Scheduling Policy to reflect new topology layer and support the case that compute nodes (NodeManager or TaskTracker) and data nodes are separated into different VMs, but still benefit from physical host locality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira