[jira] [Commented] (HDDS-2031) Choose datanode for pipeline creation based on network topology

["Stephen O'Donnell (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDDS-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034379#comment-17034379 ] 

Stephen O'Donnell commented on HDDS-2031:
-----------------------------------------

[~Sammi] I have been looking into the Network Topology feature to evaluate if it is completed. Is this change intended to go into the RatisPipelineProvider, as currently it does not look as though it considers NetworkTopology when assigning new pipelines. The code to assign a pipeline, I think, is:

{code}
    List<DatanodeDetails> dns =
        nodeManager.getNodes(NodeState.HEALTHY)
            .parallelStream()
            .filter(dn -> !dnsUsed.contains(dn))
            .limit(factor.getNumber())
            .collect(Collectors.toList());
{code}

As you can see it simply picks unused nodes at random. Please correct my understanding if I have got this wrong.

If this is a key part of the implementation, are you planning to work on it soon?

> Choose datanode for pipeline creation based on network topology
> ---------------------------------------------------------------
>
>                 Key: HDDS-2031
>                 URL: https://issues.apache.org/jira/browse/HDDS-2031
>             Project: Hadoop Distributed Data Store
>          Issue Type: Sub-task
>            Reporter: Sammi Chen
>            Assignee: Sammi Chen
>            Priority: Major
>
> There are regular heartbeats between datanodes in a pipeline. Choose datanodes based on network topology, to guarantee data reliability and reduce heartbeat network traffic latency.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org