You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Baolong Mao (Jira)" <ji...@apache.org> on 2021/01/14 01:00:03 UTC
[jira] [Updated] (HDDS-4656) Add a container balancer tool or
service for SCM
[ https://issues.apache.org/jira/browse/HDDS-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Baolong Mao updated HDDS-4656:
------------------------------
Description:
When an existing Ozone cluster is nearly full, we have to add more datanodes into the Ozone cluster, but there are two issue we must face.
- When new allocate container request coming, SCM should better to choose the datanodes in low usage, if not, the performance will getting pool.
- For read request, the existing datanodes stored lots of blocks, so they are responsible for serving the read request and supply the data stream service, meanwhile, the new coming datanodes can help nothing.
If we have a balancer tool just like hdfs balancer, we can move the block or container from some high usage datanodes to low, I think this is one of necessary tools for Ozone.
container balancer design doc https://docs.google.com/document/d/15PdYaP6aLB18ptbcOK3XWlL4Y1PLfIP-ll30KjNPZ2g/edit?usp=sharing
was:
When an existing Ozone cluster is nearly full, we have to add more datanodes into the Ozone cluster, but there are two issue we must face.
- When new allocate container request coming, SCM should better to choose the datanodes in low usage, if not, the performance will getting pool.
- For read request, the existing datanodes stored lots of blocks, so they are responsible for serving the read request and supply the data stream service, meanwhile, the new coming datanodes can help nothing.
If we have a balancer tool just like hdfs balancer, we can move the block or container from some high usage datanodes to low, I think this is one of necessary tools for Ozone.
> Add a container balancer tool or service for SCM
> ------------------------------------------------
>
> Key: HDDS-4656
> URL: https://issues.apache.org/jira/browse/HDDS-4656
> Project: Hadoop Distributed Data Store
> Issue Type: New Feature
> Components: SCM, Tools
> Affects Versions: 1.1.0
> Reporter: Baolong Mao
> Assignee: Baolong Mao
> Priority: Major
>
> When an existing Ozone cluster is nearly full, we have to add more datanodes into the Ozone cluster, but there are two issue we must face.
> - When new allocate container request coming, SCM should better to choose the datanodes in low usage, if not, the performance will getting pool.
> - For read request, the existing datanodes stored lots of blocks, so they are responsible for serving the read request and supply the data stream service, meanwhile, the new coming datanodes can help nothing.
> If we have a balancer tool just like hdfs balancer, we can move the block or container from some high usage datanodes to low, I think this is one of necessary tools for Ozone.
> container balancer design doc https://docs.google.com/document/d/15PdYaP6aLB18ptbcOK3XWlL4Y1PLfIP-ll30KjNPZ2g/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org