You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/07/14 15:02:54 UTC
[GitHub] [ozone] bshashikant opened a new pull request #2416: HDDS-5441. Disallow same set of DNs to be part of multiple pipelines
bshashikant opened a new pull request #2416:
URL: https://github.com/apache/ozone/pull/2416
## What changes were proposed in this pull request?
The patch disallows same set of datanodes to participate in multiple pipelines.Right now, the behaviour is controlled by a config.
## What is the link to the Apache JIRA
[https://issues.apache.org/jira/browse/HDDS-5441]
## How was this patch tested?
Added unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] nandakumar131 commented on a change in pull request #2416: HDDS-5441. Disallow same set of DNs to be part of multiple pipelines
Posted by GitBox <gi...@apache.org>.
nandakumar131 commented on a change in pull request #2416:
URL: https://github.com/apache/ozone/pull/2416#discussion_r680141848
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/SCMCommonPlacementPolicy.java
##########
@@ -314,4 +316,14 @@ public ContainerPlacementStatus validateContainerPlacement(
return new ContainerPlacementStatusDefault(
(int)currentRackCount, requiredRacks, numRacks);
}
+
+ /**
+ * Removes the datanode peers from all the existing pipelines for this dn.
+ */
+ public void removePeers(DatanodeDetails dn,
+ List<DatanodeDetails> healthyList) {
+ if (ScmUtils.shouldRemovePeers(conf)) {
Review comment:
Instead of calling `ScmUtils.shouldRemovePeers(conf)` repeatedly, can this value be stored in an instance variable?
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
##########
@@ -885,6 +888,29 @@ public int minPipelineLimit(List<DatanodeDetails> dnList) {
return Collections.min(pipelineCountList);
}
+ @Override
+ public Collection<DatanodeDetails> getPeerList(DatanodeDetails dn) {
+ HashSet<DatanodeDetails> dns = new HashSet<>();
+ Preconditions.checkNotNull(dn);
+ Set<PipelineID> pipelines =
+ nodeStateManager.getPipelineByDnID(dn.getUuid());
+ if (!pipelines.isEmpty()) {
+ pipelines.forEach(id -> {
+ try {
+ Pipeline pipeline =
+ scmContext.getScm().getPipelineManager().getPipeline(id);
Review comment:
We can cache the `PipelineManager` instance outside of `forEach` instead of making `scmContext.getScm().getPipelineManager()` call inside `forEach` every time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] nandakumar131 merged pull request #2416: HDDS-5441. Disallow same set of DNs to be part of multiple pipelines
Posted by GitBox <gi...@apache.org>.
nandakumar131 merged pull request #2416:
URL: https://github.com/apache/ozone/pull/2416
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] dineshchitlangia commented on a change in pull request #2416: HDDS-5441. Disallow same set of DNs to be part of multiple pipelines
Posted by GitBox <gi...@apache.org>.
dineshchitlangia commented on a change in pull request #2416:
URL: https://github.com/apache/ozone/pull/2416#discussion_r675672349
##########
File path: hadoop-hdds/common/src/main/resources/ozone-default.xml
##########
@@ -843,6 +843,14 @@
by the no of metadata volumes reported per dn.
</description>
</property>
+ <property>
+ <name>ozone.scm.datanode.disallow.same.peers</name>
+ <value>false</value>
+ <tag>OZONE, SCM, PIPELINE</tag>
+ <description>Disallows same set of datanodes to participate in multiple
+ pipelines when set top true. Default is set to false.
Review comment:
```suggestion
pipelines when set to true. Default is set to false.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org