You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/01/17 21:58:31 UTC
[GitHub] [ozone] sodonnel opened a new pull request #2995: HDDS-6177. Extend container info command to include replica details
sodonnel opened a new pull request #2995:
URL: https://github.com/apache/ozone/pull/2995
## What changes were proposed in this pull request?
When debugging some recent cluster issues related to containers, it would be useful if the details of the replicas were available in the output from the `ozone admin container info <id>` command.
Ideally, the ContainerReplica object, which is used to stored the replicas details on SCM would be re-used for this purpose, but it is only available in the `hadoop-hdds-server-scm` package, while the client code lives in the common package. Creating a dependency
Therefore I have created a new class called ContainerReplicaInfo and corresponding proto definition to send the replica details from SCM to the client, and hence display on the command output.
With the change in place, the output from the command looks like below, with the line starting "Replicas":
```
ozone admin container info 2
Container id: 2
Pipeline id: ce499ca8-e9be-4ba4-91e7-0c7f33bb0ccb
Container State: OPEN
Datanodes: [2de8adb0-504c-4890-8d68-cc699c02cf01/ozone_datanode_2.ozone_default,
cd79c851-277a-4b89-8c6b-e1cc8f549ebc/ozone_datanode_1.ozone_default,
29b032e6-f76a-44a8-a582-427fc8a83aa9/ozone_datanode_3.ozone_default]
Replicas: [State: OPEN; Origin: cd79c851-277a-4b89-8c6b-e1cc8f549ebc; Location: cd79c851-277a-4b89-8c6b-e1cc8f549ebc/ozone_datanode_1.ozone_default,
State: OPEN; Origin: 2de8adb0-504c-4890-8d68-cc699c02cf01; Location: 2de8adb0-504c-4890-8d68-cc699c02cf01/ozone_datanode_2.ozone_default,
State: OPEN; Origin: 29b032e6-f76a-44a8-a582-427fc8a83aa9; Location: 29b032e6-f76a-44a8-a582-427fc8a83aa9/ozone_datanode_3.ozone_default]
```
More verbose output is available in the JSON output:
```
ozone admin container info 2 --json
{
"containerInfo" : {
"state" : "OPEN",
"replicationConfig" : {
"replicationFactor" : "THREE",
"requiredNodes" : 3,
"replicationType" : "RATIS"
},
"usedBytes" : 0,
"numberOfKeys" : 0,
"lastUsed" : "2022-01-17T11:47:04.997Z",
"stateEnterTime" : "2022-01-17T11:45:49.415Z",
"owner" : "om1",
"containerID" : 2,
"deleteTransactionId" : 0,
"sequenceId" : 702,
"open" : true,
"replicationType" : "RATIS",
"replicationFactor" : "THREE"
},
"pipeline" : {
<snipped existing pipeline>
},
"replicas" : [ {
"containerID" : 2,
"state" : "OPEN",
"datanodeDetails" : {
"level" : 0,
"cost" : 0,
"uuid" : "cd79c851-277a-4b89-8c6b-e1cc8f549ebc",
"uuidString" : "cd79c851-277a-4b89-8c6b-e1cc8f549ebc",
"ipAddress" : "172.28.0.7",
"hostName" : "ozone_datanode_1.ozone_default",
"ports" : [ {
"name" : "REPLICATION",
"value" : 9886
}, {
"name" : "RATIS",
"value" : 9858
}, {
"name" : "RATIS_ADMIN",
"value" : 9857
}, {
"name" : "RATIS_SERVER",
"value" : 9856
}, {
"name" : "STANDALONE",
"value" : 9859
} ],
"setupTime" : 0,
"persistedOpState" : "IN_SERVICE",
"persistedOpStateExpiryEpochSec" : 0,
"initialVersion" : 0,
"currentVersion" : 1,
"signature" : -1230887375,
"networkLocation" : "/default-rack",
"networkName" : "cd79c851-277a-4b89-8c6b-e1cc8f549ebc",
"networkFullPath" : "/default-rack/cd79c851-277a-4b89-8c6b-e1cc8f549ebc",
"numOfLeaves" : 1
},
"placeOfBirth" : "cd79c851-277a-4b89-8c6b-e1cc8f549ebc",
"sequenceId" : 0,
"keyCount" : 0,
"bytesUsed" : 0
}, {
"containerID" : 2,
"state" : "OPEN",
"datanodeDetails" : {
"level" : 0,
"cost" : 0,
"uuid" : "2de8adb0-504c-4890-8d68-cc699c02cf01",
"uuidString" : "2de8adb0-504c-4890-8d68-cc699c02cf01",
"ipAddress" : "172.28.0.8",
"hostName" : "ozone_datanode_2.ozone_default",
"ports" : [ {
"name" : "REPLICATION",
"value" : 9886
}, {
"name" : "RATIS",
"value" : 9858
}, {
"name" : "RATIS_ADMIN",
"value" : 9857
}, {
"name" : "RATIS_SERVER",
"value" : 9856
}, {
"name" : "STANDALONE",
"value" : 9859
} ],
"setupTime" : 0,
"persistedOpState" : "IN_SERVICE",
"persistedOpStateExpiryEpochSec" : 0,
"initialVersion" : 0,
"currentVersion" : 1,
"signature" : 771714490,
"networkLocation" : "/default-rack",
"networkName" : "2de8adb0-504c-4890-8d68-cc699c02cf01",
"networkFullPath" : "/default-rack/2de8adb0-504c-4890-8d68-cc699c02cf01",
"numOfLeaves" : 1
},
"placeOfBirth" : "2de8adb0-504c-4890-8d68-cc699c02cf01",
"sequenceId" : 702,
"keyCount" : 213,
"bytesUsed" : 2181120
}, {
"containerID" : 2,
"state" : "OPEN",
"datanodeDetails" : {
"level" : 0,
"cost" : 0,
"uuid" : "29b032e6-f76a-44a8-a582-427fc8a83aa9",
"uuidString" : "29b032e6-f76a-44a8-a582-427fc8a83aa9",
"ipAddress" : "172.28.0.6",
"hostName" : "ozone_datanode_3.ozone_default",
"ports" : [ {
"name" : "REPLICATION",
"value" : 9886
}, {
"name" : "RATIS",
"value" : 9858
}, {
"name" : "RATIS_ADMIN",
"value" : 9857
}, {
"name" : "RATIS_SERVER",
"value" : 9856
}, {
"name" : "STANDALONE",
"value" : 9859
} ],
"setupTime" : 0,
"persistedOpState" : "IN_SERVICE",
"persistedOpStateExpiryEpochSec" : 0,
"initialVersion" : 0,
"currentVersion" : 1,
"signature" : 1091981010,
"networkLocation" : "/default-rack",
"networkName" : "29b032e6-f76a-44a8-a582-427fc8a83aa9",
"networkFullPath" : "/default-rack/29b032e6-f76a-44a8-a582-427fc8a83aa9",
"numOfLeaves" : 1
},
"placeOfBirth" : "29b032e6-f76a-44a8-a582-427fc8a83aa9",
"sequenceId" : 702,
"keyCount" : 213,
"bytesUsed" : 2181120
} ]
}
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-6177
## How was this patch tested?
New unit tests and validated the new command via a docker-compose cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel merged pull request #2995: HDDS-6177. Extend container info command to include replica details
Posted by GitBox <gi...@apache.org>.
sodonnel merged pull request #2995:
URL: https://github.com/apache/ozone/pull/2995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #2995: HDDS-6177. Extend container info command to include replica details
Posted by GitBox <gi...@apache.org>.
sodonnel commented on pull request #2995:
URL: https://github.com/apache/ozone/pull/2995#issuecomment-1021058632
@adoroszlai I have addressed your comments - can you have another look and see if it looks OK? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #2995: HDDS-6177. Extend container info command to include replica details
Posted by GitBox <gi...@apache.org>.
sodonnel commented on pull request #2995:
URL: https://github.com/apache/ozone/pull/2995#issuecomment-1021058632
@adoroszlai I have addressed your comments - can you have another look and see if it looks OK? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #2995: HDDS-6177. Extend container info command to include replica details
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #2995:
URL: https://github.com/apache/ozone/pull/2995#discussion_r791172588
##########
File path: hadoop-hdds/tools/pom.xml
##########
@@ -83,5 +83,15 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd">
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
Review comment:
Nit: `<version>` is unnecessary here, defined in root POM.
##########
File path: hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaInfoTest.java
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.container;
+
+import org.apache.hadoop.hdds.protocol.DatanodeDetails;
+import org.apache.hadoop.hdds.protocol.MockDatanodeDetails;
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.UUID;
+
+/**
+ * Test for the ContainerReplicaInfo class.
+ */
+public class ContainerReplicaInfoTest {
Review comment:
Should be renamed to `TestContainerReplicaInfo` to be executed. (For some reason that's the naming convention for Ozone.)
https://github.com/apache/ozone/blob/2af225d0231474308ef1c0b22452ec3859c6e0a3/pom.xml#L1937
##########
File path: hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container/package-info.java
##########
@@ -0,0 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.container;
+/**
+ Test cases for SCM container client classes.
+ */
Review comment:
```suggestion
/**
Test cases for SCM container client classes.
*/
package org.apache.hadoop.hdds.scm.container;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #2995: HDDS-6177. Extend container info command to include replica details
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #2995:
URL: https://github.com/apache/ozone/pull/2995#discussion_r791172588
##########
File path: hadoop-hdds/tools/pom.xml
##########
@@ -83,5 +83,15 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd">
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
Review comment:
Nit: `<version>` is unnecessary here, defined in root POM.
##########
File path: hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaInfoTest.java
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.container;
+
+import org.apache.hadoop.hdds.protocol.DatanodeDetails;
+import org.apache.hadoop.hdds.protocol.MockDatanodeDetails;
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.UUID;
+
+/**
+ * Test for the ContainerReplicaInfo class.
+ */
+public class ContainerReplicaInfoTest {
Review comment:
Should be renamed to `TestContainerReplicaInfo` to be executed. (For some reason that's the naming convention for Ozone.)
https://github.com/apache/ozone/blob/2af225d0231474308ef1c0b22452ec3859c6e0a3/pom.xml#L1937
##########
File path: hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/scm/container/package-info.java
##########
@@ -0,0 +1,21 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.container;
+/**
+ Test cases for SCM container client classes.
+ */
Review comment:
```suggestion
/**
Test cases for SCM container client classes.
*/
package org.apache.hadoop.hdds.scm.container;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org