You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/01/24 16:08:09 UTC
[GitHub] [ozone] sodonnel opened a new pull request #3013: HDDS-6205. Add CLI command to display the latest Replication Manager report
sodonnel opened a new pull request #3013:
URL: https://github.com/apache/ozone/pull/3013
## What changes were proposed in this pull request?
In HDDS-6170 I added metrics about container health to ReplicationManager, including a report which replication manager updates each time it runs.
This Jira is to add a command to the CLI, allowing the report to be retrieved and displayed to a user.
The new command is:
```
ozone admin container report
```
The output looks like:
```
Container Summary Report generated at 2022-01-24T14:58:23Z
==========================================================
Container State Summary
=======================
OPEN: 1
CLOSING: 2
QUASI_CLOSED: 1
CLOSED: 6
DELETING: 0
DELETED: 0
Container Health Summary
========================
UNDER_REPLICATED: 4
MIS_REPLICATED: 2
OVER_REPLICATED: 0
MISSING: 2
UNHEALTHY: 0
EMPTY: 0
OPEN_UNHEALTHY: 0
QUASI_CLOSED_STUCK: 1
First 100 UNDER_REPLICATED containers:
#2, #4, #5, #9
First 100 MIS_REPLICATED containers:
#2, #4
First 100 MISSING containers:
#2, #4
First 100 QUASI_CLOSED_STUCK containers:
#9
```
Note that this report is not generated on-the-fly. It is created each time ReplicationManager wakes up and runs, and the report is stored until the next run completes. The command is therefore very light weight, and simply receives and object and displays it to the user.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-6205
## How was this patch tested?
New unit test to validate the command output.
Manual testing via a docker-compose cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on a change in pull request #3013: HDDS-6205. Add CLI command to display the latest Replication Manager report
Posted by GitBox <gi...@apache.org>.
sodonnel commented on a change in pull request #3013:
URL: https://github.com/apache/ozone/pull/3013#discussion_r794436460
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
##########
@@ -732,6 +733,12 @@ public boolean getReplicationManagerStatus() {
return scm.getReplicationManager().isRunning();
}
+ @Override
+ public ReplicationManagerReport getReplicationManagerReport() {
+ AUDIT.logWriteSuccess(buildAuditMessageForSuccess(
+ SCMAction.GET_REPLICATION_MANAGER_REPORT, null));
+ return scm.getReplicationManager().getContainerReport();
+ }
Review comment:
OK - I have pushed a new commit to make it an admin operation. The existing operations seem a bit inconsistent on what requires admin privs or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #3013: HDDS-6205. Add CLI command to display the latest Replication Manager report
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #3013:
URL: https://github.com/apache/ozone/pull/3013#discussion_r794006640
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
##########
@@ -732,6 +733,12 @@ public boolean getReplicationManagerStatus() {
return scm.getReplicationManager().isRunning();
}
+ @Override
+ public ReplicationManagerReport getReplicationManagerReport() {
+ AUDIT.logWriteSuccess(buildAuditMessageForSuccess(
+ SCMAction.GET_REPLICATION_MANAGER_REPORT, null));
+ return scm.getReplicationManager().getContainerReport();
+ }
Review comment:
> I don't know if it needs to be.
I don't know either. But it's easier to start out with more restriction and relax it later, than to realize we should have limited access in the previous release. ;)
What do you think @swagle?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel merged pull request #3013: HDDS-6205. Add CLI command to display the latest Replication Manager report
Posted by GitBox <gi...@apache.org>.
sodonnel merged pull request #3013:
URL: https://github.com/apache/ozone/pull/3013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] swagle commented on a change in pull request #3013: HDDS-6205. Add CLI command to display the latest Replication Manager report
Posted by GitBox <gi...@apache.org>.
swagle commented on a change in pull request #3013:
URL: https://github.com/apache/ozone/pull/3013#discussion_r794020085
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
##########
@@ -732,6 +733,12 @@ public boolean getReplicationManagerStatus() {
return scm.getReplicationManager().isRunning();
}
+ @Override
+ public ReplicationManagerReport getReplicationManagerReport() {
+ AUDIT.logWriteSuccess(buildAuditMessageForSuccess(
+ SCMAction.GET_REPLICATION_MANAGER_REPORT, null));
+ return scm.getReplicationManager().getContainerReport();
+ }
Review comment:
Looking at this from CLI point of view, status generally has the same privilege as start/stop because it might be accessing resources like opening fds or checking ports for example to determine status. Although we do not need to do such things, use expectation is that it is privileged IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on a change in pull request #3013: HDDS-6205. Add CLI command to display the latest Replication Manager report
Posted by GitBox <gi...@apache.org>.
sodonnel commented on a change in pull request #3013:
URL: https://github.com/apache/ozone/pull/3013#discussion_r793074612
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
##########
@@ -732,6 +733,12 @@ public boolean getReplicationManagerStatus() {
return scm.getReplicationManager().isRunning();
}
+ @Override
+ public ReplicationManagerReport getReplicationManagerReport() {
+ AUDIT.logWriteSuccess(buildAuditMessageForSuccess(
+ SCMAction.GET_REPLICATION_MANAGER_REPORT, null));
+ return scm.getReplicationManager().getContainerReport();
+ }
Review comment:
I'm not sure. The information provided by the report is not sensitive - just a cluster health report and it is not resource intensive to produce it.
Looking at the other operations in that same protocol server.
start/stop replication manager are admin ops. Viewing its status is not an admin op.
Start / stop balance is an admin op. Viewing its status is not.
I am happy to make this one an admin op, but as it is read only and not performance intensive so I don't know if it needs to be.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a change in pull request #3013: HDDS-6205. Add CLI command to display the latest Replication Manager report
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #3013:
URL: https://github.com/apache/ozone/pull/3013#discussion_r793060805
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
##########
@@ -732,6 +733,12 @@ public boolean getReplicationManagerStatus() {
return scm.getReplicationManager().isRunning();
}
+ @Override
+ public ReplicationManagerReport getReplicationManagerReport() {
+ AUDIT.logWriteSuccess(buildAuditMessageForSuccess(
+ SCMAction.GET_REPLICATION_MANAGER_REPORT, null));
+ return scm.getReplicationManager().getContainerReport();
+ }
Review comment:
Shouldn't we require admin privilege for this operation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] swagle commented on a change in pull request #3013: HDDS-6205. Add CLI command to display the latest Replication Manager report
Posted by GitBox <gi...@apache.org>.
swagle commented on a change in pull request #3013:
URL: https://github.com/apache/ozone/pull/3013#discussion_r794020085
##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
##########
@@ -732,6 +733,12 @@ public boolean getReplicationManagerStatus() {
return scm.getReplicationManager().isRunning();
}
+ @Override
+ public ReplicationManagerReport getReplicationManagerReport() {
+ AUDIT.logWriteSuccess(buildAuditMessageForSuccess(
+ SCMAction.GET_REPLICATION_MANAGER_REPORT, null));
+ return scm.getReplicationManager().getContainerReport();
+ }
Review comment:
Looking at this from CLI point of view, status generally has the same privilege as start/stop because it might be accessing resources like opening fds or checking ports for example to determine status. Although we do not need to do such things here, user expectation is that it is privileged IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org