You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "targetoee (via GitHub)" <gi...@apache.org> on 2023/04/19 08:22:15 UTC
[GitHub] [ozone] targetoee opened a new pull request, #4591: add transitional container count in recon
targetoee opened a new pull request, #4591:
URL: https://github.com/apache/ozone/pull/4591
## What changes were proposed in this pull request?
This PR adds additional container information to Recon, which shows the total count of containers in some states, including CLOSING, QUASI_CLOSED, CLOSED, and DELETING.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8411
## How was this patch tested?
Unit test
<img width="1511" alt="Screen Shot 2023-04-19 at 4 20 44 PM" src="https://user-images.githubusercontent.com/107598572/233014278-84a77a52-4fbf-4ea8-aca9-cb04d7eb5176.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] krishnaasawa1 commented on a diff in pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
Posted by "krishnaasawa1 (via GitHub)" <gi...@apache.org>.
krishnaasawa1 commented on code in PR #4591:
URL: https://github.com/apache/ozone/pull/4591#discussion_r1183491863
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -180,6 +196,12 @@ public Response getClusterState() {
.setTotalDatanodes(datanodeDetails.size())
.setHealthyDatanodes(healthyDatanodes)
.setOpenContainers(containerStateCounts.getOpenContainersCount())
+ .setClosingContainers(containerStateCounts.getClosingContainersCount())
+ .setQuasiClosedContainers(
+ containerStateCounts.getQuasiClosedContainersCount())
+ .setClosedContainers(containerStateCounts.getClosedContainersCount())
Review Comment:
Yes in normal scenarios transition will be quick, but have seen large number of containers stuck and remaining in Closing state in some scenarios. So should be useful for analyzing, if easily visible. But we should be careful of cluttering overview page and should have at appropriate place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] devmadhuu commented on a diff in pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #4591:
URL: https://github.com/apache/ozone/pull/4591#discussion_r1172335722
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -180,6 +196,12 @@ public Response getClusterState() {
.setTotalDatanodes(datanodeDetails.size())
.setHealthyDatanodes(healthyDatanodes)
.setOpenContainers(containerStateCounts.getOpenContainersCount())
+ .setClosingContainers(containerStateCounts.getClosingContainersCount())
+ .setQuasiClosedContainers(
+ containerStateCounts.getQuasiClosedContainersCount())
+ .setClosedContainers(containerStateCounts.getClosedContainersCount())
Review Comment:
This may not be needed, as difference of total and open will give us closed container count. Rest all information on container states can be easily found out using Ozone admin CLI because we don't want to expose the transitioning container states to other users, and should be exposed to Admin users only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ArafatKhan2198 commented on a diff in pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
Posted by "ArafatKhan2198 (via GitHub)" <gi...@apache.org>.
ArafatKhan2198 commented on code in PR #4591:
URL: https://github.com/apache/ozone/pull/4591#discussion_r1172832577
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -180,6 +196,12 @@ public Response getClusterState() {
.setTotalDatanodes(datanodeDetails.size())
.setHealthyDatanodes(healthyDatanodes)
.setOpenContainers(containerStateCounts.getOpenContainersCount())
+ .setClosingContainers(containerStateCounts.getClosingContainersCount())
+ .setQuasiClosedContainers(
+ containerStateCounts.getQuasiClosedContainersCount())
+ .setClosedContainers(containerStateCounts.getClosedContainersCount())
Review Comment:
Hi @devmadhuu
@sumitagrawl has suggested that we include additional states in our state counts, such as close, closing, and quasi_closed. The state closing is especially important and can provide valuable insights into the situation being analyzed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] devmadhuu commented on a diff in pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #4591:
URL: https://github.com/apache/ozone/pull/4591#discussion_r1172340883
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -180,6 +196,12 @@ public Response getClusterState() {
.setTotalDatanodes(datanodeDetails.size())
.setHealthyDatanodes(healthyDatanodes)
.setOpenContainers(containerStateCounts.getOpenContainersCount())
+ .setClosingContainers(containerStateCounts.getClosingContainersCount())
+ .setQuasiClosedContainers(
+ containerStateCounts.getQuasiClosedContainersCount())
+ .setClosedContainers(containerStateCounts.getClosedContainersCount())
+ .setDeletingContainers(
+ containerStateCounts.getDeletingContainersCount())
.setDeletedContainers(containerStateCounts.getDeletedContainersCount())
Review Comment:
This might be duplicate of PR for [HDDS-8127](https://github.com/apache/ozone/pull/4440)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] devmadhuu commented on pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on PR #4591:
URL: https://github.com/apache/ozone/pull/4591#issuecomment-1516015885
@targetoee - Thanks for working on this patch, however transitioning of container states could be very fast in a busy cluster, and not sure how much this info will be useful to show CLOSING, QUASI_CLOSED and DELETING containers count. We show OPEN container count already and total container count, so difference will give us CLOSED container count.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
URL: https://github.com/apache/ozone/pull/4591
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #4591:
URL: https://github.com/apache/ozone/pull/4591#issuecomment-1926006577
Thank you very much for the patch. I am closing this PR __temporarily__ as there was no activity recently and it is waiting for response from its author.
It doesn't mean that this PR is not important or ignored: feel free to reopen the PR at any time.
It only means that attention of committers is not required. We prefer to keep the review queue clean. This ensures PRs in need of review are more visible, which results in faster feedback for all PRs.
If you need ANY help to finish this PR, please [contact the community](https://github.com/apache/hadoop-ozone#contact) on the mailing list or the slack channel."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4591:
URL: https://github.com/apache/ozone/pull/4591#issuecomment-1890980390
/pending
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] devmadhuu commented on a diff in pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #4591:
URL: https://github.com/apache/ozone/pull/4591#discussion_r1172843837
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -180,6 +196,12 @@ public Response getClusterState() {
.setTotalDatanodes(datanodeDetails.size())
.setHealthyDatanodes(healthyDatanodes)
.setOpenContainers(containerStateCounts.getOpenContainersCount())
+ .setClosingContainers(containerStateCounts.getClosingContainersCount())
+ .setQuasiClosedContainers(
+ containerStateCounts.getQuasiClosedContainersCount())
+ .setClosedContainers(containerStateCounts.getClosedContainersCount())
Review Comment:
@ArafatKhan2198 , these state transitions are too dynamic and quick, let's discuss this, not sure if in so dynamic transitions, if it is really useful for admins. Moreover these are directly queried using admin CLI, so better we should avoid such data and load recon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] devmadhuu commented on a diff in pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #4591:
URL: https://github.com/apache/ozone/pull/4591#discussion_r1172844613
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -180,6 +196,12 @@ public Response getClusterState() {
.setTotalDatanodes(datanodeDetails.size())
.setHealthyDatanodes(healthyDatanodes)
.setOpenContainers(containerStateCounts.getOpenContainersCount())
+ .setClosingContainers(containerStateCounts.getClosingContainersCount())
+ .setQuasiClosedContainers(
+ containerStateCounts.getQuasiClosedContainersCount())
+ .setClosedContainers(containerStateCounts.getClosedContainersCount())
Review Comment:
Also closed is not needed because it can be derived from difference
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sumitagrawl commented on a diff in pull request #4591: HDDS-8411. Show the total number of CLOSING, QUASI_CLOSED, CLOSED, DELETING Containers in Recon
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #4591:
URL: https://github.com/apache/ozone/pull/4591#discussion_r1173389987
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -180,6 +196,12 @@ public Response getClusterState() {
.setTotalDatanodes(datanodeDetails.size())
.setHealthyDatanodes(healthyDatanodes)
.setOpenContainers(containerStateCounts.getOpenContainersCount())
+ .setClosingContainers(containerStateCounts.getClosingContainersCount())
+ .setQuasiClosedContainers(
+ containerStateCounts.getQuasiClosedContainersCount())
+ .setClosedContainers(containerStateCounts.getClosedContainersCount())
Review Comment:
IMO, Quasi-Closed can be important state having special meaning. Also Closing / Deleting if stuck can be identified easily from this perspective.
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ClusterStateEndpoint.java:
##########
@@ -180,6 +196,12 @@ public Response getClusterState() {
.setTotalDatanodes(datanodeDetails.size())
.setHealthyDatanodes(healthyDatanodes)
.setOpenContainers(containerStateCounts.getOpenContainersCount())
+ .setClosingContainers(containerStateCounts.getClosingContainersCount())
+ .setQuasiClosedContainers(
+ containerStateCounts.getQuasiClosedContainersCount())
+ .setClosedContainers(containerStateCounts.getClosedContainersCount())
+ .setDeletingContainers(
+ containerStateCounts.getDeletingContainersCount())
.setDeletedContainers(containerStateCounts.getDeletedContainersCount())
Review Comment:
Deleted Container and Deleting container shows different info ... and not to be excluded from container count. Only Deleted Container needs to be excluded,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org