You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "devmadhuu (via GitHub)" <gi...@apache.org> on 2023/05/30 06:11:38 UTC
[GitHub] [ozone] devmadhuu commented on a diff in pull request #4791: HDDS-8697 Recon - Expose API for total count for blocks pending for deletion.
devmadhuu commented on code in PR #4791:
URL: https://github.com/apache/ozone/pull/4791#discussion_r1209748391
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/BlocksEndPoint.java:
##########
@@ -68,41 +68,66 @@ public BlocksEndPoint(ReconStorageContainerManagerFacade reconSCM) {
}
/**
- * This API returns list of blocks grouped by container state
+ * This API returns the total count of blocks pending deletion and a list of
+ * blocks grouped by container state
* (OPEN/CLOSING/CLOSED).
+ *
+ * Example of response:
* {
- * "OPEN": [
- * {
- * "containerId": 100,
- * "localIDList": [
- * 1,
- * 2,
- * 3,
- * 4
- * ],
- * "localIDCount": 4,
- * "txID": 1
- * }
- * ]
+ * "totalCount": 1000,
+ * "containerStateBlockInfoListMap": {
+ * "OPEN": [
+ * {
+ * "containerId": 100,
+ * "localIDList": [
+ * 1,
+ * 2,
+ * 3,
+ * 4
+ * ],
+ * "localIDCount": 4,
+ * "txID": 1
+ * }
+ * ]
+ * }
* }
+ *
* @param limit limits the number of records having list of blocks
* grouped by container state (OPEN/CLOSING/CLOSED)
* @param prevKey deletedBlocks table key to skip records before prevKey
- * @return list of blocks grouped by container state (OPEN/CLOSING/CLOSED)
+ * @return a Response object containing total count of blocks pending deletion
+ * and list of blocks grouped by container state (OPEN/CLOSING/CLOSED)
*/
@GET
@Path("/deletePending")
public Response getBlocksPendingDeletion(
@DefaultValue(DEFAULT_FETCH_COUNT) @QueryParam(RECON_QUERY_LIMIT)
- int limit,
+ int limit,
@DefaultValue(PREV_DELETED_BLOCKS_TRANSACTION_ID_DEFAULT_VALUE)
@QueryParam(RECON_QUERY_PREVKEY) long prevKey) {
if (limit < 0 || prevKey < 0) {
// Send back an empty response
return Response.status(Response.Status.NOT_ACCEPTABLE).build();
}
+
+ // Create a response map to hold totalCount and containerStateList
+ Map<String, Object> response = new HashMap<>();
+
Map<String, List<ContainerBlocksInfoWrapper>>
containerStateBlockInfoListMap = new HashMap<>();
+
+ try (
+ Table<Long,
+ StorageContainerDatanodeProtocolProtos.DeletedBlocksTransaction>
+ deletedBlocksTable = DELETED_BLOCKS.getTable(this.scmDBStore)) {
+ long totalCount = deletedBlocksTable.getEstimatedKeyCount();
Review Comment:
This doesn't give correct count. We should not use this method.
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/BlocksEndPoint.java:
##########
@@ -68,41 +68,66 @@ public BlocksEndPoint(ReconStorageContainerManagerFacade reconSCM) {
}
/**
- * This API returns list of blocks grouped by container state
+ * This API returns the total count of blocks pending deletion and a list of
+ * blocks grouped by container state
* (OPEN/CLOSING/CLOSED).
+ *
+ * Example of response:
* {
- * "OPEN": [
- * {
- * "containerId": 100,
- * "localIDList": [
- * 1,
- * 2,
- * 3,
- * 4
- * ],
- * "localIDCount": 4,
- * "txID": 1
- * }
- * ]
+ * "totalCount": 1000,
+ * "containerStateBlockInfoListMap": {
Review Comment:
@ArafatKhan2198 thanks for working on this patch, We need total count and unreplicated and replicated size mapped to those blocks. You should provide a new API, below API is only based on pagination.
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/BlocksEndPoint.java:
##########
@@ -68,41 +68,66 @@ public BlocksEndPoint(ReconStorageContainerManagerFacade reconSCM) {
}
/**
- * This API returns list of blocks grouped by container state
+ * This API returns the total count of blocks pending deletion and a list of
+ * blocks grouped by container state
* (OPEN/CLOSING/CLOSED).
+ *
+ * Example of response:
* {
- * "OPEN": [
- * {
- * "containerId": 100,
- * "localIDList": [
- * 1,
- * 2,
- * 3,
- * 4
- * ],
- * "localIDCount": 4,
- * "txID": 1
- * }
- * ]
+ * "totalCount": 1000,
+ * "containerStateBlockInfoListMap": {
+ * "OPEN": [
+ * {
+ * "containerId": 100,
+ * "localIDList": [
+ * 1,
+ * 2,
+ * 3,
+ * 4
+ * ],
+ * "localIDCount": 4,
+ * "txID": 1
+ * }
+ * ]
+ * }
* }
+ *
* @param limit limits the number of records having list of blocks
* grouped by container state (OPEN/CLOSING/CLOSED)
* @param prevKey deletedBlocks table key to skip records before prevKey
- * @return list of blocks grouped by container state (OPEN/CLOSING/CLOSED)
+ * @return a Response object containing total count of blocks pending deletion
+ * and list of blocks grouped by container state (OPEN/CLOSING/CLOSED)
*/
@GET
@Path("/deletePending")
public Response getBlocksPendingDeletion(
@DefaultValue(DEFAULT_FETCH_COUNT) @QueryParam(RECON_QUERY_LIMIT)
- int limit,
+ int limit,
@DefaultValue(PREV_DELETED_BLOCKS_TRANSACTION_ID_DEFAULT_VALUE)
@QueryParam(RECON_QUERY_PREVKEY) long prevKey) {
if (limit < 0 || prevKey < 0) {
// Send back an empty response
return Response.status(Response.Status.NOT_ACCEPTABLE).build();
}
+
+ // Create a response map to hold totalCount and containerStateList
+ Map<String, Object> response = new HashMap<>();
+
Map<String, List<ContainerBlocksInfoWrapper>>
containerStateBlockInfoListMap = new HashMap<>();
+
+ try (
+ Table<Long,
+ StorageContainerDatanodeProtocolProtos.DeletedBlocksTransaction>
+ deletedBlocksTable = DELETED_BLOCKS.getTable(this.scmDBStore)) {
Review Comment:
Pls don't change in existing API. This API should work based on pagination params only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org