You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "zhtaoxiang (via GitHub)" <gi...@apache.org> on 2023/06/02 17:11:53 UTC
[GitHub] [pinot] zhtaoxiang commented on a diff in pull request #10812: Enhancing Table Size API to return table size per replica
zhtaoxiang commented on code in PR #10812:
URL: https://github.com/apache/pinot/pull/10812#discussion_r1214592610
##########
pinot-controller/src/main/java/org/apache/pinot/controller/util/TableSizeReader.java:
##########
@@ -132,12 +144,19 @@ public TableSizeDetails getTableSizeDetails(@Nonnull String tableName, @Nonnegat
}
}
if (largestSegmentSizeOnServer != DEFAULT_SIZE_WHEN_MISSING_OR_ERROR) {
- _controllerMetrics.setValueOfTableGauge(offlineTableName,
- ControllerGauge.LARGEST_SEGMENT_SIZE_ON_SERVER,
+ _controllerMetrics.setValueOfTableGauge(offlineTableName, ControllerGauge.LARGEST_SEGMENT_SIZE_ON_SERVER,
largestSegmentSizeOnServer);
}
}
+ // Set the top level sizes to DEFAULT_SIZE_WHEN_MISSING_OR_ERROR when all segments are error
Review Comment:
Can you please help me understand why the condition is "when all segments are error". What is the major difference between "when all segments are error" and "when some segments are in error state"?
Say that a table has 1k segments, the current logic set the top level size to -1 when all 1k segments are in error state, but will not set the level size to -1 when only 1 segment is in good state. What is the major difference here?
##########
pinot-controller/src/main/java/org/apache/pinot/controller/util/TableSizeReader.java:
##########
@@ -132,12 +144,19 @@ public TableSizeDetails getTableSizeDetails(@Nonnull String tableName, @Nonnegat
}
}
if (largestSegmentSizeOnServer != DEFAULT_SIZE_WHEN_MISSING_OR_ERROR) {
- _controllerMetrics.setValueOfTableGauge(offlineTableName,
- ControllerGauge.LARGEST_SEGMENT_SIZE_ON_SERVER,
+ _controllerMetrics.setValueOfTableGauge(offlineTableName, ControllerGauge.LARGEST_SEGMENT_SIZE_ON_SERVER,
largestSegmentSizeOnServer);
}
}
+ // Set the top level sizes to DEFAULT_SIZE_WHEN_MISSING_OR_ERROR when all segments are error
+ if ((hasRealtimeTableConfig && hasOfflineTableConfig && isMissingAllRealtimeSegments && isMissingAllOfflineSegments)
Review Comment:
1. see the following table, with or without the first condition, the result does not change. Please check.
<img width="1549" alt="Screenshot 2023-06-02 at 10 08 38 AM" src="https://github.com/apache/pinot/assets/9796617/93b38569-79f0-4e13-b049-897a334db3c7">
2. see my reply above
##########
pinot-controller/src/main/java/org/apache/pinot/controller/util/TableSizeReader.java:
##########
@@ -248,7 +279,7 @@ public TableSubTypeSizeDetails getTableSubtypeSize(String tableNameWithType, int
String segment = entry.getKey();
SegmentSizeDetails sizeDetails = entry.getValue();
// Iterate over all segment size info, update reported size, track max segment size and number of errored servers
- long segmentLevelMax = -1L;
+ long segmentLevelMax = DEFAULT_SIZE_WHEN_MISSING_OR_ERROR;
Review Comment:
I agree.
It's just a minor suggestion to simplify the code a little bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org