You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/12/08 22:58:27 UTC
[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6336: Segment reset API
mcvsubbu commented on a change in pull request #6336:
URL: https://github.com/apache/incubator-pinot/pull/6336#discussion_r538870662
##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
##########
@@ -1768,6 +1770,63 @@ public int reloadSegment(String tableNameWithType, String segmentName) {
return numMessagesSent;
}
+ /**
+ * Resets a segment by disabling and then enabling the segment
+ */
+ public void resetSegment(String tableNameWithType, String segmentName) {
+ IdealState idealState = getTableIdealState(tableNameWithType);
+ Preconditions.checkState(idealState != null, "Could not find ideal state for table: %s", tableNameWithType);
+ ExternalView externalView = getTableExternalView(tableNameWithType);
+ Preconditions.checkState(externalView != null, "Could not find external view for table: %s", tableNameWithType);
+ Set<String> instanceSet = idealState.getInstanceSet(segmentName);
+ Preconditions
+ .checkState(CollectionUtils.isNotEmpty(instanceSet), "Could not find segment: %s in ideal state for table: %s");
+ Map<String, String> externalViewStateMap = externalView.getStateMap(segmentName);
+ List<String> partitions = Lists.newArrayList(segmentName);
+ for (String instance : instanceSet) {
+ if (externalViewStateMap == null || SegmentStateModel.ERROR.equals(externalViewStateMap.get(instance))) {
+ _helixAdmin.resetPartition(_helixClusterName, instance, tableNameWithType, partitions);
Review comment:
Can you document what this API call does (or, is supposed to do)? Can it throw some exceptions that we need to catch (or not catch)?
##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
##########
@@ -1768,6 +1770,63 @@ public int reloadSegment(String tableNameWithType, String segmentName) {
return numMessagesSent;
}
+ /**
+ * Resets a segment by disabling and then enabling the segment
+ */
+ public void resetSegment(String tableNameWithType, String segmentName) {
+ IdealState idealState = getTableIdealState(tableNameWithType);
+ Preconditions.checkState(idealState != null, "Could not find ideal state for table: %s", tableNameWithType);
+ ExternalView externalView = getTableExternalView(tableNameWithType);
+ Preconditions.checkState(externalView != null, "Could not find external view for table: %s", tableNameWithType);
+ Set<String> instanceSet = idealState.getInstanceSet(segmentName);
+ Preconditions
+ .checkState(CollectionUtils.isNotEmpty(instanceSet), "Could not find segment: %s in ideal state for table: %s");
+ Map<String, String> externalViewStateMap = externalView.getStateMap(segmentName);
+ List<String> partitions = Lists.newArrayList(segmentName);
+ for (String instance : instanceSet) {
+ if (externalViewStateMap == null || SegmentStateModel.ERROR.equals(externalViewStateMap.get(instance))) {
+ _helixAdmin.resetPartition(_helixClusterName, instance, tableNameWithType, partitions);
+ } else {
+ _helixAdmin.enablePartition(false, _helixClusterName, instance, tableNameWithType, partitions);
+ }
+ _helixAdmin.enablePartition(true, _helixClusterName, instance, tableNameWithType, partitions);
Review comment:
Dont we need to wait until externalview stabiliizes before calling enable again?
##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
##########
@@ -1768,6 +1770,63 @@ public int reloadSegment(String tableNameWithType, String segmentName) {
return numMessagesSent;
}
+ /**
+ * Resets a segment by disabling and then enabling the segment
+ */
+ public void resetSegment(String tableNameWithType, String segmentName) {
+ IdealState idealState = getTableIdealState(tableNameWithType);
+ Preconditions.checkState(idealState != null, "Could not find ideal state for table: %s", tableNameWithType);
+ ExternalView externalView = getTableExternalView(tableNameWithType);
+ Preconditions.checkState(externalView != null, "Could not find external view for table: %s", tableNameWithType);
+ Set<String> instanceSet = idealState.getInstanceSet(segmentName);
+ Preconditions
+ .checkState(CollectionUtils.isNotEmpty(instanceSet), "Could not find segment: %s in ideal state for table: %s");
+ Map<String, String> externalViewStateMap = externalView.getStateMap(segmentName);
+ List<String> partitions = Lists.newArrayList(segmentName);
+ for (String instance : instanceSet) {
+ if (externalViewStateMap == null || SegmentStateModel.ERROR.equals(externalViewStateMap.get(instance))) {
+ _helixAdmin.resetPartition(_helixClusterName, instance, tableNameWithType, partitions);
+ } else {
+ _helixAdmin.enablePartition(false, _helixClusterName, instance, tableNameWithType, partitions);
+ }
Review comment:
I remember we used to have this disable/enable (aka toggle) in segments API before, we had a lot of problems with it. We used it for refreshing segments, and then we moved to use the refresh helix message instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org