You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2021/04/28 00:08:51 UTC
[GitHub] [helix] dasahcc commented on a change in pull request #1714: [Replica Level Throttle] Change throttling logic to per message
dasahcc commented on a change in pull request #1714:
URL: https://github.com/apache/helix/pull/1714#discussion_r621693298
##########
File path: helix-core/src/main/java/org/apache/helix/controller/stages/IntermediateStateCalcStage.java
##########
@@ -631,104 +631,53 @@ private void chargePendingTransition(Resource resource, CurrentStateOutput curre
}
/**
- * Check the status on throttling at every level (cluster, resource, instance) and set
- * intermediatePartitionStateMap accordingly per partition.
- * @param throttleController
- * @param resourceName
- * @param partition
- * @param currentStateOutput
- * @param bestPossiblePartitionStateMap
- * @param partitionsThrottled
- * @param intermediatePartitionStateMap
- * @param rebalanceType
- * @param cache
+ * Check the status for a single message on throttling at every level (cluster, resource, replica) and set
+ * intermediatePartitionStateMap accordingly for that replica.
+ * @param throttleController throttle controller object for throttling quota
+ * @param resourceName the resource for throttling check
+ * @param partition the partition for throttling check
+ * @param messageToThrottle the message to throttle
+ * @param messagesThrottled messages have been throttled
+ * @param intermediatePartitionStateMap output for this stage of intermediate map
+ * @param rebalanceType the rebalance type to charge quota
+ * @param cache cache for controller required metadata
*/
- private void throttleStateTransitionsForPartition(
- StateTransitionThrottleController throttleController, String resourceName,
- Partition partition, CurrentStateOutput currentStateOutput,
- PartitionStateMap bestPossiblePartitionStateMap, Set<Partition> partitionsThrottled,
+ private void throttleStateTransitionsForReplica(StateTransitionThrottleController throttleController,
+ String resourceName, Partition partition, Message messageToThrottle, Set<Message> messagesThrottled,
PartitionStateMap intermediatePartitionStateMap, RebalanceType rebalanceType,
ResourceControllerDataProvider cache) {
-
- Map<String, String> currentStateMap =
- currentStateOutput.getCurrentStateMap(resourceName, partition);
- Map<String, String> bestPossibleMap = bestPossiblePartitionStateMap.getPartitionMap(partition);
- Set<String> allInstances = new HashSet<>(currentStateMap.keySet());
- allInstances.addAll(bestPossibleMap.keySet());
- Map<String, String> intermediateMap = new HashMap<>();
-
boolean hasReachedThrottlingLimit = false;
if (throttleController.shouldThrottleForResource(rebalanceType, resourceName)) {
hasReachedThrottlingLimit = true;
if (logger.isDebugEnabled()) {
- LogUtil.logDebug(logger, _eventId,
- String.format("Throttled on partition: %s in resource: %s",
- partition.getPartitionName(), resourceName));
+ LogUtil.logDebug(logger, _eventId, String.format(
+ "Throttled because of cluster/resource quota is full for message {%s} on partition {%s} in resource {%s}",
+ messageToThrottle.getId(), partition.getPartitionName(), resourceName));
}
} else {
- // throttle if any of the instances are not able to accept state transitions
- for (String instance : allInstances) {
- String currentState = currentStateMap.get(instance);
- String bestPossibleState = bestPossibleMap.get(instance);
- if (bestPossibleState != null && !bestPossibleState.equals(currentState)
- && !cache.getDisabledInstancesForPartition(resourceName, partition.getPartitionName())
- .contains(instance)) {
- if (throttleController.shouldThrottleForInstance(rebalanceType, instance)) {
- hasReachedThrottlingLimit = true;
- if (logger.isDebugEnabled()) {
- LogUtil.logDebug(logger, _eventId,
- String.format(
- "Throttled because of instance: %s for partition: %s in resource: %s",
- instance, partition.getPartitionName(), resourceName));
- }
- break;
+ // Since message already generated, we can assume the current state is not null and target state is not null
+ if (!cache.getDisabledInstancesForPartition(resourceName, partition.getPartitionName())
+ .contains(messageToThrottle.getTgtName())) {
+ if (throttleController.shouldThrottleForInstance(rebalanceType, messageToThrottle.getTgtName())) {
+ hasReachedThrottlingLimit = true;
+ if (logger.isDebugEnabled()) {
+ LogUtil.logDebug(logger, _eventId, String.format(
+ "Throttled because of instance level quota is full on instance {%s} for message {%s} of partition {%s} in resource {%s}",
+ messageToThrottle.getId(), messageToThrottle.getTgtName(), partition.getPartitionName(), resourceName));
}
}
}
}
+ // If there is a room to hold existing messages, charge this message.
Review comment:
For your question:
1. Yes, even this function is called more times than single partition, there is no cost operations, such as new object creation / deletion, data collection looping. The only operations are read/update the map in throttle controllers and intermediate state map.
2. Usually you will have 3x as the one partition for throttling. Because a lot of messages have dependencies, which are removed by MessageSelectionStage. From historical data in production log, you can see that the number of messages is far less than what we have now.
Finally, as I said, this is non avoidable operation. Because each of the throttling charge operation success or fail can lead different result of next message rebalance type. The only thing we can do is trying to make sure no cost operations for this simple function call and that is also the reason I am trying to let this function to be reviewed independently instead of combing it into a big logic code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org