You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "clarax (via GitHub)" <gi...@apache.org> on 2023/10/18 08:18:01 UTC
[PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
clarax opened a new pull request, #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685
## What is the purpose of the change
This is to implement the scaling report comparison algorithm for event suppression by using hashcodes of the parallelism maps. Originally I used the full string of report message which contains the metrics fluctuating with current load. As long as parallelism map doesn't change, we don't need to generate new events within the defined interval.
## Brief change log
- Save hashcode of the parallelism map to metadata labels
- Use the hashcode to compare two scaling report for advice
## Verifying this change
- Updated and added unit tests.
- Verified in integration test env.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): No
- The public API, i.e., is any changes to the `CustomResourceDescriptors`: No
- Core observer or reconciler logic that is regularly executed: No
## Documentation
- Does this pull request introduce a new feature? No
- If yes, how is the feature documented? N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#issuecomment-1772027448
Refactored as @gyfora suggusted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1366649482
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
I agree, we can keep the interval as a fixed parameter to the reporter methods. That way we don't need custom config handling in the operator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "1996fanrui (via GitHub)" <gi...@apache.org>.
1996fanrui commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1367619403
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
+ eventRecorder.triggerEvent(
+ context.getResource(),
+ EventRecorder.Type.valueOf(type.name()),
+ reason,
+ message,
+ EventRecorder.Component.Operator,
+ messageKey,
+ context.getKubernetesClient());
+ }
+
+ @Override
+ public void handleScalingEvent(
+ KubernetesJobAutoScalerContext context,
+ Map<JobVertexID, ScalingSummary> scalingSummaries,
+ boolean scaled,
+ @Nonnull Duration interval) {
+ if (scaled) {
+ AutoScalerEventHandler.super.handleScalingEvent(
+ context, scalingSummaries, scaled, interval);
} else {
- eventRecorder.triggerEventByInterval(
+ var conf = context.getConfiguration();
+ var scalingReport = AutoScalerEventHandler.scalingReport(scalingSummaries, scaled);
+ var labels = Map.of(PARALLELISM_MAP_KEY, getParallelismHashCode(scalingSummaries));
Review Comment:
Hi @clarax ,IIUC, the current strategy is: when the scaled is disabled, we want to remove duplicates based on `getParallelismHashCode(scalingSummaries)` and interval, right?
If yes, how about we don't change the `AutoScalerEventHandler` and consider `getParallelismHashCode(scalingSummaries)` as the messageKey?
And refactor ingthe `EventUtils`:
- Ignoring repeating message based on the `generatedMessageKey` and `interval`
- `generatedMessageKey = messageKey != null ? messageKey : message;`
- For existed event, when the {@link #MESSAGE_KEY} exists in label, it's messageKey, otherwise the message is messageKey.
Here is my POC commit[1], it maybe clearer. If the POC has the same effect as the current PR, I like this POC.
Because the interface is clearer and `AutoScalerEventHandler` doesn't care the logic about `handleScalingEvent`.
Please correct me if my understanding is wrong, thanks~
[1] https://github.com/1996fanrui/flink-kubernetes-operator/commit/14e8f189f4d129eba23050ea37fe8c012f0a22d2
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
Review Comment:
`interval` isn't used here, so it doesn't take effect.
From this implementation, the interface deserves to be discussed how to support interval capabilities more reasonably.
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/EventUtils.java:
##########
@@ -117,53 +119,53 @@ public static boolean createByInterval(
EventRecorder.Component component,
Consumer<Event> eventListener,
@Nullable String messageKey,
- Duration interval) {
-
+ @Nullable BiPredicate<Map<String, String>, Instant> suppressionPredicate,
+ @Nullable Map<String, String> labels) {
String eventName =
generateEventName(
target, type, reason, messageKey != null ? messageKey : message, component);
Event existing = findExistingEvent(client, target, eventName);
if (existing != null) {
- if (Objects.equals(existing.getMessage(), message)
- && Instant.now()
- .isBefore(
- Instant.parse(existing.getLastTimestamp())
- .plusMillis(interval.toMillis()))) {
- return false;
- } else {
- createUpdatedEvent(existing, client, message, eventListener);
+ if (suppressionPredicate != null
+ && existing.getMetadata() != null
+ && suppressionPredicate.test(
+ existing.getMetadata().getLabels(),
+ Instant.parse(existing.getLastTimestamp()))) {
return false;
}
+ updatedEventWithLabels(existing, client, message, eventListener, labels);
+ return false;
} else {
- createNewEvent(
- client, target, type, reason, message, component, eventListener, eventName);
+ Event event = buildEvent(target, type, reason, message, component, eventName);
+ setLabels(event, labels);
+ eventListener.accept(client.resource(event).createOrReplace());
return true;
}
}
- private static void createUpdatedEvent(
+ private static void updatedEventWithLabels(
Event existing,
KubernetesClient client,
String message,
- Consumer<Event> eventListener) {
+ Consumer<Event> eventListener,
+ @Nullable Map<String, String> labels) {
existing.setLastTimestamp(Instant.now().toString());
existing.setCount(existing.getCount() + 1);
existing.setMessage(message);
+ setLabels(existing, labels);
eventListener.accept(client.resource(existing).createOrReplace());
}
- private static void createNewEvent(
- KubernetesClient client,
- HasMetadata target,
- EventRecorder.Type type,
- String reason,
- String message,
- EventRecorder.Component component,
- Consumer<Event> eventListener,
- String eventName) {
- Event event = buildEvent(target, type, reason, message, component, eventName);
- eventListener.accept(client.resource(event).createOrReplace());
+ private static void setLabels(Event existing, @Nullable Map<String, String> labels) {
+ if (existing.getMetadata() == null) {
Review Comment:
If labels is null, is this `setLabels` necessary?
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/EventUtils.java:
##########
@@ -117,53 +119,53 @@ public static boolean createByInterval(
EventRecorder.Component component,
Consumer<Event> eventListener,
@Nullable String messageKey,
- Duration interval) {
-
+ @Nullable BiPredicate<Map<String, String>, Instant> suppressionPredicate,
+ @Nullable Map<String, String> labels) {
String eventName =
generateEventName(
target, type, reason, messageKey != null ? messageKey : message, component);
Event existing = findExistingEvent(client, target, eventName);
if (existing != null) {
- if (Objects.equals(existing.getMessage(), message)
- && Instant.now()
- .isBefore(
- Instant.parse(existing.getLastTimestamp())
- .plusMillis(interval.toMillis()))) {
- return false;
- } else {
- createUpdatedEvent(existing, client, message, eventListener);
+ if (suppressionPredicate != null
+ && existing.getMetadata() != null
+ && suppressionPredicate.test(
+ existing.getMetadata().getLabels(),
+ Instant.parse(existing.getLastTimestamp()))) {
return false;
}
+ updatedEventWithLabels(existing, client, message, eventListener, labels);
+ return false;
} else {
- createNewEvent(
- client, target, type, reason, message, component, eventListener, eventName);
+ Event event = buildEvent(target, type, reason, message, component, eventName);
+ setLabels(event, labels);
+ eventListener.accept(client.resource(event).createOrReplace());
return true;
}
}
- private static void createUpdatedEvent(
+ private static void updatedEventWithLabels(
Event existing,
KubernetesClient client,
String message,
- Consumer<Event> eventListener) {
+ Consumer<Event> eventListener,
+ @Nullable Map<String, String> labels) {
existing.setLastTimestamp(Instant.now().toString());
existing.setCount(existing.getCount() + 1);
existing.setMessage(message);
+ setLabels(existing, labels);
eventListener.accept(client.resource(existing).createOrReplace());
}
- private static void createNewEvent(
- KubernetesClient client,
- HasMetadata target,
- EventRecorder.Type type,
- String reason,
- String message,
- EventRecorder.Component component,
- Consumer<Event> eventListener,
- String eventName) {
- Event event = buildEvent(target, type, reason, message, component, eventName);
- eventListener.accept(client.resource(event).createOrReplace());
+ private static void setLabels(Event existing, @Nullable Map<String, String> labels) {
+ if (existing.getMetadata() == null) {
+ var metaData = new ObjectMeta();
+ metaData.setLabels(labels);
+ } else if (existing.getMetadata().getLabels() == null) {
+ existing.getMetadata().setLabels(labels);
+ } else {
+ existing.getMetadata().setLabels(labels);
+ }
Review Comment:
```suggestion
} else {
existing.getMetadata().setLabels(labels);
}
```
Same behavior for different code conditions, they can be simplified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1370455045
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java:
##########
@@ -201,8 +201,8 @@ private static ConfigOptions.OptionBuilder autoScalerConfig(String key) {
.withDescription(
"A (semicolon-separated) list of vertex ids in hexstring for which to disable scaling. Caution: For non-sink vertices this will still scale their downstream operators until https://issues.apache.org/jira/browse/FLINK-31215 is implemented.");
- public static final ConfigOption<Duration> SCALING_REPORT_INTERVAL =
- autoScalerConfig("scaling.report.interval")
+ public static final ConfigOption<Duration> SCALING_EVENT_INTERVAL =
+ autoScalerConfig("scaling.event.interval")
.durationType()
.defaultValue(Duration.ofSeconds(1800))
.withDescription("Time interval to resend the identical event");
Review Comment:
> As this comment[1] mentioned: `all config keys to work with the "old" syntax at least in the 1.7.0 release.`, so please update it, thanks!
>
> Sorry for that, I didn't support the old key during decoupling autoscaler and kubernetes-operator.
>
> [#686 (comment)](https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1369777698)
updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "1996fanrui (via GitHub)" <gi...@apache.org>.
1996fanrui commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368185629
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
+ eventRecorder.triggerEvent(
+ context.getResource(),
+ EventRecorder.Type.valueOf(type.name()),
+ reason,
+ message,
+ EventRecorder.Component.Operator,
+ messageKey,
+ context.getKubernetesClient());
+ }
+
+ @Override
+ public void handleScalingEvent(
+ KubernetesJobAutoScalerContext context,
+ Map<JobVertexID, ScalingSummary> scalingSummaries,
+ boolean scaled,
+ @Nonnull Duration interval) {
+ if (scaled) {
+ AutoScalerEventHandler.super.handleScalingEvent(
+ context, scalingSummaries, scaled, interval);
} else {
- eventRecorder.triggerEventByInterval(
+ var conf = context.getConfiguration();
+ var scalingReport = AutoScalerEventHandler.scalingReport(scalingSummaries, scaled);
+ var labels = Map.of(PARALLELISM_MAP_KEY, getParallelismHashCode(scalingSummaries));
Review Comment:
Hi @gyfora , thanks for your explanation!
> you set a new key every time the parallelism changes but only when scaling is off
I set it based on the logic of current PR. Do you mean my POC logic isn't exactly same with the current PR? I may have some misunderstand here.
FYI: I do this POC due to I want to design an interface for support the kubernetes requirement, and let it as generic as possible. If design a generic interface is very hard here, the current design is good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368195931
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
Review Comment:
Hi Fanrui, KubernetesAutoScalerEventHandler doesn't support interval based dedupe. As discussed, we'd like to keep it in the interface method signature but the actually implementation can choose to ignore it. It is possible we expand the support for those event too as we need for scaling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#issuecomment-1771652868
Sorry @clarax I think I gave confusing feedback and didn't express myself clearly. I was thinking something like this:
```
interface AutoScalerEventHandler {
void handleGenericEvent(Ctx, Type, Reson, Message, Key, Interval);
default void handleScalingEvent(Ctx, Map<JobVertexId, ScalingSummary> summaries, boolean scalingEnabled, Interval) {
// Provide default implementation without proper deduplication
handleGenericEvent(...., toMessage(summaries), interval);
}
}
```
Then for Kubernetes we have the custom label logic encapsulated:
```
class KubernetesAutoScalerEventHandler {
...
@Override
void handleScalingEvent(Ctx, Map<JobVertexId, ScalingSummary> summaries, boolean scalingEnabled, Interval) {
var labels = createLabels(summaries)
getOldEvent, compare labels, whatever we need to do we do not leak to ScalingExecutor
}
}
```
This way the ScalingExecutor simply calls:
```
eventHandler.handleScalingEvent(...)
```
And we could remove the predicate/label logic that really do not belong there as that is very Kubernetes specific and move it to the `KubernetesAutoScalerEventHandler` implementation.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368208203
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -32,20 +40,69 @@
*/
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
+ String SCALING_SUMMARY_ENTRY =
+ " Vertex ID %s | Parallelism %d -> %d | Processing capacity %.2f -> %.2f | Target data rate %.2f";
+ String SCALING_SUMMARY_HEADER_SCALING_DISABLED = "Recommended parallelism change:";
+ String SCALING_SUMMARY_HEADER_SCALING_ENABLED = "Scaling vertices:";
+ String SCALING_REPORT_REASON = "ScalingReport";
+ String EVENT_MESSAGE_KEY = "ScalingExecutor";
Review Comment:
This should be renamed to `SCALING_REPORT_KEY`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "1996fanrui (via GitHub)" <gi...@apache.org>.
1996fanrui commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1369808629
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java:
##########
@@ -201,8 +201,8 @@ private static ConfigOptions.OptionBuilder autoScalerConfig(String key) {
.withDescription(
"A (semicolon-separated) list of vertex ids in hexstring for which to disable scaling. Caution: For non-sink vertices this will still scale their downstream operators until https://issues.apache.org/jira/browse/FLINK-31215 is implemented.");
- public static final ConfigOption<Duration> SCALING_REPORT_INTERVAL =
- autoScalerConfig("scaling.report.interval")
+ public static final ConfigOption<Duration> SCALING_EVENT_INTERVAL =
+ autoScalerConfig("scaling.event.interval")
.durationType()
.defaultValue(Duration.ofSeconds(1800))
.withDescription("Time interval to resend the identical event");
Review Comment:
```suggestion
.withDeprecatedKeys(
deprecatedOperatorConfigKey(
"scaling.event.interval"))
.withDescription("Time interval to resend the identical event");
```
As this comment[1] mentioned: ` all config keys to work with the "old" syntax at least in the 1.7.0 release. `, so please update it, thanks!
Sorry for that, I didn't support the old key during decoupling autoscaler and kubernetes-operator.
https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1369777698
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1365657154
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
And we could even have a default implementation within the interface itself to turn it into a generic event in case we are not running on the operator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1365742349
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
I don't understand the suggestion @clarax what do you think my proposal? Have those 2 methods in the interface and add a default implementation for `handleScalingEvent`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1366039222
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
Updated for two methods. I don't understand "a default implementation within the interface itself to turn it into a generic event in case we are not running on the operator". I made the second method a generic method for any suppression callers want to use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "1996fanrui (via GitHub)" <gi...@apache.org>.
1996fanrui commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368211844
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
+ eventRecorder.triggerEvent(
+ context.getResource(),
+ EventRecorder.Type.valueOf(type.name()),
+ reason,
+ message,
+ EventRecorder.Component.Operator,
+ messageKey,
+ context.getKubernetesClient());
+ }
+
+ @Override
+ public void handleScalingEvent(
+ KubernetesJobAutoScalerContext context,
+ Map<JobVertexID, ScalingSummary> scalingSummaries,
+ boolean scaled,
+ @Nonnull Duration interval) {
+ if (scaled) {
+ AutoScalerEventHandler.super.handleScalingEvent(
+ context, scalingSummaries, scaled, interval);
} else {
- eventRecorder.triggerEventByInterval(
+ var conf = context.getConfiguration();
+ var scalingReport = AutoScalerEventHandler.scalingReport(scalingSummaries, scaled);
+ var labels = Map.of(PARALLELISM_MAP_KEY, getParallelismHashCode(scalingSummaries));
Review Comment:
Thanks for the clarification!
Sounds good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#issuecomment-1776429332
> Another thing we have been discussing with @1996fanrui
>
> The interval config should probably be renamed `scaling.report.interval->scaling.event.interval` this way we can use it generally in the future for autoscaler triggered events.
>
> We should also make sure that the simple `handleEvent` method also respects the interval if specified. And we should probably use the interval also for ineffective scaling events. I know that some of these changes are not directly related to this PR but it may be better to clean it up so we leave it in a good state afterwards.
Resolved all requested changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1369655435
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java:
##########
@@ -201,8 +201,8 @@ private static ConfigOptions.OptionBuilder autoScalerConfig(String key) {
.withDescription(
"A (semicolon-separated) list of vertex ids in hexstring for which to disable scaling. Caution: For non-sink vertices this will still scale their downstream operators until https://issues.apache.org/jira/browse/FLINK-31215 is implemented.");
- public static final ConfigOption<Duration> SCALING_REPORT_INTERVAL =
- autoScalerConfig("scaling.report.interval")
+ public static final ConfigOption<Duration> SCALING_EVENT_INTERVAL =
+ autoScalerConfig("scaling.event.interval")
Review Comment:
Docs seem to be inconsistent with this and should regenerated after renaming to `event.interval`
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/EventUtils.java:
##########
@@ -197,4 +198,19 @@ private static Event buildEvent(
.endMetadata()
.build();
}
+
+ private static boolean intervalCheck(Event existing, @Nullable Duration interval) {
+ return interval != null
+ && Instant.now()
+ .isBefore(
+ Instant.parse(existing.getLastTimestamp())
+ .plusMillis(interval.toMillis()));
+ }
+
+ private static boolean labelCheck(
+ Event existing, Predicate<Map<String, String>> dedupePredicate) {
+ return dedupePredicate == null
+ || (existing.getMetadata() != null
+ && dedupePredicate.test(existing.getMetadata().getLabels()));
+ }
Review Comment:
I may misunderstand something but seems like labels are basically ignored when the `interval == null` . In that case intervalCheck is always false. Is this intentional?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1370421339
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/EventUtils.java:
##########
@@ -197,4 +198,19 @@ private static Event buildEvent(
.endMetadata()
.build();
}
+
+ private static boolean intervalCheck(Event existing, @Nullable Duration interval) {
+ return interval != null
+ && Instant.now()
+ .isBefore(
+ Instant.parse(existing.getLastTimestamp())
+ .plusMillis(interval.toMillis()));
+ }
+
+ private static boolean labelCheck(
+ Event existing, Predicate<Map<String, String>> dedupePredicate) {
+ return dedupePredicate == null
+ || (existing.getMetadata() != null
+ && dedupePredicate.test(existing.getMetadata().getLabels()));
+ }
Review Comment:
makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368147631
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
+ eventRecorder.triggerEvent(
+ context.getResource(),
+ EventRecorder.Type.valueOf(type.name()),
+ reason,
+ message,
+ EventRecorder.Component.Operator,
+ messageKey,
+ context.getKubernetesClient());
+ }
+
+ @Override
+ public void handleScalingEvent(
+ KubernetesJobAutoScalerContext context,
+ Map<JobVertexID, ScalingSummary> scalingSummaries,
+ boolean scaled,
+ @Nonnull Duration interval) {
+ if (scaled) {
+ AutoScalerEventHandler.super.handleScalingEvent(
+ context, scalingSummaries, scaled, interval);
} else {
- eventRecorder.triggerEventByInterval(
+ var conf = context.getConfiguration();
+ var scalingReport = AutoScalerEventHandler.scalingReport(scalingSummaries, scaled);
+ var labels = Map.of(PARALLELISM_MAP_KEY, getParallelismHashCode(scalingSummaries));
Review Comment:
@1996fanrui The problem with your approach is that it breaks the messageKey logic for scaling events (you set a new key every time the parallelism changes but only when scaling is off)
The general problem with the current interface is that it already contains many Kubernetes specific parts (messageKey) but not enough to properly implement new functionality (such as the one we are trying here to deduplicate events based on certain logic, like the actual parallelism overrides)
In other interfaces like the StateStore we decided to go with specific methods, and I think that's what we should do here as well. I prefer @clarax s solution as it allows us complete flexibility in the Kubernetes implementation without pushing anything on to the interface.
To give a bit more background on the `messageKey` , we use that to define the specific event when triggering in Kubernetes such that we keep the history (last triggered, etc) without always creating new event objects. This is completely independent of the interval / parallelism based triggering that are trying to do here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora merged PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax closed pull request #685: [FLINK-33187] using hashcode for parallelism map comparison
URL: https://github.com/apache/flink-kubernetes-operator/pull/685
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1365727935
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
Thank you for calling out. I was not sure if I should have a separate method, not sure what this handler interfaceshould be handing. We can move the logic of handleGenericAutoscalerEven to the interface and make it an abstract class. What do you think? @gyfora
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "1996fanrui (via GitHub)" <gi...@apache.org>.
1996fanrui commented on PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#issuecomment-1776540163
Hi, the ci fails, and please run the `mvn clean install -DskipTests -Pgenerate-docs` again, thanks
https://github.com/apache/flink-kubernetes-operator/actions/runs/6621898931/job/17987047236?pr=685#step:5:19405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1367157112
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
Interval is needed for all AutoScalerHandler. The reason to remove it from parameter list is context contains config which defines the interval. I think this is the advantage using a context to wrap up all info to pass so we don't need to use so many parameters in the method. The same reason we don't need to pass scalingEnabled in handleScalingEvent. Thank you for constructing the context during refactoring. @1996fanrui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1367164471
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
I still think it's better to pass Interval / scalingEnabled directly. They are part of the intended behaviour of the method. If we don't pass it, we rely on implementers to figure out how exactly to get this from the config. It would be super easy to miss and come up with an incorrect implementation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1367373231
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
Updated the PR with the change. Added comments to explain too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "1996fanrui (via GitHub)" <gi...@apache.org>.
1996fanrui commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368068093
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
+ eventRecorder.triggerEvent(
+ context.getResource(),
+ EventRecorder.Type.valueOf(type.name()),
+ reason,
+ message,
+ EventRecorder.Component.Operator,
+ messageKey,
+ context.getKubernetesClient());
+ }
+
+ @Override
+ public void handleScalingEvent(
+ KubernetesJobAutoScalerContext context,
+ Map<JobVertexID, ScalingSummary> scalingSummaries,
+ boolean scaled,
+ @Nonnull Duration interval) {
+ if (scaled) {
+ AutoScalerEventHandler.super.handleScalingEvent(
+ context, scalingSummaries, scaled, interval);
} else {
- eventRecorder.triggerEventByInterval(
+ var conf = context.getConfiguration();
+ var scalingReport = AutoScalerEventHandler.scalingReport(scalingSummaries, scaled);
+ var labels = Map.of(PARALLELISM_MAP_KEY, getParallelismHashCode(scalingSummaries));
Review Comment:
cc @gyfora
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368209456
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
+ eventRecorder.triggerEvent(
+ context.getResource(),
+ EventRecorder.Type.valueOf(type.name()),
+ reason,
+ message,
+ EventRecorder.Component.Operator,
+ messageKey,
+ context.getKubernetesClient());
+ }
+
+ @Override
+ public void handleScalingEvent(
+ KubernetesJobAutoScalerContext context,
+ Map<JobVertexID, ScalingSummary> scalingSummaries,
+ boolean scaled,
+ @Nonnull Duration interval) {
+ if (scaled) {
+ AutoScalerEventHandler.super.handleScalingEvent(
+ context, scalingSummaries, scaled, interval);
} else {
- eventRecorder.triggerEventByInterval(
+ var conf = context.getConfiguration();
+ var scalingReport = AutoScalerEventHandler.scalingReport(scalingSummaries, scaled);
+ var labels = Map.of(PARALLELISM_MAP_KEY, getParallelismHashCode(scalingSummaries));
Review Comment:
Hi @1996fanrui, Thanks for the proposed alternative, just adding to @gyfora 's comment,
"To give a bit more background on the messageKey , we use that to define the specific event when triggering in Kubernetes such that we keep the history (last triggered, etc) without always creating new event objects. This is completely independent of the interval / parallelism based triggering that are trying to do here.
"
Overriding messageKey will totally break the design of using messageKey to keep history of certain type of events at minimal cost. We keep using the same Kubernetes event object and just need to increment the count of the object and updating the messsages. The dedupe logic by interval is independent where we don't even update the kubenetes event object or trigger any operator events. The behaviors of your POC is different from this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368208991
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/autoscaler/KubernetesAutoScalerEventHandler.java:
##########
@@ -43,26 +52,71 @@ public void handleEvent(
String reason,
String message,
@Nullable String messageKey,
- @Nullable Duration interval) {
- if (interval == null) {
- eventRecorder.triggerEvent(
- context.getResource(),
- EventRecorder.Type.valueOf(type.name()),
- reason,
- message,
- EventRecorder.Component.Operator,
- messageKey,
- context.getKubernetesClient());
+ @Nonnull Duration interval) {
+ eventRecorder.triggerEvent(
+ context.getResource(),
+ EventRecorder.Type.valueOf(type.name()),
+ reason,
+ message,
+ EventRecorder.Component.Operator,
+ messageKey,
+ context.getKubernetesClient());
+ }
+
+ @Override
+ public void handleScalingEvent(
+ KubernetesJobAutoScalerContext context,
+ Map<JobVertexID, ScalingSummary> scalingSummaries,
+ boolean scaled,
+ @Nonnull Duration interval) {
+ if (scaled) {
+ AutoScalerEventHandler.super.handleScalingEvent(
+ context, scalingSummaries, scaled, interval);
} else {
- eventRecorder.triggerEventByInterval(
+ var conf = context.getConfiguration();
+ var scalingReport = AutoScalerEventHandler.scalingReport(scalingSummaries, scaled);
+ var labels = Map.of(PARALLELISM_MAP_KEY, getParallelismHashCode(scalingSummaries));
Review Comment:
@1996fanrui in Claras PR the message key is fixed: `ScalingExecutor` in your POC PR the message key is derived from the parallelism, so it changes all the time. They are not the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1370420200
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/EventUtils.java:
##########
@@ -197,4 +198,19 @@ private static Event buildEvent(
.endMetadata()
.build();
}
+
+ private static boolean intervalCheck(Event existing, @Nullable Duration interval) {
+ return interval != null
+ && Instant.now()
+ .isBefore(
+ Instant.parse(existing.getLastTimestamp())
+ .plusMillis(interval.toMillis()));
+ }
+
+ private static boolean labelCheck(
+ Event existing, Predicate<Map<String, String>> dedupePredicate) {
+ return dedupePredicate == null
+ || (existing.getMetadata() != null
+ && dedupePredicate.test(existing.getMetadata().getLabels()));
+ }
Review Comment:
when Interval == null, we don't dedupe. This is intentional. It is commented in the interface method param and unit tested.
but when the config is null, we use the default value of 30 min. It is also documented and unit tested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1370421148
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java:
##########
@@ -201,8 +201,8 @@ private static ConfigOptions.OptionBuilder autoScalerConfig(String key) {
.withDescription(
"A (semicolon-separated) list of vertex ids in hexstring for which to disable scaling. Caution: For non-sink vertices this will still scale their downstream operators until https://issues.apache.org/jira/browse/FLINK-31215 is implemented.");
- public static final ConfigOption<Duration> SCALING_REPORT_INTERVAL =
- autoScalerConfig("scaling.report.interval")
+ public static final ConfigOption<Duration> SCALING_EVENT_INTERVAL =
+ autoScalerConfig("scaling.event.interval")
Review Comment:
updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "1996fanrui (via GitHub)" <gi...@apache.org>.
1996fanrui commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1366621112
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
The reason for removing the interval from this interface is because interval just works for kubernetesEventHandler, right?
I wanna whether it's needed for Standalone Autoscaler, if it's needed, we should keep the `interval` here.
Actually, I prefer keep it first. And removing it if we are sure it's really not needed in the future. WDYT? cc @gyfora
Also, if the interval is not needed here. `AutoScalerOptions#SCALING_REPORT_INTERVAL` should be moved into `flink-kubernetes-operator` module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1365655692
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -33,19 +35,15 @@
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
- /**
- * Handle the event.
- *
- * @param interval When interval is great than 0, events that repeat within the interval will be
- * ignored.
- */
+ /** Handle the event. */
Review Comment:
I think the entire eventHandler interface is getting pretty weird. It was already not very nice before this change but now it's even worse because we are leaking too many internals of the operator event triggering mechanism.
I think we need to break up the handleEvent into two new methods.
```
// Would be the current logic but without the interval logic
handleGenericAutoscalerEvent(...)
// Handle scaling events separately. Then the operator can hide the complex interval/label triggering logic
handleScalingEvent(Context, Map<JobVertexID, ScalingSummary> scalingSummaries, scalingEnabled)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368208203
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -32,20 +40,69 @@
*/
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
+ String SCALING_SUMMARY_ENTRY =
+ " Vertex ID %s | Parallelism %d -> %d | Processing capacity %.2f -> %.2f | Target data rate %.2f";
+ String SCALING_SUMMARY_HEADER_SCALING_DISABLED = "Recommended parallelism change:";
+ String SCALING_SUMMARY_HEADER_SCALING_ENABLED = "Scaling vertices:";
+ String SCALING_REPORT_REASON = "ScalingReport";
+ String EVENT_MESSAGE_KEY = "ScalingExecutor";
Review Comment:
This should be renamed to `SCALING_MESSAGE_KEY`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368211917
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java:
##########
@@ -32,20 +40,69 @@
*/
@Experimental
public interface AutoScalerEventHandler<KEY, Context extends JobAutoScalerContext<KEY>> {
+ String SCALING_SUMMARY_ENTRY =
+ " Vertex ID %s | Parallelism %d -> %d | Processing capacity %.2f -> %.2f | Target data rate %.2f";
+ String SCALING_SUMMARY_HEADER_SCALING_DISABLED = "Recommended parallelism change:";
+ String SCALING_SUMMARY_HEADER_SCALING_ENABLED = "Scaling vertices:";
+ String SCALING_REPORT_REASON = "ScalingReport";
+ String EVENT_MESSAGE_KEY = "ScalingExecutor";
Review Comment:
will update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368183966
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/EventUtils.java:
##########
@@ -117,53 +119,53 @@ public static boolean createByInterval(
EventRecorder.Component component,
Consumer<Event> eventListener,
@Nullable String messageKey,
- Duration interval) {
-
+ @Nullable BiPredicate<Map<String, String>, Instant> suppressionPredicate,
+ @Nullable Map<String, String> labels) {
String eventName =
generateEventName(
target, type, reason, messageKey != null ? messageKey : message, component);
Event existing = findExistingEvent(client, target, eventName);
if (existing != null) {
- if (Objects.equals(existing.getMessage(), message)
- && Instant.now()
- .isBefore(
- Instant.parse(existing.getLastTimestamp())
- .plusMillis(interval.toMillis()))) {
- return false;
- } else {
- createUpdatedEvent(existing, client, message, eventListener);
+ if (suppressionPredicate != null
+ && existing.getMetadata() != null
+ && suppressionPredicate.test(
+ existing.getMetadata().getLabels(),
+ Instant.parse(existing.getLastTimestamp()))) {
return false;
}
+ updatedEventWithLabels(existing, client, message, eventListener, labels);
+ return false;
} else {
- createNewEvent(
- client, target, type, reason, message, component, eventListener, eventName);
+ Event event = buildEvent(target, type, reason, message, component, eventName);
+ setLabels(event, labels);
+ eventListener.accept(client.resource(event).createOrReplace());
return true;
}
}
- private static void createUpdatedEvent(
+ private static void updatedEventWithLabels(
Event existing,
KubernetesClient client,
String message,
- Consumer<Event> eventListener) {
+ Consumer<Event> eventListener,
+ @Nullable Map<String, String> labels) {
existing.setLastTimestamp(Instant.now().toString());
existing.setCount(existing.getCount() + 1);
existing.setMessage(message);
+ setLabels(existing, labels);
eventListener.accept(client.resource(existing).createOrReplace());
}
- private static void createNewEvent(
- KubernetesClient client,
- HasMetadata target,
- EventRecorder.Type type,
- String reason,
- String message,
- EventRecorder.Component component,
- Consumer<Event> eventListener,
- String eventName) {
- Event event = buildEvent(target, type, reason, message, component, eventName);
- eventListener.accept(client.resource(event).createOrReplace());
+ private static void setLabels(Event existing, @Nullable Map<String, String> labels) {
+ if (existing.getMetadata() == null) {
+ var metaData = new ObjectMeta();
+ metaData.setLabels(labels);
+ } else if (existing.getMetadata().getLabels() == null) {
+ existing.getMetadata().setLabels(labels);
+ } else {
+ existing.getMetadata().setLabels(labels);
+ }
Review Comment:
Good catch. Will update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]
Posted by "clarax (via GitHub)" <gi...@apache.org>.
clarax commented on code in PR #685:
URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1368187442
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/EventUtils.java:
##########
@@ -117,53 +119,53 @@ public static boolean createByInterval(
EventRecorder.Component component,
Consumer<Event> eventListener,
@Nullable String messageKey,
- Duration interval) {
-
+ @Nullable BiPredicate<Map<String, String>, Instant> suppressionPredicate,
+ @Nullable Map<String, String> labels) {
String eventName =
generateEventName(
target, type, reason, messageKey != null ? messageKey : message, component);
Event existing = findExistingEvent(client, target, eventName);
if (existing != null) {
- if (Objects.equals(existing.getMessage(), message)
- && Instant.now()
- .isBefore(
- Instant.parse(existing.getLastTimestamp())
- .plusMillis(interval.toMillis()))) {
- return false;
- } else {
- createUpdatedEvent(existing, client, message, eventListener);
+ if (suppressionPredicate != null
+ && existing.getMetadata() != null
+ && suppressionPredicate.test(
+ existing.getMetadata().getLabels(),
+ Instant.parse(existing.getLastTimestamp()))) {
return false;
}
+ updatedEventWithLabels(existing, client, message, eventListener, labels);
+ return false;
} else {
- createNewEvent(
- client, target, type, reason, message, component, eventListener, eventName);
+ Event event = buildEvent(target, type, reason, message, component, eventName);
+ setLabels(event, labels);
+ eventListener.accept(client.resource(event).createOrReplace());
return true;
}
}
- private static void createUpdatedEvent(
+ private static void updatedEventWithLabels(
Event existing,
KubernetesClient client,
String message,
- Consumer<Event> eventListener) {
+ Consumer<Event> eventListener,
+ @Nullable Map<String, String> labels) {
existing.setLastTimestamp(Instant.now().toString());
existing.setCount(existing.getCount() + 1);
existing.setMessage(message);
+ setLabels(existing, labels);
eventListener.accept(client.resource(existing).createOrReplace());
}
- private static void createNewEvent(
- KubernetesClient client,
- HasMetadata target,
- EventRecorder.Type type,
- String reason,
- String message,
- EventRecorder.Component component,
- Consumer<Event> eventListener,
- String eventName) {
- Event event = buildEvent(target, type, reason, message, component, eventName);
- eventListener.accept(client.resource(event).createOrReplace());
+ private static void setLabels(Event existing, @Nullable Map<String, String> labels) {
+ if (existing.getMetadata() == null) {
Review Comment:
Currently we don't call with non null value. But it doesn't hurt to allow caller to set it to null as needed. It will be considered as different labels from empty map so we don't accidentally dedupe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org