You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2026/10/01 20:28:41 UTC
[PR] [SPARK-59931] Simplify the K8s event messages of errors and `SchedulingFailure` [spark-kubernetes-operator]
dongjoon-hyun opened a new pull request, #920:
URL: https://github.com/apache/spark-kubernetes-operator/pull/920
### What changes were proposed in this pull request?
This PR aims to simplify the K8s event messages of errors and `SchedulingFailure`.
- `EventUtils.describe` uses the message of the API server for a rejected request, like the K8s built-in controllers, instead of the fabric8 message with the request URL and the `Status` dump. It also no longer repeats a cause which reads the same.
- The `SchedulingFailure` event describes the failure instead of carrying the stack trace. The resource status keeps the stack trace.
- `docs/configuration.md` documents the event messages.
### Why are the changes needed?
K8s events reach more readers than the resource status, e.g. via the built-in `view` ClusterRole and event exporters. Stack traces and request URLs are not actionable for them, and they push the actual cause out of the 1024-character event message.
### Does this PR introduce _any_ user-facing change?
No, because K8s events have not been released yet. The resource status and the operator log are unchanged.
### How was this patch tested?
Pass the CIs with the newly added and updated test cases.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59931] Simplify the K8s event messages of errors and `SchedulingFailure` [spark-kubernetes-operator]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #920:
URL: https://github.com/apache/spark-kubernetes-operator/pull/920#issuecomment-5948313885
Merged to main
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59931] Simplify the K8s event messages of errors and `SchedulingFailure` [spark-kubernetes-operator]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #920: [SPARK-59931] Simplify the K8s event messages of errors and `SchedulingFailure`
URL: https://github.com/apache/spark-kubernetes-operator/pull/920
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59931] Simplify the K8s event messages of errors and `SchedulingFailure` [spark-kubernetes-operator]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #920:
URL: https://github.com/apache/spark-kubernetes-operator/pull/920#discussion_r4164102167
##########
spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/EventUtils.java:
##########
@@ -203,16 +210,24 @@ public static String describe(Throwable throwable) {
if (throwable == null) {
return "";
}
- StringBuilder builder = new StringBuilder(describeSingle(throwable));
+ String description = describeSingle(throwable);
Throwable rootCause = rootCauseOf(throwable);
- if (rootCause != null) {
- builder.append(", caused by: ").append(describeSingle(rootCause));
+ if (rootCause == null) {
+ return description;
}
- return builder.toString();
+ String rootCauseDescription = describeSingle(rootCause);
+ return rootCauseDescription.equals(description)
+ ? description
+ : description + ", caused by: " + rootCauseDescription;
}
private static String describeSingle(Throwable throwable) {
String message = throwable.getMessage();
+ if (throwable instanceof KubernetesClientException e
+ && e.getStatus() != null
+ && StringUtils.isNotBlank(e.getStatus().getMessage())) {
+ message = e.getStatus().getMessage();
+ }
String type = throwable.getClass().getSimpleName();
return StringUtils.isBlank(message) ? type : type + ": " + message;
}
Review Comment:
Thank you, @peter-toth. Right, `waitForResult` wraps a cause which is neither an `IOException` nor a `KubernetesClientException` with the same message, and the vertx `NoStackTraceTimeoutException` is a `TimeoutException`. I applied your suggestion with a test case. One nit: the strict read treats a timeout as an absent resource, so a timeout ends in `SchedulingFailure` only after the create retries of `getOrCreateSecondaryResource` run out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59931] Simplify the K8s event messages of errors and `SchedulingFailure` [spark-kubernetes-operator]
Posted by "peter-toth (via GitHub)" <gi...@apache.org>.
peter-toth commented on code in PR #920:
URL: https://github.com/apache/spark-kubernetes-operator/pull/920#discussion_r4163637223
##########
spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/EventUtils.java:
##########
@@ -203,16 +210,24 @@ public static String describe(Throwable throwable) {
if (throwable == null) {
return "";
}
- StringBuilder builder = new StringBuilder(describeSingle(throwable));
+ String description = describeSingle(throwable);
Throwable rootCause = rootCauseOf(throwable);
- if (rootCause != null) {
- builder.append(", caused by: ").append(describeSingle(rootCause));
+ if (rootCause == null) {
+ return description;
}
- return builder.toString();
+ String rootCauseDescription = describeSingle(rootCause);
+ return rootCauseDescription.equals(description)
+ ? description
+ : description + ", caused by: " + rootCauseDescription;
}
private static String describeSingle(Throwable throwable) {
String message = throwable.getMessage();
+ if (throwable instanceof KubernetesClientException e
+ && e.getStatus() != null
+ && StringUtils.isNotBlank(e.getStatus().getMessage())) {
+ message = e.getStatus().getMessage();
+ }
String type = throwable.getClass().getSimpleName();
return StringUtils.isBlank(message) ? type : type + ": " + message;
}
Review Comment:
**Finding 1.** The dedupe compares `Type: message`, so it misses a wrapper that repeats its cause's message under another type. A request timeout is that case. `OperationSupport.waitForResult` rethrows the vertx `NoStackTraceTimeoutException` as `new KubernetesClientException(t.getMessage(), t)`, with no `Status`. I sent a POST to the mock server with a 1 s request timeout and a 3 s delay, and `describe` returned:
```
KubernetesClientException: The timeout period of 1000ms has been exceeded while executing POST /api/v1/namespaces/ns-1/configmaps for server localhost:50411, caused by: NoStackTraceTimeoutException: The timeout period of 1000ms has been exceeded while executing POST /api/v1/namespaces/ns-1/configmaps for server localhost:50411
```
So the request URL shows up twice. The transient filter keeps this out of most warnings. `ReconcileError` and `CleanupError` don't filter, though. A timeout can also end in `SchedulingFailure` through the strict read in `getOrCreateSecondaryResource`.
Comparing only the messages fixes it. With the change below, the same probe gives one copy, and a 403 and a non-`Status` 503 read as before. `EventUtilsTest` stays green.
```suggestion
String message = messageOf(throwable);
String description = describeSingle(throwable, message);
Throwable rootCause = rootCauseOf(throwable);
if (rootCause == null) {
return description;
}
String rootCauseMessage = messageOf(rootCause);
// A wrapper which repeats the message of its cause, e.g. the one fabric8 throws for a request
// timeout, adds nothing but its type.
if (StringUtils.isNotBlank(message) && message.equals(rootCauseMessage)) {
return description;
}
return description + ", caused by: " + describeSingle(rootCause, rootCauseMessage);
}
private static String messageOf(Throwable throwable) {
if (throwable instanceof KubernetesClientException e
&& e.getStatus() != null
&& StringUtils.isNotBlank(e.getStatus().getMessage())) {
return e.getStatus().getMessage();
}
return throwable.getMessage();
}
private static String describeSingle(Throwable throwable, String message) {
String type = throwable.getClass().getSimpleName();
return StringUtils.isBlank(message) ? type : type + ": " + message;
```
The javadoc's "unless it reads the same as the exception" would then become "unless its message is the same". A test could pin it with `new KubernetesClientException(msg, new TimeoutException(msg))`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org