You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "gyfora (via GitHub)" <gi...@apache.org> on 2024/07/07 07:02:21 UTC
[PR] [FLINK-35776] Simplify job status handling [flink-kubernetes-operator]
gyfora opened a new pull request, #851:
URL: https://github.com/apache/flink-kubernetes-operator/pull/851
## What is the purpose of the change
There is a fairly complicated listing / observe logic for jobs currently that is no longer necessary as we have a stable logic to always record the jobID in the status before submission.
## Brief change log
- Replace complex resource specific job listing/matching logic with JobId based querying
- Clean up unnecessary codepaths
- Tests
## Verifying this change
The change is already covered by existing tests to a large extent.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changes to the `CustomResourceDescriptors`: no
- Core observer or reconciler logic that is regularly executed: yes
## Documentation
- Does this pull request introduce a new feature? no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35776] Simplify job status handling [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora merged PR #851:
URL: https://github.com/apache/flink-kubernetes-operator/pull/851
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35776] Simplify job status handling [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on code in PR #851:
URL: https://github.com/apache/flink-kubernetes-operator/pull/851#discussion_r1677082378
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/service/AbstractFlinkService.java:
##########
@@ -263,14 +268,24 @@ protected SocketAddress getSocketAddress(RestClusterClient<String> clusterClient
}
@Override
- public Collection<JobStatusMessage> listJobs(Configuration conf) throws Exception {
+ public Optional<JobStatusMessage> getJobStatus(Configuration conf, JobID jobId)
+ throws Exception {
try (var clusterClient = getClusterClient(conf)) {
return clusterClient
.sendRequest(
JobsOverviewHeaders.getInstance(),
EmptyMessageParameters.getInstance(),
EmptyRequestBody.getInstance())
- .thenApply(JobStatusUtils::toJobStatusMessage)
+ .thenApply(
+ mjd -> {
+ if (mjd.getJobs() == null) {
+ return Optional.<JobStatusMessage>empty();
+ }
+ return mjd.getJobs().stream()
+ .filter(jd -> jd.getJobId().equals(jobId))
+ .findAny()
+ .map(JobStatusUtils::toJobStatusMessage);
+ })
Review Comment:
I tried this @ferenc-csaky however the spotless line length limit breaks the lines a way that the chained optional is basically unreadable. I would prefer to keep it this way for readability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35776] Simplify job status handling [flink-kubernetes-operator]
Posted by "afedulov (via GitHub)" <gi...@apache.org>.
afedulov commented on PR #851:
URL: https://github.com/apache/flink-kubernetes-operator/pull/851#issuecomment-2228232857
@gyfora
> as we have a stable logic to always record the jobID in the status before submission.
could you point to the related changes to understand the context a bit better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35776] Simplify job status handling [flink-kubernetes-operator]
Posted by "ferenc-csaky (via GitHub)" <gi...@apache.org>.
ferenc-csaky commented on code in PR #851:
URL: https://github.com/apache/flink-kubernetes-operator/pull/851#discussion_r1668491036
##########
flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/service/AbstractFlinkService.java:
##########
@@ -263,14 +268,24 @@ protected SocketAddress getSocketAddress(RestClusterClient<String> clusterClient
}
@Override
- public Collection<JobStatusMessage> listJobs(Configuration conf) throws Exception {
+ public Optional<JobStatusMessage> getJobStatus(Configuration conf, JobID jobId)
+ throws Exception {
try (var clusterClient = getClusterClient(conf)) {
return clusterClient
.sendRequest(
JobsOverviewHeaders.getInstance(),
EmptyMessageParameters.getInstance(),
EmptyRequestBody.getInstance())
- .thenApply(JobStatusUtils::toJobStatusMessage)
+ .thenApply(
+ mjd -> {
+ if (mjd.getJobs() == null) {
+ return Optional.<JobStatusMessage>empty();
+ }
+ return mjd.getJobs().stream()
+ .filter(jd -> jd.getJobId().equals(jobId))
+ .findAny()
+ .map(JobStatusUtils::toJobStatusMessage);
+ })
Review Comment:
nit: Possibility of eliminate the null-check and do it in 1 command:
```java
.thenApply(
mjd ->
Optional.ofNullable(mjd.getJobs())
.orElse(Collections.emptyList())
.stream()
.filter(jd -> jd.getJobId().equals(jobId))
.findAny()
.map(JobStatusUtils::toJobStatusMessage))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35776] Simplify job status handling [flink-kubernetes-operator]
Posted by "gyfora (via GitHub)" <gi...@apache.org>.
gyfora commented on PR #851:
URL: https://github.com/apache/flink-kubernetes-operator/pull/851#issuecomment-2228450216
> @gyfora
>
> > as we have a stable logic to always record the jobID in the status before submission.
>
> could you point to the related changes (ticket/PR/commit) to understand the context a bit better?
https://github.com/apache/flink-kubernetes-operator/blob/main/flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/deployment/ApplicationReconciler.java#L216-L218
https://github.com/apache/flink-kubernetes-operator/blob/main/flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/sessionjob/SessionJobReconciler.java#L86-L88
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org