You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/09/28 04:18:05 UTC
[GitHub] [flink] reswqa opened a new pull request, #20912: [FLINK-29413] Exceptions during job graph serialization lock up client
reswqa opened a new pull request, #20912:
URL: https://github.com/apache/flink/pull/20912
## What is the purpose of the change
The parallel serialization introduced in [FLINK-26675](https://issues.apache.org/jira/browse/FLINK-26675) does not properly handle errors.
StreamConfig#triggerSerializationAndReturnFuture returns a future that is never completed if the serialization fails, because all we do is throw an exception but that isn't propagated anywhere.
As a result StreamingJobGraphGenerator#createJobGraph blocks indefinitely.
## Brief change log
- *Propagate exception to the `serializationFuture` when a serialization exception is encountered.*
## Verifying this change
Please make sure both new and modified tests in this PR follows the conventions defined in our code quality guide: https://flink.apache.org/contributing/code-style-and-quality-common.html#testing
This change added two tests in `StreamingJobGraphGeneratorTest` verified the serialization exceptions of coordinator and streamConfig respectively.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] reswqa commented on a diff in pull request #20912: [FLINK-29431] Exceptions during job graph serialization lock up client
Posted by GitBox <gi...@apache.org>.
reswqa commented on code in PR #20912:
URL: https://github.com/apache/flink/pull/20912#discussion_r981980774
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamConfig.java:
##########
@@ -176,10 +175,18 @@ public CompletableFuture<StreamConfig> triggerSerializationAndReturnFuture(
Function.identity())),
this.config,
CHAINED_TASK_CONFIG);
- } catch (IOException e) {
- throw new StreamTaskException(
- "Could not serialize object for key chained task config.",
- e);
+ } catch (IOException | StreamTaskException e) {
+ StreamTaskException serializationException;
+ if (e instanceof IOException) {
+ serializationException =
+ new StreamTaskException(
+ "Could not serialize object for key chained task config.",
+ e);
+ } else {
+ serializationException = (StreamTaskException) e;
+ }
+ serializationFuture.completeExceptionally(serializationException);
+ throw serializationException;
Review Comment:
ok, fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #20912: [FLINK-29431] Exceptions during job graph serialization lock up client
Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #20912:
URL: https://github.com/apache/flink/pull/20912#issuecomment-1260369392
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "744824256d8307953c3bc4b0afe71ad3f00be460",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "744824256d8307953c3bc4b0afe71ad3f00be460",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 744824256d8307953c3bc4b0afe71ad3f00be460 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] xintongsong closed pull request #20912: [FLINK-29431] Exceptions during job graph serialization lock up client
Posted by GitBox <gi...@apache.org>.
xintongsong closed pull request #20912: [FLINK-29431] Exceptions during job graph serialization lock up client
URL: https://github.com/apache/flink/pull/20912
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] xintongsong commented on a diff in pull request #20912: [FLINK-29431] Exceptions during job graph serialization lock up client
Posted by GitBox <gi...@apache.org>.
xintongsong commented on code in PR #20912:
URL: https://github.com/apache/flink/pull/20912#discussion_r981962104
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamConfig.java:
##########
@@ -176,10 +175,18 @@ public CompletableFuture<StreamConfig> triggerSerializationAndReturnFuture(
Function.identity())),
this.config,
CHAINED_TASK_CONFIG);
- } catch (IOException e) {
- throw new StreamTaskException(
- "Could not serialize object for key chained task config.",
- e);
+ } catch (IOException | StreamTaskException e) {
+ StreamTaskException serializationException;
+ if (e instanceof IOException) {
+ serializationException =
+ new StreamTaskException(
+ "Could not serialize object for key chained task config.",
+ e);
+ } else {
+ serializationException = (StreamTaskException) e;
+ }
+ serializationFuture.completeExceptionally(serializationException);
+ throw serializationException;
Review Comment:
There's no point of throwing an exception in this thread. I think we can simply catch any throwable and complete the future with them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org