You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Zhile (via GitHub)" <gi...@apache.org> on 2026/07/01 13:39:36 UTC
[PR] [FLINK-40040][cdc-runtime] Wrap coordinator failure in SerializedThrowable across the RPC boundary [flink-cdc]
Zhile opened a new pull request, #4456:
URL: https://github.com/apache/flink-cdc/pull/4456
## What is the purpose of the change
Schema-evolution coordination responses are dropped when the failure's cause is a class that only lives in the user classloader (e.g. `com.mysql.cj.exceptions.ConnectionIsClosedException`). `flink-rpc-akka` deserializes the coordination response with an isolated classloader and throws `ClassNotFoundException`, so the response never reaches `SchemaOperator`, which then blocks on `responseFuture.get(rpcTimeout)` and fails with a misleading `TimeoutException` -- turning a transient error into a restart loop that only a full job restart recovers from.
Observed in production (Flink CDC 3.6.0 on Flink 1.20.3):
```
ERROR org.apache.pekko.remote.Remoting - com.mysql.cj.exceptions.ConnectionIsClosedException
java.lang.ClassNotFoundException: com.mysql.cj.exceptions.ConnectionIsClosedException
at org.apache.pekko.util.ClassLoaderObjectInputStream.resolveClass(...)
at ...MiscMessageSerializer.deserializeStatusFailure(...)
```
JIRA: https://issues.apache.org/jira/browse/FLINK-40040
## Brief change log
- `SchemaRegistry.failJob` now wraps the cause into `SerializedThrowable` (idempotent, not re-wrapped if already one) before calling `handleUnrecoverableError` (which completes pending coordination responses exceptionally) and `context.failJob`. `SerializedThrowable` carries the original exception as bytes plus a stringified stack trace, so the receiving side can deserialize it without the original class and still see the real cause.
- `SchemaRegistry.runInEventLoop`'s catch block now routes through `failJob` instead of duplicating the fail path, so every exit shares the wrapping.
- Applies to both the regular and distributed `SchemaCoordinator`, since both complete pending response futures via the shared `failJob` -> `handleUnrecoverableError` path.
## Verifying this change
Added `SchemaRegistryFailJobSerializationTest`:
- `failJobWrapsExceptionIntoSerializedThrowable` asserts the failure delivered to the coordinator context is a `SerializedThrowable`.
- `wrappedFailureSurvivesIsolatedClassloaderButRawOneDoesNot` shows a raw exception whose cause is a classloader-isolated type fails to deserialize (`ClassNotFoundException`), while the `SerializedThrowable`-wrapped one deserializes and preserves the real cause text.
## Does this pull request potentially affect one of the following parts
- Dependencies (does it add or upgrade a dependency): no
- Public API / SDK: no
- Serializers / state: no (wraps only the failure carried over the coordinator RPC channel; no state format change)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40040][cdc-runtime] Wrap coordinator failure in SerializedThrowable across the RPC boundary [flink-cdc]
Posted by "Zhile (via GitHub)" <gi...@apache.org>.
Zhile commented on PR #4456:
URL: https://github.com/apache/flink-cdc/pull/4456#issuecomment-4915213412
please help review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40040][cdc-runtime] Wrap coordinator failure in SerializedThrowable across the RPC boundary [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on PR #4456:
URL: https://github.com/apache/flink-cdc/pull/4456#issuecomment-4899514458
Could you please add a pipeline E2e case for this patch? I believe we can reproduce this issue by putting different jar dependencies in JM and TM docker containers.
Also, rebasing latest `master` branch may eliminate most CI failing noises.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40040][cdc-runtime] Wrap coordinator failure in SerializedThrowable across the RPC boundary [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on PR #4456:
URL: https://github.com/apache/flink-cdc/pull/4456#issuecomment-4856288926
Could you please check the failing test cases?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org