You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Matthias Pohl (Jira)" <ji...@apache.org> on 2023/10/17 12:24:00 UTC
[jira] [Commented] (FLINK-33278) RemotePekkoRpcActorTest.failsRpcResultImmediatelyIfRemoteRpcServiceIsNotAvailable fails on AZP
[ https://issues.apache.org/jira/browse/FLINK-33278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776185#comment-17776185 ]
Matthias Pohl commented on FLINK-33278:
---------------------------------------
That's a suspicious one. I'm not able to reproduce it locally with over 20000 repetitions. It's hard to tell what's the cause of this issue due to the lack of debug logs. I'm wondering whether we should enable debug logging for the pekko module using {{LogLevelExtension}} to get a bit more out of the test run in the future. Enabling it for all modules (even when written to its own log files like we do it for the ZooKeeper dependency) might be a bit of an overkill considering how many RPC calls are sent.
I would assume that it's not only a 1.19.0 issue but also affects 1.18.0 since we migrated to Pekko.
[~chesnay] do you have any other suggestions on that one? I couldn't find any error like that in Jira (considering Akka instead of Pekko as a substring).
> RemotePekkoRpcActorTest.failsRpcResultImmediatelyIfRemoteRpcServiceIsNotAvailable fails on AZP
> ----------------------------------------------------------------------------------------------
>
> Key: FLINK-33278
> URL: https://issues.apache.org/jira/browse/FLINK-33278
> Project: Flink
> Issue Type: Bug
> Components: Runtime / RPC
> Affects Versions: 1.19.0
> Reporter: Sergey Nuyanzin
> Priority: Critical
> Labels: test-stability
>
> This build [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=53740&view=logs&j=0e7be18f-84f2-53f0-a32d-4a5e4a174679&t=7c1d86e3-35bd-5fd5-3b7c-30c126a78702&l=6563]
> fails as
> {noformat}
> Oct 15 01:02:20 Multiple Failures (1 failure)
> Oct 15 01:02:20 -- failure 1 --
> Oct 15 01:02:20 [Any cause is instance of class 'class org.apache.flink.runtime.rpc.exceptions.RecipientUnreachableException']
> Oct 15 01:02:20 Expecting any element of:
> Oct 15 01:02:20 [java.util.concurrent.CompletionException: java.util.concurrent.TimeoutException: Invocation of [RemoteRpcInvocation(SerializedValueRespondingGateway.getSerializedValue())] at recipient [pekko.tcp://flink@localhost:38231/user/rpc/8c211f34-41e5-4efe-93bd-8eca6c590a7f] timed out. This is usually caused by: 1) Pekko failed sending the message silently, due to problems like oversized payload or serialization failures. In that case, you should find detailed error information in the logs. 2) The recipient needs more time for responding, due to problems like slow machines or network jitters. In that case, you can try to increase pekko.ask.timeout.
> Oct 15 01:02:20 at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
> Oct 15 01:02:20 at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947)
> Oct 15 01:02:20 at org.apache.flink.runtime.rpc.pekko.RemotePekkoRpcActorTest.lambda$failsRpcResultImmediatelyIfRemoteRpcServiceIsNotAvailable$1(RemotePekkoRpcActorTest.java:168)
> Oct 15 01:02:20 ...(63 remaining lines not displayed - this can be changed with Assertions.setMaxStackTraceElementsDisplayed),
> ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)