You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Greg Harris (Jira)" <ji...@apache.org> on 2024/01/23 22:40:00 UTC

[jira] [Commented] (KAFKA-15343) Fix MirrorConnectIntegrationTests causing ci build failures.

    [ https://issues.apache.org/jira/browse/KAFKA-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810124#comment-17810124 ] 

Greg Harris commented on KAFKA-15343:
-------------------------------------

I opened an issue in Gradle for this bug, since it's not unique to Kafka and can appear in any project which uses sockets: [https://github.com/gradle/gradle/issues/27801]

I have also been working to reduce the number of socket/client leaks through https://issues.apache.org/jira/browse/KAFKA-15845 and have made a lot of progress in stopping the leaks we have currently, and should be able to prevent new ones in the future.

> Fix MirrorConnectIntegrationTests causing ci build failures.
> ------------------------------------------------------------
>
>                 Key: KAFKA-15343
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15343
>             Project: Kafka
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 3.6.0
>            Reporter: Prasanth Kumar
>            Priority: Major
>
> There are several instances of tests interacting badly with gradle daemon(s) running on ports that the kafka broker previously used. After going through the debug logs we observed a few retrying kafka clients trying to connect to broker which got shutdown and the gradle worker chose the same port on which broker was running. Later in the build, the gradle daemon attempted to connect to the worker and could not, triggering a failure. Ideally gradle would not exit when connected to from an invalid client - in testing with netcat, it would often handle these without dying. However there appear to be some cases where the daemon dies completely. Both the broker code and the gradle workers bind to port 0, resulting in the OS assigning it an unused port. This does avoid conflicts, but does not ensure that long lived clients do not attempt to connect to these ports afterwards. It's possible that closing the client in between may be enough to work around this issue. Till then we will disable the test to avoid the ci blocker from testing the code changes.
> *MirrorConnectorsIntegrationBaseTest and extending Tests*
> {code:java}
> [2023-07-04T11:48:16.128Z] 2023-07-04T11:47:46.804+0000 [DEBUG] [TestEventLogger] 
> MirrorConnectorsWithCustomForwardingAdminIntegrationTest > testReplicateSourceDefault() STANDARD_OUT
> [2023-07-04T11:48:16.128Z] 2023-07-04T11:47:46.804+0000 [DEBUG] [TestEventLogger]     [2023-07-04 11:47:46,799]
>  INFO primary REST service: http://localhost:43809/connectors (org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest:224)
> [2023-07-04T11:48:16.128Z] 2023-07-04T11:47:46.804+0000 [DEBUG] [TestEventLogger]     [2023-07-04 11:47:46,799] 
> INFO backup REST service: http://localhost:43323/connectors (org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest:225)
> [2023-07-04T11:48:16.128Z] 2023-07-04T11:47:46.804+0000 [DEBUG] [TestEventLogger]     [2023-07-04 11:47:46,799] 
> INFO primary brokers: localhost:37557 (org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest:226)
> [2023-07-04T11:59:12.968Z] 2023-07-04T11:59:12.900+0000 [DEBUG] [org.gradle.internal.remote.internal.inet.TcpIncomingConnector] 
> Accepted connection from /127.0.0.1:47660 to /127.0.0.1:37557.
> [2023-07-04T11:59:13.233Z] 
> org.gradle.internal.remote.internal.MessageIOException: Could not read message from '/127.0.0.1:47660'.
> [2023-07-04T11:59:12.970Z] 2023-07-04T11:59:12.579+0000 [DEBUG] 
> [org.gradle.internal.remote.internal.inet.TcpIncomingConnector] Listening on [d6bf30cb-bca2-46d9-8aeb-b9fd0497f54d port:37557, addresses:[localhost/127.0.0.1]].
> [2023-07-04T11:59:46.519Z] 2023-07-04T11:59:13.014+0000 [ERROR] 
> [system.err] org.gradle.internal.remote.internal.ConnectException: Could not connect to server [d6bf30cb-bca2-46d9-8aeb-b9fd0497f54d port:37557, addresses:[/127.0.0.1]]. Tried addresses: [/127.0.0.1]. {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)