[jira] [Created] (KAFKA-13379) When I migrate nodes from cluster A to cluster B, cluster A reports an error

["donglei (Jira)" <ji...@apache.org>]

donglei created KAFKA-13379:
-------------------------------

             Summary: When I migrate nodes from cluster A to cluster B, cluster A reports an error
                 Key: KAFKA-13379
                 URL: https://issues.apache.org/jira/browse/KAFKA-13379
             Project: Kafka
          Issue Type: Bug
            Reporter: donglei


We have two kafka clusters：
 * ClusterA version：0.10.0.1
 * ClusterB version：0.10.2.1

Recently, due to the shortage of resources in cluster B, three machines are offline from cluster A and ready to be added to cluster B. the offline steps are as follows:
 * 1. Migrate the topic replicas on the three brokers to other brokers.
 * 2. Run the `bin/kafka-server-stop.sh` command on the three brokers to close the Kafka service.
 * 3. Clean the installation directory, data directory and log directory of Kafka service on the three brokers.

When the threee brokers offline from ClusterA, the ClusterA operates normally.

Then, we add the three brokers to cluster B in turn. After adding, the warning logs in server.log on the three brokers are as follows:
{code:java}
[2021-10-15 11:12:12,140] INFO [Kafka Server 12], started (kafka.server.KafkaServer)
[2021-10-15 11:12:12,323] WARN Attempting to send response via channel for which there is no open connection, connection id 5 (kafka.network.Processor)
[2021-10-15 11:12:18,445] WARN Attempting to send response via channel for which there is no open connection, connection id 1 (kafka.network.Processor)
[2021-10-15 11:12:29,527] WARN Attempting to send response via channel for which there is no open connection, connection id 3 (kafka.network.Processor)
[2021-10-15 11:12:31,585] WARN Attempting to send response via channel for which there is no open connection, connection id 1 (kafka.network.Processor)
[2021-10-15 11:12:31,728] WARN Attempting to send response via channel for which there is no open connection, connection id 10 (kafka.network.Processor)
[2021-10-15 11:12:57,526] WARN Attempting to send response via channel for which there is no open connection, connection id 0 (kafka.network.Processor){code}
At the same time, we found that all brokers in clusterA also began to report error logs as follows:
{code:java}
[2021-10-15 11:13:02,187] ERROR Closing socket for xxx:9092-xxxx:49691 because of error (kafka.network.Processor)
kafka.network.InvalidRequestException: Error getting request for apiKey: 3 and apiVersion: 2
        at kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:95)
        at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:87)
        at kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:488)
        at kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:483)
        at scala.collection.Iterator$class.foreach(Iterator.scala:727)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at kafka.network.Processor.processCompletedReceives(SocketServer.scala:483)
        at kafka.network.Processor.run(SocketServer.scala:413)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Invalid version for API key 3: 2
        at org.apache.kafka.common.protocol.ProtoUtils.schemaFor(ProtoUtils.java:31)
        at org.apache.kafka.common.protocol.ProtoUtils.requestSchema(ProtoUtils.java:44)
        at org.apache.kafka.common.protocol.ProtoUtils.parseRequest(ProtoUtils.java:60)
        at org.apache.kafka.common.requests.MetadataRequest.parse(MetadataRequest.java:96)
        at org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:48)
        at kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:92)
        ... 10 more
{code}
Is there any error in the way that my broker goes offline from clusterA ? Why are there no exceptions after the three brokers go offline? Adding them to clusterB will cause an error in clusterA service?

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)