You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Brian (JIRA)" <ji...@apache.org> on 2019/02/25 21:46:00 UTC
[jira] [Created] (KAFKA-8000) Java client uses IP address for
different kafka broker, preventing reconnection
Brian created KAFKA-8000:
----------------------------
Summary: Java client uses IP address for different kafka broker, preventing reconnection
Key: KAFKA-8000
URL: https://issues.apache.org/jira/browse/KAFKA-8000
Project: Kafka
Issue Type: Bug
Components: clients
Affects Versions: 2.1.1
Reporter: Brian
I've been observing this issue occurring most times I kill a Kafka Broker (for sake of exposing bugs, I periodically kill one of my kafka brokers). This bug results in large latency spikes. And sometimes prevents the java Kafka Client from ever being able to resume transmitting messages to a partition.
VERSIONS:
* I'm using Kafka 2.1.1, in confluentinc/cp-kafka:5.1.2. I also saw this in 5.0.1.
* I'm using the Kafka client built from git sha 9f7e6b291309286e3e3c1610e98d978773c9d504. I needed to cut this release since it includes the fix
https://issues.apache.org/jira/browse/KAFKA-7890 for a catastrophic bug we were seeing
OBSERVED ISSUE:
* Create kafka cluster with three brokers.
* Setup topic with replication factor of three
* Create example service that reads and writes from Kafka partition
* Kill brokers periodically.
In the following log, you see a situation where Node 30 has been killed intentionally. As expected, you'll see that the Kafka Client cannot find Node 30 any longer.
However, what isn't expected is that when the Kafka Client attempts to connect to a different node it becomes confused about which Node has which IP address. You'll see lines such as "Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b)", even though previous metadata updates state "10.20.2.109" is the IP address for node 30, not Node 20!
You'll notice that this behavior continues for a long period of time, even once the Kafka Client observes that node 30 is offline in MetadataCache update the following
```
MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
```
The full logs I saw are
```
ordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.11.109:9092 (id: 30 rack: us-west2-c)
17:47:44.355 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 34 to 34 for partition exampletopic3-0
17:47:44.355 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 334 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.11.109:9092 (id: 30 rack: us-west2-c), 10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 30, replicas = [30,20,10], isr = [30,20], offlineReplicas = [10])], controller = 10.20.11.109:9092 (id: 30 rack: us-west2-c))}
poll emptiness = false
17:47:44.380 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30053 for partition exampletopic3-0
17:47:44.585 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30053 to node 10.20.11.109:9092 (id: 30 rack: us-west2-c)
17:47:44.585 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=622931687, epoch=787) for node 30. Added 0 partition(s), altered 1 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:44.585 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(exampletopic3-0), toForget=(), implied=()) to broker 10.20.11.109:9092 (id: 30 rack: us-west2-c)
17:47:44.605 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 30 sent an incremental fetch response for session 622931687 with 1 response partition(s)
17:47:44.605 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Fetch READ_UNCOMMITTED at offset 30053 for partition exampletopic3-0 returned fetch data (error=NONE, highWaterMark=30059, lastStableOffset = 30059, logStartOffset = 0, abortedTransactions = null, recordsSizeInBytes=846)
17:47:44.605 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30059 to node 10.20.11.109:9092 (id: 30 rack: us-west2-c)
17:47:44.605 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=622931687, epoch=788) for node 30. Added 0 partition(s), altered 1 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:44.605 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(exampletopic3-0), toForget=(), implied=()) to broker 10.20.11.109:9092 (id: 30 rack: us-west2-c)
poll emptiness = false
17:47:44.631 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30059 for partition exampletopic3-0
17:47:44.708 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 30 sent an incremental fetch response for session 622931687 with 1 response partition(s)
17:47:44.708 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Fetch READ_UNCOMMITTED at offset 30059 for partition exampletopic3-0 returned fetch data (error=NONE, highWaterMark=30060, lastStableOffset = 30060, logStartOffset = 0, abortedTransactions = null, recordsSizeInBytes=141)
17:47:44.769 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.io.EOFException: null
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:96)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:644)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:575)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:44.769 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:44.770 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:44.770 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
NOTE: Node 30 disconnected! I think Node 30 was the group coordinator
17:47:44.793 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:44.793 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
NOTE: Tries to connect to node 20 now? But uses 10.20.2.109.
TRIES TO CONNECT TO NODE 20 NOW!!!
FAILS!!! Because it tries to use the same address as the address from node 30.
17:47:44.793 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:44.793 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:44.812 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection with /10.20.11.109 disconnected
java.io.EOFException: null
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:96)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:644)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:575)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.pollNoWakeup(ConsumerNetworkClient.java:304)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1027)
17:47:44.813 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection with /10.20.11.109 disconnected
java.io.EOFException: null
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:96)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:644)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:575)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.pollNoWakeup(ConsumerNetworkClient.java:304)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$HeartbeatThread.run(AbstractCoordinator.java:1027)
17:47:44.813 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 2147483617 disconnected.
17:47:44.813 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 30 disconnected.
17:47:44.813 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null) is unavailable or invalid, will attempt rediscovery
17:47:44.813 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending FindCoordinator request to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:44.813 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:44.822 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:44.836 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 34 to 34 for partition exampletopic3-0
17:47:44.836 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 335 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.11.109:9092 (id: 30 rack: us-west2-c), 10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 30, replicas = [30,20,10], isr = [30,20], offlineReplicas = [10])], controller = 10.20.11.109:9092 (id: 30 rack: us-west2-c))}
17:47:44.839 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:44.839 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:44.839 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:44.839 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:44.855 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received FindCoordinator response ClientResponse(receivedTimeMs=1550972864855, latencyMs=42, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=2, clientId=consumer-1, correlationId=3618), responseBody=FindCoordinatorResponse(throttleTimeMs=0, errorMessage='null', error=NONE, node=10.20.11.109:9092 (id: 30 rack: null)))
17:47:44.855 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Discovered group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null)
17:47:44.855 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null) is unavailable or invalid, will attempt rediscovery
17:47:44.893 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
TRIES TO CONNECT TO NODE 30, USING THE ADDRESS OF NODE 20.
Absurd, because NODE 30's IP address is 10.20.11.110 not 10.20.11.109.
17:47:44.935 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node
10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:44.935 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending FindCoordinator request to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:44.953 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:44.953 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:44.953 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:44.953 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:44.954 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received FindCoordinator response ClientResponse(receivedTimeMs=1550972864954, latencyMs=19, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=2, clientId=consumer-1, correlationId=3620), responseBody=FindCoordinatorResponse(throttleTimeMs=0, errorMessage='null', error=NONE, node=10.20.11.109:9092 (id: 30 rack: null)))
17:47:44.954 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Discovered group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null)
17:47:44.954 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initiating connection to node 10.20.11.109:9092 (id: 2147483617 rack: null) using address /10.20.11.109
connect(id=2147483617, address=/10.20.11.109:9092...)
17:47:44.974 [Thread-0] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:661)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1341)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1304)
at org.apache.kafka.Main$1.run(Main.java:130)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:44.975 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 2147483617 disconnected.
17:47:44.975 [Thread-0] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection to node 2147483617 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:44.975 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Cancelled request with header RequestHeader(apiKey=OFFSET_COMMIT, apiVersion=6, clientId=consumer-1, correlationId=3621) due to node 2147483617 being disconnected
17:47:44.975 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null) is unavailable or invalid, will attempt rediscovery
17:47:45.004 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.037 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initialize connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) for sending metadata request
17:47:45.037 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:45.038 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending FindCoordinator request to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:45.055 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.079 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:45.080 [Thread-0] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:235)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:656)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1341)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1304)
at org.apache.kafka.Main$1.run(Main.java:130)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:45.080 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 30 disconnected.
17:47:45.080 [Thread-0] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:45.098 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 34 to 34 for partition exampletopic3-0
17:47:45.098 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 336 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.11.109:9092 (id: 30 rack: us-west2-c), 10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 30, replicas = [30,20,10], isr = [30,20], offlineReplicas = [10])], controller = 10.20.11.109:9092 (id: 30 rack: us-west2-c))}
NOTE: This is a successful metadata refresh request.
17:47:45.099 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received FindCoordinator response ClientResponse(receivedTimeMs=1550972865099, latencyMs=61, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=2, clientId=consumer-1, correlationId=3622), responseBody=FindCoordinatorResponse(throttleTimeMs=0, errorMessage='null', error=NONE, node=10.20.11.109:9092 (id: 30 rack: null)))
17:47:45.099 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Discovered group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null)
17:47:45.100 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initiating connection to node 10.20.11.109:9092 (id: 2147483617 rack: null) using address /10.20.11.109
connect(id=2147483617, address=/10.20.11.109:9092...)
17:47:45.105 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.122 [Thread-0] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:661)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1341)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1304)
at org.apache.kafka.Main$1.run(Main.java:130)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:45.122 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 2147483617 disconnected.
17:47:45.122 [Thread-0] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection to node 2147483617 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:45.122 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Cancelled request with header RequestHeader(apiKey=OFFSET_COMMIT, apiVersion=6, clientId=consumer-1, correlationId=3624) due to node 2147483617 being disconnected
17:47:45.122 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null) is unavailable or invalid, will attempt rediscovery
17:47:45.143 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending FindCoordinator request to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:45.159 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.173 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:45.191 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:45.191 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:45.192 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:45.192 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.227 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initialize connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) for sending metadata request
17:47:45.227 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:45.227 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received FindCoordinator response ClientResponse(receivedTimeMs=1550972865227, latencyMs=84, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=2, clientId=consumer-1, correlationId=3625), responseBody=FindCoordinatorResponse(throttleTimeMs=0, errorMessage='null', error=NONE, node=10.20.11.109:9092 (id: 30 rack: null)))
17:47:45.227 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Discovered group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null)
17:47:45.228 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null) is unavailable or invalid, will attempt rediscovery
17:47:45.244 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.298 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.332 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending FindCoordinator request to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:45.332 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:45.333 [Thread-0] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:235)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:656)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1341)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1304)
at org.apache.kafka.Main$1.run(Main.java:130)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:45.333 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 30 disconnected.
17:47:45.333 [Thread-0] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:45.352 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.369 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 34 to 34 for partition exampletopic3-0
17:47:45.370 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 337 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.11.109:9092 (id: 30 rack: us-west2-c), 10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 30, replicas = [30,20,10], isr = [30,20], offlineReplicas = [10])], controller = 10.20.11.109:9092 (id: 30 rack: us-west2-c))}
17:47:45.376 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received FindCoordinator response ClientResponse(receivedTimeMs=1550972865376, latencyMs=44, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=2, clientId=consumer-1, correlationId=3626), responseBody=FindCoordinatorResponse(throttleTimeMs=0, errorMessage='null', error=NONE, node=10.20.11.109:9092 (id: 30 rack: null)))
17:47:45.376 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Discovered group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null)
17:47:45.376 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initiating connection to node 10.20.11.109:9092 (id: 2147483617 rack: null) using address /10.20.11.109
connect(id=2147483617, address=/10.20.11.109:9092...)
17:47:45.396 [Thread-0] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:661)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1341)
at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1304)
at org.apache.kafka.Main$1.run(Main.java:130)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:45.396 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 2147483617 disconnected.
17:47:45.396 [Thread-0] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection to node 2147483617 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:45.396 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Cancelled request with header RequestHeader(apiKey=OFFSET_COMMIT, apiVersion=6, clientId=consumer-1, correlationId=3628) due to node 2147483617 being disconnected
17:47:45.396 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null) is unavailable or invalid, will attempt rediscovery
17:47:45.396 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending FindCoordinator request to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:45.403 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.454 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.498 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:45.498 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received FindCoordinator response ClientResponse(receivedTimeMs=1550972865498, latencyMs=102, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=2, clientId=consumer-1, correlationId=3629), responseBody=FindCoordinatorResponse(throttleTimeMs=0, errorMessage='null', error=NONE, node=10.20.11.109:9092 (id: 30 rack: null)))
17:47:45.498 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Discovered group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null)
17:47:45.498 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Group coordinator 10.20.11.109:9092 (id: 2147483617 rack: null) is unavailable or invalid, will attempt rediscovery
17:47:45.498 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending FindCoordinator request to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:45.506 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.521 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 34 to 35 for partition exampletopic3-0
17:47:45.521 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 338 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b), 10.20.11.109:9092 (id: 30 rack: us-west2-c)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.11.109:9092 (id: 30 rack: us-west2-c))}
17:47:45.522 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received FindCoordinator response ClientResponse(receivedTimeMs=1550972865522, latencyMs=24, disconnected=false, requestHeader=RequestHeader(apiKey=FIND_COORDINATOR, apiVersion=2, clientId=consumer-1, correlationId=3631), responseBody=FindCoordinatorResponse(throttleTimeMs=0, errorMessage='null', error=NONE, node=10.20.2.110:9092 (id: 20 rack: null)))
17:47:45.522 [Thread-0] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Discovered group coordinator 10.20.2.110:9092 (id: 2147483627 rack: null)
17:47:45.522 [Thread-0] INFO org.apache.kafka.clients.ClusterConnectionStates - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Hostname for node 2147483627 changed from 10.20.2.109 to 10.20.2.110.
17:47:45.522 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initiating connection to node 10.20.2.110:9092 (id: 2147483627 rack: null) using address /10.20.2.110
connect(id=2147483627, address=/10.20.2.110:9092...)
17:47:45.546 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:45.546 [Thread-0] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Created socket with SO_RCVBUF = 65905, SO_SNDBUF = 131810, SO_TIMEOUT = 0 to node 2147483627
17:47:45.547 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Completed connection to node 2147483627. Fetching API versions.
17:47:45.547 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initiating API versions fetch from node 2147483627.
17:47:45.568 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:45.568 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:45.568 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:45.568 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.568 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Recorded API versions for node 2147483627: (Produce(0): 0 to 7 [usable: 7], Fetch(1): 0 to 10 [usable: 10], ListOffsets(2): 0 to 4 [usable: 4], Metadata(3): 0 to 7 [usable: 7], LeaderAndIsr(4): 0 to 1 [usable: 1], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 4 [usable: 4], ControlledShutdown(7): 0 to 1 [usable: 1], OffsetCommit(8): 0 to 6 [usable: 6], OffsetFetch(9): 0 to 5 [usable: 5], FindCoordinator(10): 0 to 2 [usable: 2], JoinGroup(11): 0 to 3 [usable: 3], Heartbeat(12): 0 to 2 [usable: 2], LeaveGroup(13): 0 to 2 [usable: 2], SyncGroup(14): 0 to 2 [usable: 2], DescribeGroups(15): 0 to 2 [usable: 2], ListGroups(16): 0 to 2 [usable: 2], SaslHandshake(17): 0 to 1 [usable: 1], ApiVersions(18): 0 to 2 [usable: 2], CreateTopics(19): 0 to 3 [usable: 3], DeleteTopics(20): 0 to 3 [usable: 3], DeleteRecords(21): 0 to 1 [usable: 1], InitProducerId(22): 0 to 1 [usable: 1], OffsetForLeaderEpoch(23): 0 to 2 [usable: 2], AddPartitionsToTxn(24): 0 to 1 [usable: 1], AddOffsetsToTxn(25): 0 to 1 [usable: 1], EndTxn(26): 0 to 1 [usable: 1], WriteTxnMarkers(27): 0 [usable: 0], TxnOffsetCommit(28): 0 to 2 [usable: 2], DescribeAcls(29): 0 to 1 [usable: 1], CreateAcls(30): 0 to 1 [usable: 1], DeleteAcls(31): 0 to 1 [usable: 1], DescribeConfigs(32): 0 to 2 [usable: 2], AlterConfigs(33): 0 to 1 [usable: 1], AlterReplicaLogDirs(34): 0 to 1 [usable: 1], DescribeLogDirs(35): 0 to 1 [usable: 1], SaslAuthenticate(36): 0 [usable: 0], CreatePartitions(37): 0 to 1 [usable: 1], CreateDelegationToken(38): 0 to 1 [usable: 1], RenewDelegationToken(39): 0 to 1 [usable: 1], ExpireDelegationToken(40): 0 to 1 [usable: 1], DescribeDelegationToken(41): 0 to 1 [usable: 1], DeleteGroups(42): 0 to 1 [usable: 1], ElectPreferredLeaders(43): UNSUPPORTED)
17:47:45.595 [Thread-0] ERROR org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Offset commit failed on partition exampletopic3-0 at offset 30059: The coordinator is loading and hence can't process requests.
17:47:45.598 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending Heartbeat request to coordinator 10.20.2.110:9092 (id: 2147483627 rack: null)
17:47:45.620 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.670 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.700 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received successful Heartbeat response
17:47:45.720 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.726 [Thread-0] ERROR org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Offset commit failed on partition exampletopic3-0 at offset 30059: The coordinator is loading and hence can't process requests.
17:47:45.773 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.827 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.880 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:45.880 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30059 for partition exampletopic3-0
17:47:45.932 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:45.932 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:45.950 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:45.950 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:45.950 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:45.950 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.002 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.052 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 1373, value = bb094f23-f7c7-47b7-b62f-c467bc42e462
STILL UNRECEIVED, DELAY = : 1449, value = dcdc5599-7b1a-43de-8d9d-5bbdf05db5c5
STILL UNRECEIVED, DELAY = : 1300, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:46.082 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30060 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:46.082 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built full fetch (sessionId=1651867364, epoch=INITIAL) for node 20 with 1 partition(s).
17:47:46.082 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED FullFetchRequest(exampletopic3-0) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
poll emptiness = false
RECEIVING TOOK LONG TIME: 1449
17:47:46.103 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.106 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30060 for partition exampletopic3-0
17:47:46.154 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.204 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.214 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent a full fetch response that created a new incremental fetch session 1250380980 with 1 response partition(s)
17:47:46.214 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Fetch READ_UNCOMMITTED at offset 30060 for partition exampletopic3-0 returned fetch data (error=NONE, highWaterMark=30061, lastStableOffset = 30061, logStartOffset = 0, abortedTransactions = null, recordsSizeInBytes=141)
17:47:46.254 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.305 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 1601, value = bb094f23-f7c7-47b7-b62f-c467bc42e462
STILL UNRECEIVED, DELAY = : 1528, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:46.310 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:46.310 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=1) for node 20. Added 0 partition(s), altered 1 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:46.310 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(exampletopic3-0), toForget=(), implied=()) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
poll emptiness = false
RECEIVING TOOK LONG TIME: 1602
17:47:46.335 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:46.355 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.381 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:46.399 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:46.400 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:46.400 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:46.400 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.454 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.505 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 1753, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:46.535 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initialize connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) for sending metadata request
17:47:46.535 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:46.553 [Thread-0] DEBUG org.apache.kafka.common.network.Selector - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1256)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1200)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1176)
at org.apache.kafka.Main$1.run(Main.java:113)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:46.553 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 30 disconnected.
17:47:46.553 [Thread-0] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:46.554 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:46.556 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.608 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:46.656 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:46.660 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.715 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.765 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.816 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 2076, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:46.859 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:46.859 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:46.859 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 339 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
17:47:46.859 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:46.859 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=2) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:46.859 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:46.869 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.924 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:46.977 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:46.980 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:47.031 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.085 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:47.085 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:47.103 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:47.104 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:47.104 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:47.104 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.157 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 2400, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:47.211 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.264 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:47.305 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:47.315 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.368 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.389 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:47.398 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:47.418 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:47.418 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:47.418 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:47.418 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.470 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 2727, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:47.509 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:47.509 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=3) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:47.509 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:47.523 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.576 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:47.627 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.632 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:47.681 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.731 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.782 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 3053, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:47.835 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.864 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:47.890 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:47.944 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:47.957 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:47.998 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:47.998 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:48.016 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:48.017 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:48.017 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:48.017 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.037 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:48.070 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.121 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.137 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:48.138 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 340 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
STILL UNRECEIVED, DELAY = : 3378, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:48.160 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:48.160 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=4) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:48.160 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:48.173 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.218 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending asynchronous auto-commit of offsets \{exampletopic3-0=OffsetAndMetadata{offset=30061, leaderEpoch=null, metadata=''}}
17:47:48.224 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.236 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:48.236 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Completed asynchronous auto-commit of offsets \{exampletopic3-0=OffsetAndMetadata{offset=30061, leaderEpoch=null, metadata=''}}
poll emptiness = true
17:47:48.277 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.285 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:48.328 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.381 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.431 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.483 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 3708, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:48.533 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.584 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.589 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
poll emptiness = true
17:47:48.607 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:48.607 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:48.607 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:48.607 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.616 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:48.659 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.699 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:48.699 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending Heartbeat request to coordinator 10.20.2.110:9092 (id: 2147483627 rack: null)
17:47:48.710 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.760 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.804 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received successful Heartbeat response
17:47:48.810 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 4035, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:48.817 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:48.817 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=5) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:48.817 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:48.865 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:48.918 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:48.937 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:48.971 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:48.971 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:48.995 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:48.995 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:48.995 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:48.995 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.048 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.102 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 4358, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:49.140 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:49.155 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.209 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:49.264 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.288 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:49.315 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.348 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:49.370 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.422 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.449 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:49.449 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 341 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
17:47:49.474 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 4710, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:49.492 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:49.492 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=6) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:49.492 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:49.526 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.578 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.588 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
poll emptiness = true
17:47:49.606 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:49.606 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:49.606 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:49.606 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.615 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:49.660 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.715 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.767 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.818 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 5036, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:49.869 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:49.924 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:49.956 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:49.976 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.022 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:50.026 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.077 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.127 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:50.127 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
STILL UNRECEIVED, DELAY = : 5378, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:50.160 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:50.160 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=7) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:50.160 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:50.171 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:50.171 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:50.171 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:50.171 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.225 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:50.275 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.285 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:50.330 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.384 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.434 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.467 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
STILL UNRECEIVED, DELAY = : 5705, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:50.487 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.539 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:50.593 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.607 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:50.644 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.693 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:50.698 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.715 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:50.731 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:50.731 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:50.731 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:50.731 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.784 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.796 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:50.796 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 342 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
STILL UNRECEIVED, DELAY = : 6028, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:50.810 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:50.810 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=8) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:50.810 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:50.835 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.885 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:50.939 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:50.940 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:50.993 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.047 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.099 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 6361, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:51.149 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.200 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:51.255 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.263 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:51.308 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:51.308 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:51.326 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:51.326 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:51.326 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:51.326 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.344 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:51.381 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.434 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 6684, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:51.466 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:51.466 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=9) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:51.466 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:51.487 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.542 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:51.590 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:51.596 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.649 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.703 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.756 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.778 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending Heartbeat request to coordinator 10.20.2.110:9092 (id: 2147483627 rack: null)
STILL UNRECEIVED, DELAY = : 7010, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:51.797 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:51.797 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received successful Heartbeat response
17:47:51.810 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.861 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:51.913 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:51.915 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.925 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:51.943 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:51.943 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:51.943 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:51.943 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.994 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:51.995 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:52.048 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.099 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.099 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:52.099 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 343 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
STILL UNRECEIVED, DELAY = : 7334, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:52.116 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:52.116 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=10) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:52.116 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:52.150 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.200 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:52.236 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:52.250 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.303 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.357 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.410 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 7656, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:52.463 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:52.463 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:52.484 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:52.484 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:52.484 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:52.484 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.536 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:52.561 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:52.587 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.637 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.643 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:52.689 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.740 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 7980, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:52.762 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:52.762 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=11) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:52.762 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:52.790 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.818 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:52.835 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:52.835 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:52.835 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:52.835 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:52.884 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:52.886 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.936 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:52.989 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.043 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 8303, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:53.093 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.100 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:53.143 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:53.194 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.210 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:53.244 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.293 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:53.294 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.349 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.393 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:53.393 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 344 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
17:47:53.404 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 8629, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:53.411 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending asynchronous auto-commit of offsets \{exampletopic3-0=OffsetAndMetadata{offset=30061, leaderEpoch=null, metadata=''}}
17:47:53.411 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:53.411 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=12) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:53.411 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:53.431 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:53.431 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Completed asynchronous auto-commit of offsets \{exampletopic3-0=OffsetAndMetadata{offset=30061, leaderEpoch=null, metadata=''}}
17:47:53.458 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.510 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:53.535 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:53.563 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.616 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:53.616 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:53.634 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:53.634 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:53.634 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:53.634 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.687 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.705 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:53.724 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:53.725 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:53.725 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:53.725 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 8954, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:53.776 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.830 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:53.859 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:53.882 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.936 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:53.939 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:53.989 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.039 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 9277, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:54.060 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:54.060 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=13) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:54.060 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:54.093 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.144 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:54.184 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:54.196 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.250 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.300 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.351 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 9603, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:54.393 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:54.402 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.452 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:54.506 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.529 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:54.560 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.594 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:54.613 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.664 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.695 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:54.695 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 345 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
17:47:54.703 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
STILL UNRECEIVED, DELAY = : 9951, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:54.733 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:54.733 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=14) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:54.733 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:54.747 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:54.747 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:54.747 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:54.747 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.780 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending Heartbeat request to coordinator 10.20.2.110:9092 (id: 2147483627 rack: null)
17:47:54.798 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received successful Heartbeat response
17:47:54.799 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:54.850 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:54.850 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:54.852 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:54.867 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:54.867 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:54.867 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:54.867 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.917 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:54.970 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.020 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 10270, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:55.071 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.121 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:55.174 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.191 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:55.226 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.257 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:55.278 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.333 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.383 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 10611, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:55.393 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:55.393 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=15) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:55.393 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:55.434 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.484 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:55.516 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:55.535 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.588 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.643 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.661 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:55.682 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:55.682 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:55.682 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:55.682 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.695 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
STILL UNRECEIVED, DELAY = : 10934, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:55.733 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.783 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:55.783 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:55.802 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:55.803 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:55.803 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:55.803 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:55.842 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:55.854 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.909 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:55.918 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:55.963 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.015 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.020 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:56.020 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 346 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
STILL UNRECEIVED, DELAY = : 11261, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:56.043 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:56.043 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=16) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:56.043 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:56.068 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.120 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:56.167 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:56.174 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.226 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.277 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.330 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 11588, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:56.383 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.435 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:56.486 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.494 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:56.537 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.579 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:56.589 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.640 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.690 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 11913, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:56.695 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:56.695 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=17) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:56.695 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:56.743 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:56.743 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:56.760 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:56.760 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:56.760 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:56.761 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:56.814 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.820 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:56.850 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:56.866 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:56.866 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:56.866 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:56.866 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.917 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:56.969 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 12238, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:57.020 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.021 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:57.071 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:57.125 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.142 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:57.180 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.227 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:57.235 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.286 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.328 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:57.328 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 347 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
17:47:57.337 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 12562, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:57.344 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:57.344 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=18) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:57.344 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:57.389 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.440 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:57.466 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:57.492 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.544 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.595 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.650 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 12885, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:57.700 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.719 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:57.736 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:57.736 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:57.736 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:57.736 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:57.736 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:57.754 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:57.755 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:57.755 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:57.755 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:57.789 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:57.806 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.860 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.867 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:57.867 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending Heartbeat request to coordinator 10.20.2.110:9092 (id: 2147483627 rack: null)
17:47:57.911 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.965 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:57.968 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received successful Heartbeat response
STILL UNRECEIVED, DELAY = : 13211, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:57.993 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:57.993 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=19) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:57.993 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:58.017 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.072 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:58.114 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:58.126 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.177 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.227 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.281 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 13535, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:58.330 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:58.333 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.383 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.412 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending asynchronous auto-commit of offsets \{exampletopic3-0=OffsetAndMetadata{offset=30061, leaderEpoch=null, metadata=''}}
poll emptiness = true
17:47:58.430 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:58.434 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.437 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:58.437 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Completed asynchronous auto-commit of offsets \{exampletopic3-0=OffsetAndMetadata{offset=30061, leaderEpoch=null, metadata=''}}
17:47:58.484 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.534 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.585 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:58.585 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:58.603 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:58.603 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:58.603 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:58.603 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.612 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
STILL UNRECEIVED, DELAY = : 13855, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:58.638 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:58.638 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 348 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
17:47:58.638 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:58.638 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=20) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:58.638 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:58.654 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.707 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:58.760 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.761 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:58.810 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.861 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.912 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:58.914 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:58.931 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:58.931 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:58.931 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:58.931 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 14181, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:58.984 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.039 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:59.086 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:59.093 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.143 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.166 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:59.197 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.249 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 14505, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:59.287 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:59.287 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=21) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:59.287 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:59.300 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.354 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:59.404 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.405 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:59.454 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:47:59.454 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:47:59.472 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:59.472 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:47:59.472 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:47:59.472 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.526 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.576 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 14824, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:59.628 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.642 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:59.681 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:47:59.728 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:47:59.733 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.783 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.836 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.844 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:47:59.861 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:47:59.861 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:47:59.862 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:47:59.862 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:47:59.910 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:47:59.912 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 15150, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:47:59.932 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:47:59.932 [Thread-0] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 349 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
17:47:59.932 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:59.932 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=22) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:47:59.933 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:47:59.966 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.017 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:48:00.053 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:48:00.070 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.121 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.175 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.225 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 15473, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:48:00.276 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.327 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:48:00.378 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.379 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:48:00.430 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.461 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:48:00.481 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:48:00.481 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:48:00.500 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:48:00.500 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:48:00.500 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:48:00.500 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.552 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 15800, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:48:00.582 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:48:00.582 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=23) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:48:00.582 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:48:00.607 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.659 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:48:00.703 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:48:00.711 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.762 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.814 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.859 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:48:00.877 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:48:00.877 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 30 disconnected.
17:48:00.877 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 30 (/10.20.11.109:9092) could not be established. Broker may not be available.
17:48:00.877 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.888 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending Heartbeat request to coordinator 10.20.2.110:9092 (id: 2147483627 rack: null)
STILL UNRECEIVED, DELAY = : 16125, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:48:00.907 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Received successful Heartbeat response
17:48:00.931 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:00.934 [Thread-0] DEBUG org.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending metadata request (type=MetadataRequest, topics=exampletopic3) to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:48:00.983 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:48:01.029 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:48:01.034 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.085 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.114 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:48:01.140 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.193 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.217 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updating last seen epoch from 35 to 35 for partition exampletopic3-0
17:48:01.217 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 350 to MetadataCache\{cluster=Cluster(id = kd3GKmi_RHmTmyJja7HU0g, nodes = [10.20.2.110:9092 (id: 20 rack: us-west2-b)], partitions = [Partition(topic = exampletopic3, partition = 0, leader = 20, replicas = [30,20,10], isr = [20], offlineReplicas = [30,10])], controller = 10.20.2.110:9092 (id: 20 rack: us-west2-b))}
STILL UNRECEIVED, DELAY = : 16452, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:48:01.234 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Added READ_UNCOMMITTED fetch request for partition exampletopic3-0 at offset 30061 to node 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:48:01.234 [Thread-0] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Built incremental fetch (sessionId=1250380980, epoch=24) for node 20. Added 0 partition(s), altered 0 partition(s), removed 0 partition(s) out of 1 partition(s)
17:48:01.234 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Sending READ_UNCOMMITTED IncrementalFetchRequest(toSend=(), toForget=(), implied=(exampletopic3-0)) to broker 10.20.2.110:9092 (id: 20 rack: us-west2-b)
17:48:01.243 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.293 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:48:01.344 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.357 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:48:01.394 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) for sending metadata request
17:48:01.394 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.2.109:9092 (id: 20 rack: us-west2-b) using address /10.20.2.109
connect(id=20, address=/10.20.2.109:9092...)
17:48:01.417 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.2.109 disconnected
java.net.ConnectException: Socket is not connected
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:48:01.418 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Node 20 disconnected.
17:48:01.418 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node 20 (/10.20.2.109:9092) could not be established. Broker may not be available.
17:48:01.418 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.468 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.522 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
STILL UNRECEIVED, DELAY = : 16778, value = 7aa1dd6e-1fb3-4c41-a59f-ef82fd75647a
17:48:01.572 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.626 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
poll emptiness = true
17:48:01.677 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.682 [Thread-0] DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Committed offset 30061 for partition exampletopic3-0
17:48:01.727 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:48:01.756 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node 10.20.11.109:9092 (id: 30 rack: us-west2-c) using address /10.20.11.109
connect(id=30, address=/10.20.11.109:9092...)
17:48:01.761 [kafka-coordinator-heartbeat-thread | KafkaExampleConsume4] DEBUG org.apache.kafka.clients.FetchSessionHandler - [Consumer clientId=consumer-1, groupId=KafkaExampleConsume4] Node 20 sent an incremental fetch response for session 1250380980 with 0 response partition(s), 1 implied partition(s)
17:48:01.777 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.common.network.Selector - [Producer clientId=producer-1] Connection with /10.20.11.109 disconnected
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:216)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:521)
at org.apache.kafka.common.network.Selector.poll(Selector.java:473)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:535)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:311)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
at java.base/java.lang.Thread.run(Thread.java:834)
17:48:01.777 [kafka-producer-network-thread | producer-1] DEBUG org
```
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)