You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Donny Nadolny (Jira)" <ji...@apache.org> on 2021/09/01 20:15:00 UTC
[jira] [Created] (KAFKA-13263) TimeoutException when calling
initializeTransactions() in a healthy cluster, persisting until a broker
was restarted
Donny Nadolny created KAFKA-13263:
-------------------------------------
Summary: TimeoutException when calling initializeTransactions() in a healthy cluster, persisting until a broker was restarted
Key: KAFKA-13263
URL: https://issues.apache.org/jira/browse/KAFKA-13263
Project: Kafka
Issue Type: Bug
Affects Versions: 2.6.2
Reporter: Donny Nadolny
Summary:
- we had multiple transactional kafka clients (different Apache Flink applications) stop working at the same time, all of them logging {{org.apache.kafka.common.errors.TimeoutException: Timeout expired while initializing transactional state in 60000ms.}}
- this problem persisted for several hours
- restarting the kafka process on a single broker (the leader for a {{__transaction_state}} partition that seemed to be the culprit) immediately the problem
Some details:
- the kafka cluster seemed healthy the whole time: no brokers were down, no under-replicated partitions, etc. No non-transactional clients seemed to be affected
- we first tried changing the {{transactional.id}} that the clients use, to see if some existing transaction state was the problem - it didn't work, same exception
- our transaction & cleanup timeouts are longer than the default: 1 hour for {{transaction.max.timeout.ms}} and 1 day for {{transaction.remove.expired.transaction.cleanup.interval.ms}}. The problem lasted for several hours but less than 1 day
- we diagnosed this by noticing that the "Initialized transactionalId" log lines in kafka from the TransactionCoordinator were present for all partitions except for one partition: {{__transaction_state-29}}. The leader of that partition (broker 1266) was the broker we restarted which fixed this.
- immediately after restarting the process on broker 1266, a follower took over, and shortly after that 1266 (the preferred leader) again became leader, and we didn't get any more timeouts
Logs:
All logs cluster-wide with {{__transaction_state-29}} from the time when this started happening, lightly redacted with {{--removed--}}:
{code:java}
[2021-08-31 18:22:09.596915] INFO TransactionCoordinator:66 - [TransactionCoordinator id=1232] Initialized transactionalId Source: flinkapp -> Try.get -> flat map to entry -> Sink: kafka-sink: flinkapp-b7463a48c0aedd250dc469cc1ffc08dc-50 with producerId 2116001 and producer epoch 11068 on partition __transaction_state-29
[2021-08-31 18:22:30.069676] DEBUG KafkaController:62 - [Controller id=1214] Topics not in preferred replica for broker 1266 Map(--removed--, __transaction_state-29 -> ArrayBuffer(1266, 1232, 1187), --removed--.
[2021-08-31 18:22:30.070326] INFO KafkaController:66 - [Controller id=1214] Starting replica leader election (PREFERRED) for partitions --removed--,__transaction_state-29,--removed triggered by AutoTriggered
[2021-08-31 18:22:30.096160] INFO logger:66 - [Controller id=1214 epoch=157] Changed partition __transaction_state-29 from OnlinePartition to OnlinePartition with state LeaderAndIsr(leader=1266, leaderEpoch=222, isr=List(1187, 1232, 1266), zkVersion=403)
[2021-08-31 18:22:30.101069] TRACE logger:54 - [Controller id=1214 epoch=157] Sending become-follower LeaderAndIsr request LeaderAndIsrPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], addingReplicas=[], removingReplicas=[], isNew=false) to broker 1187 for partition __transaction_state-29
[2021-08-31 18:22:30.102259] TRACE logger:54 - [Broker id=1187] Handling LeaderAndIsr request correlationId 9187 from controller 1214 epoch 157 starting the become-follower transition for partition __transaction_state-29 with leader 1266
[2021-08-31 18:22:30.102374] INFO logger:66 - [Broker id=1187] Follower __transaction_state-29 starts at leader epoch 222 from offset 2199146 with high watermark 2199146. Previous leader epoch was 221.
[2021-08-31 18:22:30.103121] INFO ReplicaFetcherManager:66 - [ReplicaFetcherManager on broker 1187] Removed fetcher for partitions Set(pmplatform_recon.reconciliation_state_updated.bson-3, __transaction_state-29)
[2021-08-31 18:22:30.106119] INFO ReplicaFetcherManager:66 - [ReplicaFetcherManager on broker 1187] Added fetcher to broker 1266 for partitions Map(__transaction_state-29 -> (offset=2199146, leaderEpoch=222))
[2021-08-31 18:22:30.106277] TRACE logger:54 - [Broker id=1187] Completed LeaderAndIsr request correlationId 9187 from controller 1214 epoch 157 for the become-follower transition for partition __transaction_state-29 with leader 1266
[2021-08-31 18:22:30.109601] INFO ReplicaFetcherThread:66 - [ReplicaFetcher replicaId=1187, leaderId=1266, fetcherId=1] Retrying leaderEpoch request for partition __transaction_state-29 as the leader reported an error: UNKNOWN_LEADER_EPOCH
[2021-08-31 18:22:30.109999] INFO TransactionStateManager:66 - [Transaction State Manager 1187]: No cached transaction metadata found for __transaction_state-29 during become-follower transition
[2021-08-31 18:22:30.123016] TRACE logger:54 - [Controller id=1214 epoch=157] Sending become-leader LeaderAndIsr request LeaderAndIsrPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], addingReplicas=[], removingReplicas=[], isNew=false) to broker 1266 for partition __transaction_state-29
[2021-08-31 18:22:30.133232] TRACE logger:54 - [Controller id=1214 epoch=157] Sending become-follower LeaderAndIsr request LeaderAndIsrPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], addingReplicas=[], removingReplicas=[], isNew=false) to broker 1232 for partition __transaction_state-29
[2021-08-31 18:22:30.134325] TRACE logger:54 - [Broker id=1232] Handling LeaderAndIsr request correlationId 8854 from controller 1214 epoch 157 starting the become-follower transition for partition __transaction_state-29 with leader 1266
[2021-08-31 18:22:30.134665] INFO logger:66 - [Broker id=1232] Follower __transaction_state-29 starts at leader epoch 222 from offset 2199146 with high watermark 2199146. Previous leader epoch was 221.
[2021-08-31 18:22:30.135064] INFO ReplicaFetcherManager:66 - [ReplicaFetcherManager on broker 1232] Removed fetcher for partitions Set(--removed--, __transaction_state-29)
[2021-08-31 18:22:30.137097] INFO ReplicaFetcherThread:66 - [ReplicaFetcher replicaId=1266, leaderId=1232, fetcherId=1] Partition __transaction_state-29 has an older epoch (221) than the current leader. Will await the new LeaderAndIsr state before resuming fetching.
[2021-08-31 18:22:30.137356] WARN ReplicaFetcherThread:70 - [ReplicaFetcher replicaId=1266, leaderId=1232, fetcherId=1] Partition __transaction_state-29 marked as failed
[2021-08-31 18:22:30.140900] INFO ReplicaFetcherManager:66 - [ReplicaFetcherManager on broker 1232] Added fetcher to broker 1266 for partitions Map(__transaction_state-29 -> (offset=2199146, leaderEpoch=222))
[2021-08-31 18:22:30.142692] TRACE logger:54 - [Broker id=1232] Completed LeaderAndIsr request correlationId 8854 from controller 1214 epoch 157 for the become-follower transition for partition __transaction_state-29 with leader 1266
[2021-08-31 18:22:30.142935] INFO ReplicaFetcherThread:66 - [ReplicaFetcher replicaId=1232, leaderId=1266, fetcherId=1] Retrying leaderEpoch request for partition __transaction_state-29 as the leader reported an error: UNKNOWN_LEADER_EPOCH
[2021-08-31 18:22:30.143842] INFO TransactionStateManager:66 - [Transaction State Manager 1232]: Unloaded transaction metadata TxnMetadataCacheEntry(coordinatorEpoch=220, numTransactionalEntries=21) for __transaction_state-29 on become-follower transition
[2021-08-31 18:22:30.146822] TRACE logger:54 - [Broker id=1266] Handling LeaderAndIsr request correlationId 8863 from controller 1214 epoch 157 starting the become-leader transition for partition __transaction_state-29
[2021-08-31 18:22:30.147722] TRACE logger:54 - [Broker id=1237] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8873
[2021-08-31 18:22:30.148472] TRACE logger:54 - [Broker id=1196] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8920
[2021-08-31 18:22:30.148704] TRACE logger:54 - [Broker id=1243] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8833
[2021-08-31 18:22:30.148825] TRACE logger:54 - [Broker id=1231] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8834
[2021-08-31 18:22:30.148852] TRACE logger:54 - [Broker id=1206] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9115
[2021-08-31 18:22:30.148997] TRACE logger:54 - [Broker id=1260] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9109
[2021-08-31 18:22:30.148999] TRACE logger:54 - [Broker id=1275] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8895
[2021-08-31 18:22:30.149063] TRACE logger:54 - [Broker id=1254] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8818
[2021-08-31 18:22:30.149100] TRACE logger:54 - [Broker id=1195] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9049
[2021-08-31 18:22:30.149123] TRACE logger:54 - [Broker id=1187] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9188
[2021-08-31 18:22:30.149166] TRACE logger:54 - [Broker id=1262] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8804
[2021-08-31 18:22:30.149449] TRACE logger:54 - [Broker id=1218] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9308
[2021-08-31 18:22:30.149573] TRACE logger:54 - [Broker id=1258] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8818
[2021-08-31 18:22:30.149793] TRACE logger:54 - [Broker id=1250] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9040
[2021-08-31 18:22:30.149805] TRACE logger:54 - [Broker id=1256] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9029
[2021-08-31 18:22:30.150028] TRACE logger:54 - [Broker id=1210] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8934
[2021-08-31 18:22:30.150155] TRACE logger:54 - [Broker id=1264] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8769
[2021-08-31 18:22:30.150325] TRACE logger:54 - [Broker id=1268] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9103
[2021-08-31 18:22:30.150325] TRACE logger:54 - [Broker id=1216] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9153
[2021-08-31 18:22:30.150353] TRACE logger:54 - [Broker id=1271] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8747
[2021-08-31 18:22:30.150385] TRACE logger:54 - [Broker id=1252] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8778
[2021-08-31 18:22:30.150530] TRACE logger:54 - [Broker id=1198] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9072
[2021-08-31 18:22:30.150534] TRACE logger:54 - [Broker id=1242] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8837
[2021-08-31 18:22:30.150718] TRACE logger:54 - [Broker id=1200] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9093
[2021-08-31 18:22:30.150802] TRACE logger:54 - [Broker id=1235] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8769
[2021-08-31 18:22:30.150817] TRACE logger:54 - [Broker id=1219] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9263
[2021-08-31 18:22:30.150926] TRACE logger:54 - [Broker id=1204] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8897
[2021-08-31 18:22:30.151076] TRACE logger:54 - [Broker id=1265] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8758
[2021-08-31 18:22:30.151112] TRACE logger:54 - [Broker id=1227] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9086
[2021-08-31 18:22:30.151216] TRACE logger:54 - [Broker id=1191] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8993
[2021-08-31 18:22:30.151270] TRACE logger:54 - [Broker id=1221] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8826
[2021-08-31 18:22:30.151368] TRACE logger:54 - [Broker id=1192] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9107
[2021-08-31 18:22:30.151380] TRACE logger:54 - [Broker id=1213] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8857
[2021-08-31 18:22:30.151407] TRACE logger:54 - [Broker id=1269] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9035
[2021-08-31 18:22:30.151461] TRACE logger:54 - [Broker id=1277] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8889
[2021-08-31 18:22:30.151505] TRACE logger:54 - [Broker id=1228] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8829
[2021-08-31 18:22:30.151651] TRACE logger:54 - [Broker id=1217] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9351
[2021-08-31 18:22:30.151866] TRACE logger:54 - [Broker id=1236] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9099
[2021-08-31 18:22:30.151918] TRACE logger:54 - [Broker id=1280] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 10851
[2021-08-31 18:22:30.151979] TRACE logger:54 - [Broker id=1267] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9049
[2021-08-31 18:22:30.152051] TRACE logger:54 - [Broker id=1222] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8855
[2021-08-31 18:22:30.152085] TRACE logger:54 - [Broker id=1244] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9438
[2021-08-31 18:22:30.152126] TRACE logger:54 - [Broker id=1240] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9065
[2021-08-31 18:22:30.152202] TRACE logger:54 - [Broker id=1261] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9060
[2021-08-31 18:22:30.152309] TRACE logger:54 - [Broker id=1272] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9140
[2021-08-31 18:22:30.152364] INFO ReplicaFetcherManager:66 - [ReplicaFetcherManager on broker 1266] Removed fetcher for partitions Set(--removed--, __transaction_state-29, --removed--)
[2021-08-31 18:22:30.152415] TRACE logger:54 - [Broker id=1257] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8777
[2021-08-31 18:22:30.152462] TRACE logger:54 - [Broker id=1276] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8826
[2021-08-31 18:22:30.152588] TRACE logger:54 - [Broker id=1259] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9011
[2021-08-31 18:22:30.152766] TRACE logger:54 - [Broker id=1224] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8819
[2021-08-31 18:22:30.152830] TRACE logger:54 - [Broker id=1193] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8864
[2021-08-31 18:22:30.153018] TRACE logger:54 - [Broker id=1247] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8998
[2021-08-31 18:22:30.153084] TRACE logger:54 - [Broker id=1215] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9120
[2021-08-31 18:22:30.153107] TRACE logger:54 - [Broker id=1197] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9134
[2021-08-31 18:22:30.153163] TRACE logger:54 - [Broker id=1188] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9179
[2021-08-31 18:22:30.153211] TRACE logger:54 - [Broker id=1274] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9084
[2021-08-31 18:22:30.153270] TRACE logger:54 - [Broker id=1263] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9095
[2021-08-31 18:22:30.153296] TRACE logger:54 - [Broker id=1251] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9005
[2021-08-31 18:22:30.153428] TRACE logger:54 - [Broker id=1201] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8841
[2021-08-31 18:22:30.153569] TRACE logger:54 - [Broker id=1248] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9105
[2021-08-31 18:22:30.153641] TRACE logger:54 - [Broker id=1246] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9142
[2021-08-31 18:22:30.153701] TRACE logger:54 - [Broker id=1209] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8831
[2021-08-31 18:22:30.153847] TRACE logger:54 - [Broker id=1202] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8792
[2021-08-31 18:22:30.153995] TRACE logger:54 - [Broker id=1203] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8857
[2021-08-31 18:22:30.154230] TRACE logger:54 - [Broker id=1207] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9060
[2021-08-31 18:22:30.154265] TRACE logger:54 - [Broker id=1186] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9167
[2021-08-31 18:22:30.154301] TRACE logger:54 - [Broker id=1255] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8765
[2021-08-31 18:22:30.154673] TRACE logger:54 - [Broker id=1253] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8792
[2021-08-31 18:22:30.154771] TRACE logger:54 - [Broker id=1220] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9128
[2021-08-31 18:22:30.154827] TRACE logger:54 - [Broker id=1239] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9145
[2021-08-31 18:22:30.155245] TRACE logger:54 - [Broker id=1233] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9138
[2021-08-31 18:22:30.156050] TRACE logger:54 - [Broker id=1223] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8850
[2021-08-31 18:22:30.156134] TRACE logger:54 - [Broker id=1278] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8826
[2021-08-31 18:22:30.156245] TRACE logger:54 - [Broker id=1232] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8855
[2021-08-31 18:22:30.156833] TRACE logger:54 - [Broker id=1270] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9109
[2021-08-31 18:22:30.156855] TRACE logger:54 - [Broker id=1226] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9124
[2021-08-31 18:22:30.157341] TRACE logger:54 - [Broker id=1205] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9189
[2021-08-31 18:22:30.157533] TRACE logger:54 - [Broker id=1211] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9074
[2021-08-31 18:22:30.159415] INFO KafkaController:66 - [Controller id=1214] Partition __transaction_state-29 completed preferred replica leader election. New leader is 1266
[2021-08-31 18:22:30.160907] TRACE logger:54 - [Broker id=1241] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8866
[2021-08-31 18:22:30.163844] TRACE logger:54 - [Broker id=1184] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8952
[2021-08-31 18:22:30.165204] TRACE logger:54 - [Broker id=1214] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 9187
[2021-08-31 18:22:30.179043] TRACE logger:54 - [Broker id=1238] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8839
[2021-08-31 18:22:30.179435] TRACE logger:54 - [Broker id=1185] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8954
[2021-08-31 18:22:30.182935] TRACE logger:54 - [Broker id=1199] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8785
[2021-08-31 18:22:30.186117] INFO logger:66 - [Broker id=1266] Leader __transaction_state-29 starts at leader epoch 222 from offset 2199146 with high watermark 2199146 ISR [1187,1232,1266] addingReplicas [] removingReplicas []. Previous leader epoch was 221.
[2021-08-31 18:22:30.230864] TRACE logger:54 - [Broker id=1266] Completed LeaderAndIsr request correlationId 8863 from controller 1214 epoch 157 for the become-leader transition for partition __transaction_state-29
[2021-08-31 18:22:30.355035] INFO TransactionStateManager:66 - [Transaction State Manager 1266]: Loading transaction metadata from __transaction_state-29 at epoch 222
[2021-08-31 18:22:30.370655] TRACE logger:54 - [Broker id=1266] Cached leader info UpdateMetadataPartitionState(topicName='__transaction_state', partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187], offlineReplicas=[]) for partition __transaction_state-29 in response to UpdateMetadata request sent by controller 1214 epoch 157 with correlation id 8864
[2021-08-31 18:22:31.111207] INFO Log:66 - [Log partition=__transaction_state-29, dir=/pay/kafka/data] Truncating to 2199146 has no effect as the largest offset in the log is 2199145
[2021-08-31 18:22:31.145411] INFO Log:66 - [Log partition=__transaction_state-29, dir=/pay/kafka/data] Truncating to 2199146 has no effect as the largest offset in the log is 2199145
[2021-08-31 18:22:33.362421] INFO TransactionStateManager:66 - [Transaction State Manager 1266]: Finished loading 21 transaction metadata from __transaction_state-29 in 3008 milliseconds, of which 0 milliseconds was spent in the scheduler.
[2021-08-31 18:22:33.362600] INFO TransactionStateManager:66 - [Transaction State Manager 1266]: Completed loading transaction metadata from __transaction_state-29 for coordinator epoch 222
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)