You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/05/03 02:38:04 UTC
[jira] [Commented] (KAFKA-5130) Change InterBrokerSendThread to use
a Queue per broker
[ https://issues.apache.org/jira/browse/KAFKA-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994187#comment-15994187 ]
ASF GitHub Bot commented on KAFKA-5130:
---------------------------------------
GitHub user guozhangwang opened a pull request:
https://github.com/apache/kafka/pull/2964
KAFKA-5130: Refactor TC In-memory Cache
1. Collapsed the `ownedPartitions`, `pendingTxnMap` and the `transactionMetadataCache` into a single in-memory structure, which is a two-layered map: first keyed by the transactionTxnLog, and then valued with the current coordinatorEpoch of that map plus another map keyed by the transactional id.
2. Use `transactionalId` across the modules in transactional coordinator, attach this id with the transactional marker entries.
3. Use two keys: `transactionalId` and `txnLogPartitionId` in the writeMarkerPurgatory as well as passing it along with the TxnMarkerEntry, so that `TransactionMarkerRequestCompletionHandler` can use it to access the two-layered map upon getting responses.
4. Use one queue per `broker-id` and `txnLogPartitionId`. Also when there is a possible update on the end point associated with the `broker-id`, update the Node without clearing the queue but relying on the requests to retry in the next round.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/guozhangwang/kafka K5130-refactor-tc-inmemory-cache
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2964.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2964
----
commit 758a74c835a6fb32ae4c142b276699820644f068
Author: Guozhang Wang <wa...@gmail.com>
Date: 2017-05-02T18:09:35Z
collapse ownedPartition with MetadataCache
commit 3495d13e69123d6097c4f05bcae5f2bdcf7ccff5
Author: Guozhang Wang <wa...@gmail.com>
Date: 2017-05-03T01:21:57Z
centralize callbacks for send markers and append to log
commit c6548003d7c835857a226e57810b104a8501645e
Author: Guozhang Wang <wa...@gmail.com>
Date: 2017-05-03T01:39:09Z
minor refactoring
commit 218a572e1c612bad5f9a15d90d37b7c6c2b29d99
Author: Guozhang Wang <wa...@gmail.com>
Date: 2017-05-03T02:18:42Z
rebase from trunk
commit 19f033e69091c5a656a473374ea6031d01fcd061
Author: Guozhang Wang <wa...@gmail.com>
Date: 2017-05-03T02:32:41Z
minor fixes
----
> Change InterBrokerSendThread to use a Queue per broker
> ------------------------------------------------------
>
> Key: KAFKA-5130
> URL: https://issues.apache.org/jira/browse/KAFKA-5130
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Reporter: Damian Guy
>
> Change the {{InterBrokerSendThread}} to use a queue per broker and only attempt to send to brokers that are ready
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)