You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by guozhangwang <gi...@git.apache.org> on 2017/05/03 02:38:02 UTC

[GitHub] kafka pull request #2964: KAFKA-5130: Refactor TC In-memory Cache

GitHub user guozhangwang opened a pull request:

    https://github.com/apache/kafka/pull/2964

    KAFKA-5130: Refactor TC In-memory Cache

    1. Collapsed the `ownedPartitions`, `pendingTxnMap` and the `transactionMetadataCache` into a single in-memory structure, which is a two-layered map: first keyed by the transactionTxnLog, and then valued with the current coordinatorEpoch of that map plus another map keyed by the transactional id.
    
    2. Use `transactionalId` across the modules in transactional coordinator, attach this id with the transactional marker entries.
    
    3. Use two keys: `transactionalId` and `txnLogPartitionId` in the writeMarkerPurgatory as well as passing it along with the TxnMarkerEntry, so that `TransactionMarkerRequestCompletionHandler` can use it to access the two-layered map upon getting responses.
    
    4. Use one queue per `broker-id` and `txnLogPartitionId`. Also when there is a possible update on the end point associated with the `broker-id`, update the Node without clearing the queue but relying on the requests to retry in the next round.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/guozhangwang/kafka K5130-refactor-tc-inmemory-cache

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/2964.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2964
    
----
commit 758a74c835a6fb32ae4c142b276699820644f068
Author: Guozhang Wang <wa...@gmail.com>
Date:   2017-05-02T18:09:35Z

    collapse ownedPartition with MetadataCache

commit 3495d13e69123d6097c4f05bcae5f2bdcf7ccff5
Author: Guozhang Wang <wa...@gmail.com>
Date:   2017-05-03T01:21:57Z

    centralize callbacks for send markers and append to log

commit c6548003d7c835857a226e57810b104a8501645e
Author: Guozhang Wang <wa...@gmail.com>
Date:   2017-05-03T01:39:09Z

    minor refactoring

commit 218a572e1c612bad5f9a15d90d37b7c6c2b29d99
Author: Guozhang Wang <wa...@gmail.com>
Date:   2017-05-03T02:18:42Z

    rebase from trunk

commit 19f033e69091c5a656a473374ea6031d01fcd061
Author: Guozhang Wang <wa...@gmail.com>
Date:   2017-05-03T02:32:41Z

    minor fixes

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] kafka pull request #2964: KAFKA-5130: Refactor TC In-memory Cache

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/kafka/pull/2964


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---