[jira] [Commented] (KAFKA-5202) Handle topic deletion for ongoing transactions

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022083#comment-16022083 ] 

ASF GitHub Bot commented on KAFKA-5202:
---------------------------------------

GitHub user guozhangwang opened a pull request:

    https://github.com/apache/kafka/pull/3130

    KAFKA-5202: Handle topic deletion while trying to send txn markers

    Here is the sketch of this proposal:
    
    1. When handling `addPartition` request, the coordinator will check if all its corresponding metadata exist in the metadata cache; if there is ANY one of the partitions whose metadata is not available, return UNKNOWN_TOPIC_OR_PARTITION.
    
    2. When it is time to send the txn markers, only look for the leader node of the partition once instead of retrying, and if that information is not available, it means the partition is highly likely been removed since it was in the cache before. In this case, we just remove the partition from the metadata object and skip putting into the corresponding queue, and if all partitions' leader broker are non-available, complete this delayed operation to proceed to write the complete txn log entry.
    
    One caveat of this approach is the delete-and-recreate case, and the argument is that since all the messages are deleted anyways when deleting the topic-partition, it does not matter whether the markers are on the log partitions or not.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/guozhangwang/kafka K5202-handle-topic-deletion

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/3130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3130
    
----
commit 733f352bd9be29d1439844271f22508a2e4b2552
Author: Guozhang Wang <wa...@gmail.com>
Date:   2017-05-23T23:23:45Z

    handle topic deletion

----


> Handle topic deletion for ongoing transactions
> ----------------------------------------------
>
>                 Key: KAFKA-5202
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5202
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: clients, core, producer 
>            Reporter: Jason Gustafson
>            Assignee: Guozhang Wang
>            Priority: Blocker
>              Labels: exactly-once
>             Fix For: 0.11.0.0
>
>
> If a topic is deleted, we should remove its partitions from ongoing transactions.  If the transaction has already begun rolling forward, we have to let it continue for the rest of the included partitions, but we have the option of failing a transaction which is yet to be committed or aborted.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)