[jira] [Created] (KAFKA-5247) Consumer GroupCoordinator should continue to materialize committed offsets in offset order even for transactional offset commits

["Apurva Mehta (JIRA)" <ji...@apache.org>]

Apurva Mehta created KAFKA-5247:
-----------------------------------

             Summary: Consumer GroupCoordinator should continue to materialize committed offsets in offset order even for transactional offset commits
                 Key: KAFKA-5247
                 URL: https://issues.apache.org/jira/browse/KAFKA-5247
             Project: Kafka
          Issue Type: Bug
            Reporter: Apurva Mehta
             Fix For: 0.11.0.0


In the TxnOffsetCommit patch, we thought it was ok for the group coordinator to use "transaction order" semantics when updating the cache, but we weren't thinking about the log cleaner.

The log cleaner uses offset order when cleaning which means that the key with the largest offset always wins. So if we use transaction order when dynamically updating the cache, we will get different results from when we're loading the cache (even if the loading logic also uses transaction order).

The fix should be straightforward: we need to remember the offset in the offsets topic of the offset that we cache. Then we only update it if the new entry has a higher offset.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)