You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Sophie Blee-Goldman (Jira)" <ji...@apache.org> on 2020/06/01 19:47:00 UTC

[jira] [Created] (KAFKA-10079) Support high availability for in-memory stores

Sophie Blee-Goldman created KAFKA-10079:
-------------------------------------------

             Summary: Support high availability for in-memory stores
                 Key: KAFKA-10079
                 URL: https://issues.apache.org/jira/browse/KAFKA-10079
             Project: Kafka
          Issue Type: Bug
          Components: streams
            Reporter: Sophie Blee-Goldman
            Assignee: Sophie Blee-Goldman
             Fix For: 2.6.0


Now that KAFKA-9501 is fixed, we should make sure that KIP-441 will work with in-memory stores.

Our current technique for reporting offsets, locking all task directories and searching for offsets in the written checkpoint file, only works for persistent stores. We should make sure to compute offset sums for in-memory stores as well, for which we don't write a checkpoint file.

We should also improve the thread-level stickiness of stateful tasks, since we don't handoff stores between threads and thus any in-memory state will be wiped out if transferred between threads.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)