[jira] [Updated] (FLINK-3692) Develop a Kafka state backend

["Flink Jira Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Flink Jira Bot updated FLINK-3692:
----------------------------------
      Labels: auto-deprioritized-major auto-deprioritized-minor  (was: auto-deprioritized-major stale-minor)
    Priority: Not a Priority  (was: Minor)

This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion.


> Develop a Kafka state backend
> -----------------------------
>
>                 Key: FLINK-3692
>                 URL: https://issues.apache.org/jira/browse/FLINK-3692
>             Project: Flink
>          Issue Type: New Feature
>          Components: Runtime / State Backends
>            Reporter: Elias Levy
>            Priority: Not a Priority
>              Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> Flink clusters usually consume of a Kafka cluster.  It simplify operations if Flink could store its state checkpoints in Kafka.  This should be possibly using different topics to write to, partitioning appropriately, and using compacted topics.  This would avoid the need to run an HDFS cluster just to store Flink checkpoints.
> For inspiration you may want to take a look at how Samza checkpoints a task's local state to a Kafka topic, and how the newer Kafka consumers checkpoint their offsets to Kafka.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)