[jira] [Updated] (KUDU-2180) Change data capture (CDC) WAL-tailing support

["Grant Henke (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Grant Henke updated KUDU-2180:
------------------------------
    Labels: roadmap-candidate  (was: )

> Change data capture (CDC) WAL-tailing support
> ---------------------------------------------
>
>                 Key: KUDU-2180
>                 URL: https://issues.apache.org/jira/browse/KUDU-2180
>             Project: Kudu
>          Issue Type: New Feature
>          Components: api, consensus, master, tablet, tserver
>            Reporter: Mike Percy
>            Priority: Major
>              Labels: roadmap-candidate
>
> This is a tracking JIRA for change data capture (CDC) support in Kudu.
> This feature is not trivial and would need a significant amount of design work.
> Here are some high-level considerations and potential requirements / approaches:
> * New CDC Subscription APIs (remote and client)
> * Support for distributed consumers (necessary if subscribing to all cluster changes at scale)
> * Consensus support for permanent non-voters (in order to have a replication target)
> * Ability to retain WAL segments if a CDC subscriber has not yet caught up
> * Ability to re-sync subscribers that have fallen behind the leader's WAL. Required to support a new subscriber joining after the cluster has been running for a long time.
> In order to support re-syncing subscribers that have fallen behind the leader's WAL, we may be able to build a bridge API for a consumer to ingest the results of a tablet copy operation, without having to be a native Kudu tablet server itself.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)