You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Arun Potharaju (Jira)" <ji...@apache.org> on 2023/04/06 13:11:00 UTC
[jira] [Assigned] (KAFKA-14873) Pluggable storage for Kafka Connect internal topics
[ https://issues.apache.org/jira/browse/KAFKA-14873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun Potharaju reassigned KAFKA-14873:
--------------------------------------
Assignee: Arun Potharaju
> Pluggable storage for Kafka Connect internal topics
> ---------------------------------------------------
>
> Key: KAFKA-14873
> URL: https://issues.apache.org/jira/browse/KAFKA-14873
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Malthe Borch
> Assignee: Arun Potharaju
> Priority: Major
> Labels: needs-kip
>
> The Kafka Connect framework relies on compacted topics to store config, offset and status information for each connector.
> This conflates two kinds of data, control and content, which some people disagree with. Notably, [Azure Event Hub|https://learn.microsoft.com/en-us/azure/event-hubs/log-compaction] does not (or _did not_, because there's currently a preview release out which does have support for compacted topics albeit only at the more expensive premium tiers).
> In some deployments, it may be desirable to use a different backend for these control settings (which essentially take a key/value form), for example [Azure Table Storage|https://learn.microsoft.com/en-us/rest/api/storageservices/table-service-rest-api] – basically any key/value store that provides the Write-If-Matches primitive to update a key only if the current value matches a known value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)