[jira] [Updated] (KAFKA-21008) Consider allowing record headers to be filtered or masked before persistence in Kafka Streams state stores

["Matthias J. Sax (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-21008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias J. Sax updated KAFKA-21008:
------------------------------------
    Component/s: streams

> Consider allowing record headers to be filtered or masked before persistence in Kafka Streams state stores
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-21008
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21008
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: sanghyeok An
>            Assignee: sanghyeok An
>            Priority: Minor
>              Labels: needs-kip, streams
>
> Kafka Streams header-aware state stores introduced by KIP-1271 can persist record headers together with state values and propagate them through state store changelogs.
>  
> Record headers may contain application-level sensitive information, such as credentials, tokens, personally identifiable information, or request metadata that should not be retained at rest.
>  
> Currently, Kafka Streams does not provide a built-in mechanism to filter, remove, or mask selected headers before they are written to a header-aware state store. Applications must sanitize headers manually before every stateful operator, which is error-prone and may also modify the headers observed by downstream processors.
>  
> Kafka Streams should provide an opt-in mechanism to filter or transform the copy of the headers that is persisted by header-aware state stores.
>  
> The mechanism should:
>  *  Allow selected headers to be removed or their values to be masked.
>  * Apply consistently to both local state and state-store changelog records.
>  * Avoid mutating the headers of the in-flight input record.
>  * Preserve the current behavior when no filtering or masking policy is configured.
>  * Ensure that restored state contains the same sanitized headers that were written to the local store.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)