You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Roman Schmitz (Jira)" <ji...@apache.org> on 2023/10/15 20:05:00 UTC

[jira] [Commented] (KAFKA-15597) Enable Connect DropHeaders SMT to drop headers on a wildcard/regexp-basis

    [ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775494#comment-17775494 ] 

Roman Schmitz commented on KAFKA-15597:
---------------------------------------

KIp created: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=272927649

> Enable Connect DropHeaders SMT to drop headers on a wildcard/regexp-basis
> -------------------------------------------------------------------------
>
>                 Key: KAFKA-15597
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15597
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>            Reporter: Roman Schmitz
>            Priority: Minor
>              Labels: connect-transformation
>
> In many use cases you might not only want to drop a few specific Kafka headers but a set of headers whose names can also dynamically change (e.g. when used with some end-to-end-encryption libraries, tracing etc.). To prevent those headers following a special pattern (which may not comply with downstream system format) to be further forwarded/processed downstream (e.g.header forwarding in Http Sinks), I suggest to add regexp matching to the *apply* method instead of a set-based {*}contains{*}. Link to the relevant code:
> [https://github.com/apache/kafka/blob/7b5d640cc656443a078bda096d01910b3edfdb37/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/DropHeaders.java#L54 |http://example.com/]
> One common example is forwarding of encryption-headers that contain byte arrays to a HTTP server via HTTP Sinks which will not be accepted by the HTTP server.
> Example implementation done - link to PR: [https://github.com/apache/kafka/pull/14536]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)