You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Roman Schmitz <ro...@gmail.com> on 2023/10/15 20:02:24 UTC
KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp
Hi all,
While working with different customers I came across the case several times
that we'd like to not only explicitly remove headers by name but by pattern
/ regexp. Here is a KIP for this feature!
Please let me know if you have any comments, questions, or suggestions!
https://cwiki.apache.org/confluence/x/oYtEE
Thanks,
Roman
Re: KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp
Posted by Roman Schmitz <ro...@gmail.com>.
Hi,
Thanks, 100% agree, I fixed the description.
Thanks,
Roman
Am Do., 11. Jan. 2024 um 18:27 Uhr schrieb Mickael Maison <
mickael.maison@gmail.com>:
> Hi Roman,
>
> Thanks for the updates, this looks much better.
>
> Just a couple of small comments:
> - The type of the field is listed as "boolean". I think it should be
> string (or list)
> - Should the field be named "headers.patterns" instead of
> "headers.pattern" since it accepts a list of patterns?
>
> Thanks,
> Mickael
>
> On Thu, Jan 11, 2024 at 12:56 PM Roman Schmitz <ro...@gmail.com>
> wrote:
> >
> > Hi Mickael,
> > Hi all,
> >
> > Thanks for the feedback!
> > I have adapted the KIP description - actually much shorter and just
> > reflecting the general functionality and interface/configuration changes.
> >
> > Kindly let me know if you have any comments, questions, or suggestions
> for
> > this KIP!
> >
> > Thanks,
> > Roman
> >
> > Am Fr., 5. Jan. 2024 um 17:36 Uhr schrieb Mickael Maison <
> > mickael.maison@gmail.com>:
> >
> > > Hi Roman,
> > >
> > > Thanks for the KIP! This would be a useful improvement.
> > >
> > > Ideally you want to make a concerte proposal in the KIP instead of
> > > listing a series of options. Currently the KIP seems to list two
> > > alternatives.
> > >
> > > Also a KIP focuses on the API changes rather than on the pure
> > > implementation. It seems you're proposing adding a configuration to
> > > the DropHeaders SMT. It would be good to describe that new
> > > configuration. For example see KIP-911 which also added a
> > > configuration.
> > >
> > > Thanks,
> > > Mickael
> > >
> > > On Mon, Oct 16, 2023 at 9:50 AM Roman Schmitz <roman.schmitz@gmail.com
> >
> > > wrote:
> > > >
> > > > Hi Andrew,
> > > >
> > > > Ok, thanks for the feedback! I added a few more details and code
> examples
> > > > to explain the proposed changes.
> > > >
> > > > Thanks,
> > > > Roman
> > > >
> > > > Am So., 15. Okt. 2023 um 22:12 Uhr schrieb Andrew Schofield <
> > > > andrew_schofield_jira@outlook.com>:
> > > >
> > > > > Hi Roman,
> > > > > Thanks for the KIP. I think it’s an interesting idea, but I think
> the
> > > KIP
> > > > > document needs some
> > > > > more details added before it’s ready for review. For example,
> here’s a
> > > KIP
> > > > > in the same
> > > > > area which was delivered in an earlier version of Kafka. I think
> this
> > > is a
> > > > > good KIP to copy
> > > > > for a suitable level of detail and description (
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Filter+and+Conditional+SMTs
> > > > > ).
> > > > >
> > > > > Hope this helps.
> > > > >
> > > > > Thanks,
> > > > > Andrew
> > > > >
> > > > > > On 15 Oct 2023, at 21:02, Roman Schmitz <roman.schmitz@gmail.com
> >
> > > wrote:
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > While working with different customers I came across the case
> several
> > > > > times
> > > > > > that we'd like to not only explicitly remove headers by name but
> by
> > > > > pattern
> > > > > > / regexp. Here is a KIP for this feature!
> > > > > >
> > > > > > Please let me know if you have any comments, questions, or
> > > suggestions!
> > > > > >
> > > > > > https://cwiki.apache.org/confluence/x/oYtEE
> > > > > >
> > > > > > Thanks,
> > > > > > Roman
> > > > >
> > > > >
> > >
>
Re: KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp
Posted by Mickael Maison <mi...@gmail.com>.
Hi Roman,
Thanks for the updates, this looks much better.
Just a couple of small comments:
- The type of the field is listed as "boolean". I think it should be
string (or list)
- Should the field be named "headers.patterns" instead of
"headers.pattern" since it accepts a list of patterns?
Thanks,
Mickael
On Thu, Jan 11, 2024 at 12:56 PM Roman Schmitz <ro...@gmail.com> wrote:
>
> Hi Mickael,
> Hi all,
>
> Thanks for the feedback!
> I have adapted the KIP description - actually much shorter and just
> reflecting the general functionality and interface/configuration changes.
>
> Kindly let me know if you have any comments, questions, or suggestions for
> this KIP!
>
> Thanks,
> Roman
>
> Am Fr., 5. Jan. 2024 um 17:36 Uhr schrieb Mickael Maison <
> mickael.maison@gmail.com>:
>
> > Hi Roman,
> >
> > Thanks for the KIP! This would be a useful improvement.
> >
> > Ideally you want to make a concerte proposal in the KIP instead of
> > listing a series of options. Currently the KIP seems to list two
> > alternatives.
> >
> > Also a KIP focuses on the API changes rather than on the pure
> > implementation. It seems you're proposing adding a configuration to
> > the DropHeaders SMT. It would be good to describe that new
> > configuration. For example see KIP-911 which also added a
> > configuration.
> >
> > Thanks,
> > Mickael
> >
> > On Mon, Oct 16, 2023 at 9:50 AM Roman Schmitz <ro...@gmail.com>
> > wrote:
> > >
> > > Hi Andrew,
> > >
> > > Ok, thanks for the feedback! I added a few more details and code examples
> > > to explain the proposed changes.
> > >
> > > Thanks,
> > > Roman
> > >
> > > Am So., 15. Okt. 2023 um 22:12 Uhr schrieb Andrew Schofield <
> > > andrew_schofield_jira@outlook.com>:
> > >
> > > > Hi Roman,
> > > > Thanks for the KIP. I think it’s an interesting idea, but I think the
> > KIP
> > > > document needs some
> > > > more details added before it’s ready for review. For example, here’s a
> > KIP
> > > > in the same
> > > > area which was delivered in an earlier version of Kafka. I think this
> > is a
> > > > good KIP to copy
> > > > for a suitable level of detail and description (
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Filter+and+Conditional+SMTs
> > > > ).
> > > >
> > > > Hope this helps.
> > > >
> > > > Thanks,
> > > > Andrew
> > > >
> > > > > On 15 Oct 2023, at 21:02, Roman Schmitz <ro...@gmail.com>
> > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > While working with different customers I came across the case several
> > > > times
> > > > > that we'd like to not only explicitly remove headers by name but by
> > > > pattern
> > > > > / regexp. Here is a KIP for this feature!
> > > > >
> > > > > Please let me know if you have any comments, questions, or
> > suggestions!
> > > > >
> > > > > https://cwiki.apache.org/confluence/x/oYtEE
> > > > >
> > > > > Thanks,
> > > > > Roman
> > > >
> > > >
> >
Re: KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp
Posted by Roman Schmitz <ro...@gmail.com>.
Hi Mickael,
Hi all,
Thanks for the feedback!
I have adapted the KIP description - actually much shorter and just
reflecting the general functionality and interface/configuration changes.
Kindly let me know if you have any comments, questions, or suggestions for
this KIP!
Thanks,
Roman
Am Fr., 5. Jan. 2024 um 17:36 Uhr schrieb Mickael Maison <
mickael.maison@gmail.com>:
> Hi Roman,
>
> Thanks for the KIP! This would be a useful improvement.
>
> Ideally you want to make a concerte proposal in the KIP instead of
> listing a series of options. Currently the KIP seems to list two
> alternatives.
>
> Also a KIP focuses on the API changes rather than on the pure
> implementation. It seems you're proposing adding a configuration to
> the DropHeaders SMT. It would be good to describe that new
> configuration. For example see KIP-911 which also added a
> configuration.
>
> Thanks,
> Mickael
>
> On Mon, Oct 16, 2023 at 9:50 AM Roman Schmitz <ro...@gmail.com>
> wrote:
> >
> > Hi Andrew,
> >
> > Ok, thanks for the feedback! I added a few more details and code examples
> > to explain the proposed changes.
> >
> > Thanks,
> > Roman
> >
> > Am So., 15. Okt. 2023 um 22:12 Uhr schrieb Andrew Schofield <
> > andrew_schofield_jira@outlook.com>:
> >
> > > Hi Roman,
> > > Thanks for the KIP. I think it’s an interesting idea, but I think the
> KIP
> > > document needs some
> > > more details added before it’s ready for review. For example, here’s a
> KIP
> > > in the same
> > > area which was delivered in an earlier version of Kafka. I think this
> is a
> > > good KIP to copy
> > > for a suitable level of detail and description (
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Filter+and+Conditional+SMTs
> > > ).
> > >
> > > Hope this helps.
> > >
> > > Thanks,
> > > Andrew
> > >
> > > > On 15 Oct 2023, at 21:02, Roman Schmitz <ro...@gmail.com>
> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > While working with different customers I came across the case several
> > > times
> > > > that we'd like to not only explicitly remove headers by name but by
> > > pattern
> > > > / regexp. Here is a KIP for this feature!
> > > >
> > > > Please let me know if you have any comments, questions, or
> suggestions!
> > > >
> > > > https://cwiki.apache.org/confluence/x/oYtEE
> > > >
> > > > Thanks,
> > > > Roman
> > >
> > >
>
Re: KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp
Posted by Mickael Maison <mi...@gmail.com>.
Hi Roman,
Thanks for the KIP! This would be a useful improvement.
Ideally you want to make a concerte proposal in the KIP instead of
listing a series of options. Currently the KIP seems to list two
alternatives.
Also a KIP focuses on the API changes rather than on the pure
implementation. It seems you're proposing adding a configuration to
the DropHeaders SMT. It would be good to describe that new
configuration. For example see KIP-911 which also added a
configuration.
Thanks,
Mickael
On Mon, Oct 16, 2023 at 9:50 AM Roman Schmitz <ro...@gmail.com> wrote:
>
> Hi Andrew,
>
> Ok, thanks for the feedback! I added a few more details and code examples
> to explain the proposed changes.
>
> Thanks,
> Roman
>
> Am So., 15. Okt. 2023 um 22:12 Uhr schrieb Andrew Schofield <
> andrew_schofield_jira@outlook.com>:
>
> > Hi Roman,
> > Thanks for the KIP. I think it’s an interesting idea, but I think the KIP
> > document needs some
> > more details added before it’s ready for review. For example, here’s a KIP
> > in the same
> > area which was delivered in an earlier version of Kafka. I think this is a
> > good KIP to copy
> > for a suitable level of detail and description (
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Filter+and+Conditional+SMTs
> > ).
> >
> > Hope this helps.
> >
> > Thanks,
> > Andrew
> >
> > > On 15 Oct 2023, at 21:02, Roman Schmitz <ro...@gmail.com> wrote:
> > >
> > > Hi all,
> > >
> > > While working with different customers I came across the case several
> > times
> > > that we'd like to not only explicitly remove headers by name but by
> > pattern
> > > / regexp. Here is a KIP for this feature!
> > >
> > > Please let me know if you have any comments, questions, or suggestions!
> > >
> > > https://cwiki.apache.org/confluence/x/oYtEE
> > >
> > > Thanks,
> > > Roman
> >
> >
Re: KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp
Posted by Roman Schmitz <ro...@gmail.com>.
Hi Andrew,
Ok, thanks for the feedback! I added a few more details and code examples
to explain the proposed changes.
Thanks,
Roman
Am So., 15. Okt. 2023 um 22:12 Uhr schrieb Andrew Schofield <
andrew_schofield_jira@outlook.com>:
> Hi Roman,
> Thanks for the KIP. I think it’s an interesting idea, but I think the KIP
> document needs some
> more details added before it’s ready for review. For example, here’s a KIP
> in the same
> area which was delivered in an earlier version of Kafka. I think this is a
> good KIP to copy
> for a suitable level of detail and description (
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Filter+and+Conditional+SMTs
> ).
>
> Hope this helps.
>
> Thanks,
> Andrew
>
> > On 15 Oct 2023, at 21:02, Roman Schmitz <ro...@gmail.com> wrote:
> >
> > Hi all,
> >
> > While working with different customers I came across the case several
> times
> > that we'd like to not only explicitly remove headers by name but by
> pattern
> > / regexp. Here is a KIP for this feature!
> >
> > Please let me know if you have any comments, questions, or suggestions!
> >
> > https://cwiki.apache.org/confluence/x/oYtEE
> >
> > Thanks,
> > Roman
>
>
Re: KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp
Posted by Andrew Schofield <an...@outlook.com>.
Hi Roman,
Thanks for the KIP. I think it’s an interesting idea, but I think the KIP document needs some
more details added before it’s ready for review. For example, here’s a KIP in the same
area which was delivered in an earlier version of Kafka. I think this is a good KIP to copy
for a suitable level of detail and description (https://cwiki.apache.org/confluence/display/KAFKA/KIP-585%3A+Filter+and+Conditional+SMTs).
Hope this helps.
Thanks,
Andrew
> On 15 Oct 2023, at 21:02, Roman Schmitz <ro...@gmail.com> wrote:
>
> Hi all,
>
> While working with different customers I came across the case several times
> that we'd like to not only explicitly remove headers by name but by pattern
> / regexp. Here is a KIP for this feature!
>
> Please let me know if you have any comments, questions, or suggestions!
>
> https://cwiki.apache.org/confluence/x/oYtEE
>
> Thanks,
> Roman