You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Yeikel Santana <em...@yeikel.com> on 2023/10/07 02:13:41 UTC
Kafka Connect - Customize REST request headers
Hello everyone,
I'm currently running Kafka Connect behind a firewall that mandates the inclusion of a specific header. This situation becomes particularly challenging when forwarding requests among multiple workers, as it appears that only the Authorization header is included in the request.
I'm wondering if there's a way to customize the headers of Kafka Connect before they are forwarded between workers. From my observations, it seems that this capability may not be available[1], and only the response headers can be customized.
I'd appreciate any realistic alternatives or suggestions you may have in mind.
Thanks!
[1] https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestClient.java#L191-L198
Re: Kafka Connect - Customize REST request headers
Posted by Yeikel Santana <em...@yeikel.com>.
Thank you for the explanation, Chris.
In case it helps, what I'm looking for is similar to KIP 577[1]. My specific example involves a hard-coded key/value pair that needs to be used for pod-to-pod as I can connect to any worker without that specific header, but workers cannot communicate among themselves without it.
To clarify, my environment is behind Istio[2], where Egress Traffic can be created using the following format: `<service name>.<namespace name>.svc.cluster.local`. For example, a request among workers should be:
curl -H "Host: <service name>.<namespace name>.svc.cluster.local" workerIP:PORT
Regarding temporary solutions, I've explored options like utilizing a proxy but I am running within containers that can complicate it further, along with the possibilities of patching, recompiling, or replacing the connect-runtime jar temporarily. I think that something like this might work but I need to test it :
private static void addHeadersToRequest(HttpHeaders headers, Request req) {
req.header("Host","<service name>.<namespace name>.svc.cluster.local");
if (headers != null) {
String credentialAuthorization = headers.getHeaderString(HttpHeaders.AUTHORIZATION);
if (credentialAuthorization != null) {
req.header(HttpHeaders.AUTHORIZATION, credentialAuthorization);
}
}
}
This is of course risky and it would be significantly more convenient if this functionality is integrated into Kafka Connect itself
[1] https://cwiki.apache.org/confluence/display/KAFKA/KIP+577%3A+Allow+HTTP+Response+Headers+to+be+Configured+for+Kafka+Connect
[2] https://istio.io/
---- On Sat, 07 Oct 2023 02:05:14 -0400 Chris Egerton <fe...@gmail.com> wrote ---
Hi Yeikel,
Neat question! And thanks for the link to the RestClient code; very helpful.
I don't believe there's a way to configure Kafka Connect to add these
headers to forwarded requests right now. You may be able to do some kind of
out-of-band proxy magic to intercept forwarded requests and insert the
proper headers there.
I don't see a reason for Kafka Connect to only forward authorization
headers, even after examining the PR [1] and corresponding Jira ticket [2]
that altered the RestClient class to begin including authorization headers
in forwarded REST requests. We may be able to tweak the RestClient to
include all headers instead of just the authorization header. I know that
this doesn't help your immediate situation, but if other committers and
contributors agree that the change would be beneficial, we may be able to
include it in the next release (which may be 3.7.0, or a patch release for
3.4, 3.5, or 3.6). Alternatively, we may have to gate such a change behind
a feature flag (either a coarse-grained boolean that enables/disables
forwarding of all non-authorization headers, or more fine-grained logic
such as include/exclude lists or even regexes), which would require a KIP
and may take longer to release.
I've CC'd the dev list to gather their perspective on this potential
change, and to solicit their input on possible workarounds that may be
useful to you sooner than the next release takes place.
[1] - https://github.com/apache/kafka/pull/6791
[2] - https://issues.apache.org/jira/browse/KAFKA-8404
Cheers,
Chris
On Fri, Oct 6, 2023 at 10:14 PM Yeikel Santana <ma...@yeikel.com> wrote:
> Hello everyone,
>
> I'm currently running Kafka Connect behind a firewall that mandates the
> inclusion of a specific header. This situation becomes particularly
> challenging when forwarding requests among multiple workers, as it appears
> that only the Authorization header is included in the request.
>
> I'm wondering if there's a way to customize the headers of Kafka Connect
> before they are forwarded between workers. From my observations, it seems
> that this capability may not be available[1], and only the response headers
> can be customized.
>
> I'd appreciate any realistic alternatives or suggestions you may have in
> mind.
>
> Thanks!
>
>
>
>
>
>
> [1]
> https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestClient.java#L191-L198
Re: Kafka Connect - Customize REST request headers
Posted by Yeikel Santana <em...@yeikel.com>.
Thank you for the explanation, Chris.
In case it helps, what I'm looking for is similar to KIP 577[1]. My specific example involves a hard-coded key/value pair that needs to be used for pod-to-pod as I can connect to any worker without that specific header, but workers cannot communicate among themselves without it.
To clarify, my environment is behind Istio[2], where Egress Traffic can be created using the following format: `<service name>.<namespace name>.svc.cluster.local`. For example, a request among workers should be:
curl -H "Host: <service name>.<namespace name>.svc.cluster.local" workerIP:PORT
Regarding temporary solutions, I've explored options like utilizing a proxy but I am running within containers that can complicate it further, along with the possibilities of patching, recompiling, or replacing the connect-runtime jar temporarily. I think that something like this might work but I need to test it :
private static void addHeadersToRequest(HttpHeaders headers, Request req) {
req.header("Host","<service name>.<namespace name>.svc.cluster.local");
if (headers != null) {
String credentialAuthorization = headers.getHeaderString(HttpHeaders.AUTHORIZATION);
if (credentialAuthorization != null) {
req.header(HttpHeaders.AUTHORIZATION, credentialAuthorization);
}
}
}
This is of course risky and it would be significantly more convenient if this functionality is integrated into Kafka Connect itself
[1] https://cwiki.apache.org/confluence/display/KAFKA/KIP+577%3A+Allow+HTTP+Response+Headers+to+be+Configured+for+Kafka+Connect
[2] https://istio.io/
---- On Sat, 07 Oct 2023 02:05:14 -0400 Chris Egerton <fe...@gmail.com> wrote ---
Hi Yeikel,
Neat question! And thanks for the link to the RestClient code; very helpful.
I don't believe there's a way to configure Kafka Connect to add these
headers to forwarded requests right now. You may be able to do some kind of
out-of-band proxy magic to intercept forwarded requests and insert the
proper headers there.
I don't see a reason for Kafka Connect to only forward authorization
headers, even after examining the PR [1] and corresponding Jira ticket [2]
that altered the RestClient class to begin including authorization headers
in forwarded REST requests. We may be able to tweak the RestClient to
include all headers instead of just the authorization header. I know that
this doesn't help your immediate situation, but if other committers and
contributors agree that the change would be beneficial, we may be able to
include it in the next release (which may be 3.7.0, or a patch release for
3.4, 3.5, or 3.6). Alternatively, we may have to gate such a change behind
a feature flag (either a coarse-grained boolean that enables/disables
forwarding of all non-authorization headers, or more fine-grained logic
such as include/exclude lists or even regexes), which would require a KIP
and may take longer to release.
I've CC'd the dev list to gather their perspective on this potential
change, and to solicit their input on possible workarounds that may be
useful to you sooner than the next release takes place.
[1] - https://github.com/apache/kafka/pull/6791
[2] - https://issues.apache.org/jira/browse/KAFKA-8404
Cheers,
Chris
On Fri, Oct 6, 2023 at 10:14 PM Yeikel Santana <ma...@yeikel.com> wrote:
> Hello everyone,
>
> I'm currently running Kafka Connect behind a firewall that mandates the
> inclusion of a specific header. This situation becomes particularly
> challenging when forwarding requests among multiple workers, as it appears
> that only the Authorization header is included in the request.
>
> I'm wondering if there's a way to customize the headers of Kafka Connect
> before they are forwarded between workers. From my observations, it seems
> that this capability may not be available[1], and only the response headers
> can be customized.
>
> I'd appreciate any realistic alternatives or suggestions you may have in
> mind.
>
> Thanks!
>
>
>
>
>
>
> [1]
> https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestClient.java#L191-L198
Re: Kafka Connect - Customize REST request headers
Posted by Chris Egerton <fe...@gmail.com>.
Hi Yeikel,
Neat question! And thanks for the link to the RestClient code; very helpful.
I don't believe there's a way to configure Kafka Connect to add these
headers to forwarded requests right now. You may be able to do some kind of
out-of-band proxy magic to intercept forwarded requests and insert the
proper headers there?
I don't see a reason for Kafka Connect to only forward authorization
headers, even after examining the PR [1] and corresponding Jira ticket [2]
that altered the RestClient class to begin including authorization headers
in forwarded REST requests. We may be able to tweak the RestClient to
include all headers instead of just the authorization header. I know that
this doesn't help your immediate situation, but if other committers and
contributors agree that the change would be beneficial, we may be able to
include it in the next release (which may be 3.7.0, or a patch release for
3.4, 3.5, or 3.6). Alternatively, we may have to gate such a change behind
a feature flag (either a coarse-grained boolean that enables/disables
forwarding of all non-authorization headers, or more fine-grained logic
such as include/exclude lists or even regexes), which would require a KIP
and may take longer to release.
I've CC'd the dev list to gather their perspective on this potential
change, and to solicit their input on possible workarounds that may be
useful to you sooner than the next release takes place.
[1] - https://github.com/apache/kafka/pull/6791
[2] - https://issues.apache.org/jira/browse/KAFKA-8404
Cheers,
Chris
On Fri, Oct 6, 2023 at 10:14 PM Yeikel Santana <em...@yeikel.com> wrote:
> Hello everyone,
>
> I'm currently running Kafka Connect behind a firewall that mandates the
> inclusion of a specific header. This situation becomes particularly
> challenging when forwarding requests among multiple workers, as it appears
> that only the Authorization header is included in the request.
>
> I'm wondering if there's a way to customize the headers of Kafka Connect
> before they are forwarded between workers. From my observations, it seems
> that this capability may not be available[1], and only the response headers
> can be customized.
>
> I'd appreciate any realistic alternatives or suggestions you may have in
> mind.
>
> Thanks!
>
>
>
>
>
>
> [1]
> https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestClient.java#L191-L198
Re: Kafka Connect - Customize REST request headers
Posted by Chris Egerton <fe...@gmail.com>.
Hi Yeikel,
Neat question! And thanks for the link to the RestClient code; very helpful.
I don't believe there's a way to configure Kafka Connect to add these
headers to forwarded requests right now. You may be able to do some kind of
out-of-band proxy magic to intercept forwarded requests and insert the
proper headers there?
I don't see a reason for Kafka Connect to only forward authorization
headers, even after examining the PR [1] and corresponding Jira ticket [2]
that altered the RestClient class to begin including authorization headers
in forwarded REST requests. We may be able to tweak the RestClient to
include all headers instead of just the authorization header. I know that
this doesn't help your immediate situation, but if other committers and
contributors agree that the change would be beneficial, we may be able to
include it in the next release (which may be 3.7.0, or a patch release for
3.4, 3.5, or 3.6). Alternatively, we may have to gate such a change behind
a feature flag (either a coarse-grained boolean that enables/disables
forwarding of all non-authorization headers, or more fine-grained logic
such as include/exclude lists or even regexes), which would require a KIP
and may take longer to release.
I've CC'd the dev list to gather their perspective on this potential
change, and to solicit their input on possible workarounds that may be
useful to you sooner than the next release takes place.
[1] - https://github.com/apache/kafka/pull/6791
[2] - https://issues.apache.org/jira/browse/KAFKA-8404
Cheers,
Chris
On Fri, Oct 6, 2023 at 10:14 PM Yeikel Santana <em...@yeikel.com> wrote:
> Hello everyone,
>
> I'm currently running Kafka Connect behind a firewall that mandates the
> inclusion of a specific header. This situation becomes particularly
> challenging when forwarding requests among multiple workers, as it appears
> that only the Authorization header is included in the request.
>
> I'm wondering if there's a way to customize the headers of Kafka Connect
> before they are forwarded between workers. From my observations, it seems
> that this capability may not be available[1], and only the response headers
> can be customized.
>
> I'd appreciate any realistic alternatives or suggestions you may have in
> mind.
>
> Thanks!
>
>
>
>
>
>
> [1]
> https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestClient.java#L191-L198