You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2023/06/02 14:35:43 UTC
Potentially useful filter for debugging, etc.
All,
I've built a Filter for use with a client who has many environments,
many reverse proxies, and many application servers and were getting
confused about what was what.
Basically, it just adds a "Via" header as appropriate and has some
configurability to choose the header name, override the protocol and
server-name if requested, and to limit the number of IP-segments
reported in the value.
Would this be something useful to add to Tomcat?
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Potentially useful filter for debugging, etc.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
All,
On 6/6/23 10:07, Christopher Schultz wrote:
> Mark,
>
> On 6/6/23 06:42, Mark Thomas wrote:
>> devOn 02/06/2023 18:55, Christopher Schultz wrote:
>>> Mark,
>>>
>>> On 6/2/23 11:00, Mark Thomas wrote:
>>>> On 02/06/2023 15:35, Christopher Schultz wrote:
>>>>> All,
>>>>>
>>>>> I've built a Filter for use with a client who has many
>>>>> environments, many reverse proxies, and many application servers
>>>>> and were getting confused about what was what.
>>>>>
>>>>> Basically, it just adds a "Via" header as appropriate and has some
>>>>> configurability to choose the header name, override the protocol
>>>>> and server-name if requested, and to limit the number of
>>>>> IP-segments reported in the value.
>>>>>
>>>>> Would this be something useful to add to Tomcat?
>>>>
>>>> That implies that Tomcat is acting as a reverse proxy. Given that
>>>> functionality isn't built into Tomcat, I'd expect whatever component
>>>> is providing the reverse proxy functionality to provide the Via
>>>> header and any other appropriate debug tools. It seems a little odd
>>>> to provide a debug tool with Tomcat for a feature that Tomcat
>>>> doesn't implement.
>>>
>>> I was thinking that Tomcat here was an origin server and reporting
>>> its own identity, etc. to the reverse-proxy, which would then add its
>>> own "Via" header.
>>
>> I'm a little confused about how this all works.
>
> Sorry, I think I didn't give enough explanation.
>
>> Shouldn't the reverse proxy add the Via header before forwarding the
>> request?
>>
>> Is the Filter adding a Via header? I read the original post as it was.
>> But Via headers are for proxy's not origin servers.
>>
>> Maybe an example would help. Or a link to the Filter's source.
>
> It's possible that I may be abusing the HTTP Via header, but I don't
> think so.
>
> The idea was to provide information to the *client* about which servers
> are handling the request. The HTTP Via header seems to fit the billing,
> as a response-header and not a request-header. I'm not worried about
> request-headers at all, here.
>
> My Filter basically says 'add "Via: protocol/version myself" header to
> the response' and that's it. The reverse-proxy can augment the Via
> response header with its own. So for a simple setup where we have two
> web servers and two Tomcat servers cross-linked, the client might see a
> response header looking like this:
>
> Via: HTTP/1.1 web1, HTTP/1.1 tomcat2
>
>> I am generally in favour of adding Filters that are generally useful
>> but wary of things that appear to behave in unexpected ways.
>
> As you should be.
>
> I created this Filter to be used in a situation with multiple
> environments and where there are multiple reverse proxies between the
> client and the Tomcat server. We were tracking-down what we believed was
> a misconfiguration of one or more of the reverse-proxies that
> cross-linked environments and we needed to figure out which pair of
> reverse-proxy servers were being used in a simple way e.g. using curl.
> (Automation wasn't going to solve this one, unfortunately, given the way
> things had been deployed).
>
> So we configured the Tomcat servers to announce themselves and the httpd
> servers to do the same, using the Via header in this way.
>
> I suppose the httpd instance could set the header to include BOTH
> servers, since it really ought to know its own identity as well as the
> identity of the origin server it's connecting to, but ... I chose this
> way instead since upstream could be more complicated than httpd knows.
>
> Here is the code of the doFilter function without the source of the
> supporting functions:
>
> StringBuilder serverIdentifier = new StringBuilder(64); // 64
> characters ought to be enough for anybody
>
> String protocol = getProtocol();
> if(null == protocol) {
> protocol = httpRequest.getProtocol();
>
> if(null != protocol) {
> if(!getIncludeProtocolName()) {
> int pos = protocol.indexOf('/');
> if(0 <= pos) {
> protocol = protocol.substring(pos + 1);
> }
> }
> serverIdentifier.append(protocol);
> }
> } else {
> serverIdentifier.append(protocol);
> }
>
> String serverName = getServerName();
> if(null == serverName) {
> serverName = getTrimmedIPAddress(httpRequest.getLocalAddr());
> }
>
> serverIdentifier.append(' ').append(serverName);
>
> if(getIncludePort()) {
>
> serverIdentifier.append(':').append(httpRequest.getLocalPort());
> }
>
> httpResponse.addHeader(getHeaderName(),
> serverIdentifier.toString());
>
> Most of the stuff has reasonable defaults. If you installed this on
> localhost IPv4 and didn't override anything, you'd get "Via: HTTP/1.1 1"
> i the response header because the IP address 127.0.0.1 would have all
> but the last octet of the address removed, leaving just the "1". This is
> pretty useless for localhost, but in a subnet with multiple backends,
> you'd get e.g. "Via: HTTP/1.1 42" and "Via HTTP/1.1 43" for servers on
> x.x.x.42 and x.x.x.43 respectively.
>
> You can override everything, including which header to use, the "name"
> of the server (to get "Via: HTTP/1.1 gandalf" if that's what you want it
> to say) and the protocol (including the version number).
>
> Let me know if you have any more questions. I'm happy to share full
> source if it would be helpful.
>
> Re-reading the code (after already providing for some caching of certain
> things), I can see that it could benefit from even more caching. The
> common case will be that the Via header will always be set to the same
> thing for every single request. Sure, it's possible that some requests
> are served via HTTP and some via AJP, but I think usually the response
> header will always be the same, so it's probably worth adding a little
> bit of "quick checking" to avoid StringBuilder churn. Using shared
> structures e.g. ConcurrentHashMap may be a wash, though, so I'd be happy
> to get some feedback on whether this optimization won't really be
> effective.
After using this a little, I've discovered (obviously, in retrospect)
that the Filter can be skipped under certain circumstances -- such as
when an authentication Valve takes control.
I think maybe if we ship something like this with Tomcat, it should be a
Valve installed early in the chain instead of a Filter.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Potentially useful filter for debugging, etc.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Mark,
On 6/6/23 06:42, Mark Thomas wrote:
> devOn 02/06/2023 18:55, Christopher Schultz wrote:
>> Mark,
>>
>> On 6/2/23 11:00, Mark Thomas wrote:
>>> On 02/06/2023 15:35, Christopher Schultz wrote:
>>>> All,
>>>>
>>>> I've built a Filter for use with a client who has many environments,
>>>> many reverse proxies, and many application servers and were getting
>>>> confused about what was what.
>>>>
>>>> Basically, it just adds a "Via" header as appropriate and has some
>>>> configurability to choose the header name, override the protocol and
>>>> server-name if requested, and to limit the number of IP-segments
>>>> reported in the value.
>>>>
>>>> Would this be something useful to add to Tomcat?
>>>
>>> That implies that Tomcat is acting as a reverse proxy. Given that
>>> functionality isn't built into Tomcat, I'd expect whatever component
>>> is providing the reverse proxy functionality to provide the Via
>>> header and any other appropriate debug tools. It seems a little odd
>>> to provide a debug tool with Tomcat for a feature that Tomcat doesn't
>>> implement.
>>
>> I was thinking that Tomcat here was an origin server and reporting its
>> own identity, etc. to the reverse-proxy, which would then add its own
>> "Via" header.
>
> I'm a little confused about how this all works.
Sorry, I think I didn't give enough explanation.
> Shouldn't the reverse proxy add the Via header before forwarding the
> request?
>
> Is the Filter adding a Via header? I read the original post as it was.
> But Via headers are for proxy's not origin servers.
>
> Maybe an example would help. Or a link to the Filter's source.
It's possible that I may be abusing the HTTP Via header, but I don't
think so.
The idea was to provide information to the *client* about which servers
are handling the request. The HTTP Via header seems to fit the billing,
as a response-header and not a request-header. I'm not worried about
request-headers at all, here.
My Filter basically says 'add "Via: protocol/version myself" header to
the response' and that's it. The reverse-proxy can augment the Via
response header with its own. So for a simple setup where we have two
web servers and two Tomcat servers cross-linked, the client might see a
response header looking like this:
Via: HTTP/1.1 web1, HTTP/1.1 tomcat2
> I am generally in favour of adding Filters that are generally useful but
> wary of things that appear to behave in unexpected ways.
As you should be.
I created this Filter to be used in a situation with multiple
environments and where there are multiple reverse proxies between the
client and the Tomcat server. We were tracking-down what we believed was
a misconfiguration of one or more of the reverse-proxies that
cross-linked environments and we needed to figure out which pair of
reverse-proxy servers were being used in a simple way e.g. using curl.
(Automation wasn't going to solve this one, unfortunately, given the way
things had been deployed).
So we configured the Tomcat servers to announce themselves and the httpd
servers to do the same, using the Via header in this way.
I suppose the httpd instance could set the header to include BOTH
servers, since it really ought to know its own identity as well as the
identity of the origin server it's connecting to, but ... I chose this
way instead since upstream could be more complicated than httpd knows.
Here is the code of the doFilter function without the source of the
supporting functions:
StringBuilder serverIdentifier = new StringBuilder(64); // 64
characters ought to be enough for anybody
String protocol = getProtocol();
if(null == protocol) {
protocol = httpRequest.getProtocol();
if(null != protocol) {
if(!getIncludeProtocolName()) {
int pos = protocol.indexOf('/');
if(0 <= pos) {
protocol = protocol.substring(pos + 1);
}
}
serverIdentifier.append(protocol);
}
} else {
serverIdentifier.append(protocol);
}
String serverName = getServerName();
if(null == serverName) {
serverName = getTrimmedIPAddress(httpRequest.getLocalAddr());
}
serverIdentifier.append(' ').append(serverName);
if(getIncludePort()) {
serverIdentifier.append(':').append(httpRequest.getLocalPort());
}
httpResponse.addHeader(getHeaderName(),
serverIdentifier.toString());
Most of the stuff has reasonable defaults. If you installed this on
localhost IPv4 and didn't override anything, you'd get "Via: HTTP/1.1 1"
i the response header because the IP address 127.0.0.1 would have all
but the last octet of the address removed, leaving just the "1". This is
pretty useless for localhost, but in a subnet with multiple backends,
you'd get e.g. "Via: HTTP/1.1 42" and "Via HTTP/1.1 43" for servers on
x.x.x.42 and x.x.x.43 respectively.
You can override everything, including which header to use, the "name"
of the server (to get "Via: HTTP/1.1 gandalf" if that's what you want it
to say) and the protocol (including the version number).
Let me know if you have any more questions. I'm happy to share full
source if it would be helpful.
Re-reading the code (after already providing for some caching of certain
things), I can see that it could benefit from even more caching. The
common case will be that the Via header will always be set to the same
thing for every single request. Sure, it's possible that some requests
are served via HTTP and some via AJP, but I think usually the response
header will always be the same, so it's probably worth adding a little
bit of "quick checking" to avoid StringBuilder churn. Using shared
structures e.g. ConcurrentHashMap may be a wash, though, so I'd be happy
to get some feedback on whether this optimization won't really be effective.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Potentially useful filter for debugging, etc.
Posted by Konstantin Kolinko <kn...@gmail.com>.
вт, 6 июн. 2023 г. в 13:43, Mark Thomas <ma...@apache.org>:
>
> devOn 02/06/2023 18:55, Christopher Schultz wrote:
> > Mark,
> >
> > On 6/2/23 11:00, Mark Thomas wrote:
> >> On 02/06/2023 15:35, Christopher Schultz wrote:
> >>> All,
> >>>
> >>> I've built a Filter for use with a client who has many environments,
> >>> many reverse proxies, and many application servers and were getting
> >>> confused about what was what.
> >>>
I think that Chris wants to be able to trace origins of responses.
From what servers they originated (and maybe what proxies it came
through).
"Adding a header" generally can be done by the well-known "urlrewrite"
filter. Or by mod_header at Apache HTTPD.
Is there any interest in implementing just "adding a header"?
Is it the logic of managing the "Via" header that makes it interesting?
BTW, specification of the Via header:
https://www.rfc-editor.org/rfc/rfc9110.html#name-via
Generally, there will be other proxies in front of Tomcat. Woldn't it
be better to have the logic on those servers? It is rare to use Tomcat
itself as a proxy.
As a preceding example I am thinking about "Ray IDs" as used by Cloudflare:
https://developers.cloudflare.com/fundamentals/get-started/reference/cloudflare-ray-id/
Those are hashed.
It also looks like those IDs are assigned per-request. Essentially,
unique IDs for requests.
The general usage seems to be that the IDs are logged, and are
analyzed afterwards.
BTW, as an alternative, there exists the "Server" header, For an HTTP
connector it can be configured with "server" and
"serverRemoveAppProvidedValues" attributes on a Connector. It seems
that there is no configuration for an AJP Connector.
https://tomcat.apache.org/tomcat-10.1-doc/config/http.html#Standard_Implementation
Specification:
https://www.rfc-editor.org/rfc/rfc9110.html#name-server
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Potentially useful filter for debugging, etc.
Posted by Mark Thomas <ma...@apache.org>.
devOn 02/06/2023 18:55, Christopher Schultz wrote:
> Mark,
>
> On 6/2/23 11:00, Mark Thomas wrote:
>> On 02/06/2023 15:35, Christopher Schultz wrote:
>>> All,
>>>
>>> I've built a Filter for use with a client who has many environments,
>>> many reverse proxies, and many application servers and were getting
>>> confused about what was what.
>>>
>>> Basically, it just adds a "Via" header as appropriate and has some
>>> configurability to choose the header name, override the protocol and
>>> server-name if requested, and to limit the number of IP-segments
>>> reported in the value.
>>>
>>> Would this be something useful to add to Tomcat?
>>
>> That implies that Tomcat is acting as a reverse proxy. Given that
>> functionality isn't built into Tomcat, I'd expect whatever component
>> is providing the reverse proxy functionality to provide the Via header
>> and any other appropriate debug tools. It seems a little odd to
>> provide a debug tool with Tomcat for a feature that Tomcat doesn't
>> implement.
>
> I was thinking that Tomcat here was an origin server and reporting its
> own identity, etc. to the reverse-proxy, which would then add its own
> "Via" header.
I'm a little confused about how this all works.
Shouldn't the reverse proxy add the Via header before forwarding the
request?
Is the Filter adding a Via header? I read the original post as it was.
But Via headers are for proxy's not origin servers.
Maybe an example would help. Or a link to the Filter's source.
I am generally in favour of adding Filters that are generally useful but
wary of things that appear to behave in unexpected ways.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Potentially useful filter for debugging, etc.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Mark,
On 6/2/23 11:00, Mark Thomas wrote:
> On 02/06/2023 15:35, Christopher Schultz wrote:
>> All,
>>
>> I've built a Filter for use with a client who has many environments,
>> many reverse proxies, and many application servers and were getting
>> confused about what was what.
>>
>> Basically, it just adds a "Via" header as appropriate and has some
>> configurability to choose the header name, override the protocol and
>> server-name if requested, and to limit the number of IP-segments
>> reported in the value.
>>
>> Would this be something useful to add to Tomcat?
>
> That implies that Tomcat is acting as a reverse proxy. Given that
> functionality isn't built into Tomcat, I'd expect whatever component is
> providing the reverse proxy functionality to provide the Via header and
> any other appropriate debug tools. It seems a little odd to provide a
> debug tool with Tomcat for a feature that Tomcat doesn't implement.
I was thinking that Tomcat here was an origin server and reporting its
own identity, etc. to the reverse-proxy, which would then add its own
"Via" header.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Potentially useful filter for debugging, etc.
Posted by Mark Thomas <ma...@apache.org>.
On 02/06/2023 15:35, Christopher Schultz wrote:
> All,
>
> I've built a Filter for use with a client who has many environments,
> many reverse proxies, and many application servers and were getting
> confused about what was what.
>
> Basically, it just adds a "Via" header as appropriate and has some
> configurability to choose the header name, override the protocol and
> server-name if requested, and to limit the number of IP-segments
> reported in the value.
>
> Would this be something useful to add to Tomcat?
That implies that Tomcat is acting as a reverse proxy. Given that
functionality isn't built into Tomcat, I'd expect whatever component is
providing the reverse proxy functionality to provide the Via header and
any other appropriate debug tools. It seems a little odd to provide a
debug tool with Tomcat for a feature that Tomcat doesn't implement.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org