mod_headers best practices and headers duplicated in the response

[Luca Toscano <to...@gmail.com>]

Hi everybody,

apologies if this subject has been brought up in the past but I didn't
find much. I have been working on some bugs like
https://bz.apache.org/bugzilla/show_bug.cgi?id=62380 in which users
report responses with the same header duplicated. To keep the story
short, it seems that everything is due to the fact that sometimes
httpd fills both r->headers_out and r->err_headers_out with the same
header, and both get added to the response.

Example from 62380:

- Simple ProxyPass configuration with mod_proxy_http, the (HTTP)
backend returns a response with the X-Foo header set to baz
- Header always set X-Foo "bar" (in the httpd.conf)

This leads to the following in the response:
[..]
X-Foo: baz
[..]
X-Foo: bar
[..]

This happens because, afaict, mod_proxy_http returns the backend
headers into r->headers_out, while Header always set adds it to
r->err_headers_out. With mod_proxy_fcgi the behavior is different
since both headers end up in r->err_headers_out (so only X-Foo: bar is
present in the response).
I tried to come up with some patch, but the more I think about it the
less I am inclined to change the current behavior, but only to improve
the docs to be more explicit about the two header lists:

Header onsuccess verb [..]  -> works on r->headers_out
Header always verb [..] -> works on r->err_headers_out

So in the above baz/bar example, if the users really wants to be sure
that an override happens, it should:

Header onsuccess unset X-Foo  (onsuccess can be omitted in here, but
it seems clearer to be explicit)
Header always set X-Foo bar

Same thing for all the other verbs of course (setifempty, merge,
etc..). It is also kinda explained in
https://httpd.apache.org/docs/2.4/mod/mod_headers.html#header but in
my opinion not super clearly, it is easy to get confused (like I was)
while reading it the first 3/4 times.

Anything that I am missing or this is "only" a documentation bug fix?
(Also something worth to change to be more user friendly in 2.6 tbh).

Luca

Re: mod_headers best practices and headers duplicated in the response

[Stefan Eissing <st...@greenbytes.de>]

> Am 15.10.2018 um 13:45 schrieb Luca Toscano <to...@gmail.com>:
> 
> Hi everybody,
> 
> apologies if this subject has been brought up in the past but I didn't
> find much. I have been working on some bugs like
> https://bz.apache.org/bugzilla/show_bug.cgi?id=62380 in which users
> report responses with the same header duplicated. To keep the story
> short, it seems that everything is due to the fact that sometimes
> httpd fills both r->headers_out and r->err_headers_out with the same
> header, and both get added to the response.
> 
> Example from 62380:
> 
> - Simple ProxyPass configuration with mod_proxy_http, the (HTTP)
> backend returns a response with the X-Foo header set to baz
> - Header always set X-Foo "bar" (in the httpd.conf)
> 
> This leads to the following in the response:
> [..]
> X-Foo: baz
> [..]
> X-Foo: bar
> [..]
> 
> This happens because, afaict, mod_proxy_http returns the backend
> headers into r->headers_out, while Header always set adds it to
> r->err_headers_out. With mod_proxy_fcgi the behavior is different
> since both headers end up in r->err_headers_out (so only X-Foo: bar is
> present in the response).
> I tried to come up with some patch, but the more I think about it the
> less I am inclined to change the current behavior, but only to improve
> the docs to be more explicit about the two header lists:
> 
> Header onsuccess verb [..]  -> works on r->headers_out
> Header always verb [..] -> works on r->err_headers_out
> 
> So in the above baz/bar example, if the users really wants to be sure
> that an override happens, it should:
> 
> Header onsuccess unset X-Foo  (onsuccess can be omitted in here, but
> it seems clearer to be explicit)
> Header always set X-Foo bar
> 
> Same thing for all the other verbs of course (setifempty, merge,
> etc..). It is also kinda explained in
> https://httpd.apache.org/docs/2.4/mod/mod_headers.html#header but in
> my opinion not super clearly, it is easy to get confused (like I was)
> while reading it the first 3/4 times.
> 
> Anything that I am missing or this is "only" a documentation bug fix?
> (Also something worth to change to be more user friendly in 2.6 tbh).

Not an expert on mod_headers, but its documentation mentions "add" and "set"
with different semantics. From a user's point of view, it seems that "set"
should replace any pre-existing header(s) of the same name.

Now, this might be difficult or even impossible to fix in 2.4.x without
breakage somewhere else. Documenting existing behaviour, offering workaround
as you describe, looks good to me.

Ideally, any header manipulation would work on a _normalized_ header
dictionary, so that operations and their outcome are more clear. 

The HTTP header _serialization_ is quite flexible so that values can be added later
in processing. But this then can make things complicated. Especially since
some header field names are unlike others...

-Stefan