You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Jan Ehrhardt <ph...@ehrhardt.nl> on 2015/12/06 04:44:23 UTC
mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Some words for sysadmins: this is a rather annoying bug because there is
not any sign in the logs that the client has a content decoding failure.
You can never be sure that the user gets to see what the server sends to
the client.
Jan Ehrhardt in gmane.comp.apache.devel (Sun, 06 Dec 2015 01:34:47 +0100):
>I will try to intercept the compressed content and see if I can make a
>static page out of it, outside of Drupal and without having to log in as
>Drupal admin.
If content decoding is successful, Chrome stores the compressed content in
the cache. I saved such a file as admin_menu_encoded.txt.gz and made it
available as https://tfrtst.sessionportal.net/admin_menu_encoded.php
PHP-script:
<?php
// open the file in a binary mode
$name = './admin_menu_encoded.txt.gz';
$fp = fopen($name, 'rb');
// send the right headers
header("Content-Type: text/html; charset=utf-8");
// admin_menu_encoded.txt.gz is already gzip'ed, so make sure
// zlib.output_compression does not compress it once more.
ini_set('zlib.output_compression', '0');
header('Content-Encoding: gzip');
header("Cache-Control: no-cache, must-revalidate, post-check=0,
pre-check=0");
header("Content-Length: " . filesize($name));
// dump the file and stop the script
fpassthru($fp);
exit;
?>
The bummer: this file does not fail, whichever mod_http2 I load. Firefox
also fails with mod_http2 1.0.7+ and gives some extra information in the
headers:
X-Requested-With: XMLHttpRequest
The explanation: the drupal admin menu is loaded by JQuery, using an
Ajax-call. The return value of that call is (1) decoded (gunzipped) and
(2) loaded in a <div id="admin-menu"> almost at the end of the page.
Yet that Ajax call is not the culprit either. If I enter the URL of the
admin-menu directly Chrome reports a ERR_CONTENT_DECODING_FAILED and
Firefox:
>Content Encoding Error
>
>The page you are trying to view cannot be shown because
>it uses an invalid or unsupported form of compression.
>
>Please contact the website owners to inform them of this problem.
You will not see this message when the admin menu is loaded by JQuery.
So far the analysis at this moment. I do not know what I could investigate
now. Suggestions welcome.
--
Jan
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
Any internal redirect (such as his admin panel configuration). Serving
redirected static example content should be fine.
On Mon, Dec 7, 2015 at 10:19 AM, Stefan Eissing <
stefan.eissing@greenbytes.de> wrote:
> Oh, I see. I ran into a special_special...
>
> How would I easily trigger this in a test config?
>
> > Am 07.12.2015 um 17:13 schrieb William A Rowe Jr <wr...@rowe-clan.net>:
> >
> > Within internal_internal_redirect;
> >
> > 83750 fielding /*
> > 83750 fielding * XXX: hmm. This is because mod_setenvif and
> mod_unique_id really need
> > 83750 fielding * to do their thing on internal redirects as
> well. Perhaps this is a
> > 83750 fielding * misnamed function.
> > 83750 fielding */
> > 83750 fielding if ((access_status =
> ap_run_post_read_request(new))) {
> > 83750 fielding ap_die(access_status, new);
> > 83750 fielding return NULL;
> > 83750 fielding }
> >
> > That's some ancient mojo there :)
> >
> >
> > On Mon, Dec 7, 2015 at 6:46 AM, Stefan Eissing <
> stefan.eissing@greenbytes.de> wrote:
> > I think I just found the problem:
> >
> > I thought that h2_h2.c, line 678 was no longer needed. But under some,
> still unknown, conditions the post_read_req hook is called twice. That
> makes a table overlay twice which results in all headers_out be present
> twice.
> >
> > I will remove the filter again before installing it. Someday, I need to
> understand the double invocation of the hook...
> >
> > //Stefan
> >
> > > Am 07.12.2015 um 13:38 schrieb Jim Jagielski <ji...@jaguNET.com>:
> > >
> > > Can you retry w DEBUG level, to see if h2_response_trailers_filter()
> > > is the issue?
> > >
> > >> On Dec 6, 2015, at 6:33 AM, Jan Ehrhardt <ph...@ehrhardt.nl> wrote:
> > >>
> > >> Stefan Eissing in gmane.comp.apache.devel (Sun, 6 Dec 2015 09:26:56
> +0100):
> > >>> Jan, thanks for tracking this down to the duplicate headers. I will
> look into it.
> > >>>
> > >>> Just to be sure I got everything right: you tested against the
> mod_h2 github versions?
> > >>
> > >> I used the sources from the ASF repos. This one worked OK:
> > >>
> > >> svn co -r 1715371
> http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
> > >>
> > >> But this one had the duplicate tokens:
> > >>
> > >> svn co -r 1716210
> http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
> > >> --
> > >> Jan
> > >>
> > >
> >
> >
>
>
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by Stefan Eissing <st...@greenbytes.de>.
Oh, I see. I ran into a special_special...
How would I easily trigger this in a test config?
> Am 07.12.2015 um 17:13 schrieb William A Rowe Jr <wr...@rowe-clan.net>:
>
> Within internal_internal_redirect;
>
> 83750 fielding /*
> 83750 fielding * XXX: hmm. This is because mod_setenvif and mod_unique_id really need
> 83750 fielding * to do their thing on internal redirects as well. Perhaps this is a
> 83750 fielding * misnamed function.
> 83750 fielding */
> 83750 fielding if ((access_status = ap_run_post_read_request(new))) {
> 83750 fielding ap_die(access_status, new);
> 83750 fielding return NULL;
> 83750 fielding }
>
> That's some ancient mojo there :)
>
>
> On Mon, Dec 7, 2015 at 6:46 AM, Stefan Eissing <st...@greenbytes.de> wrote:
> I think I just found the problem:
>
> I thought that h2_h2.c, line 678 was no longer needed. But under some, still unknown, conditions the post_read_req hook is called twice. That makes a table overlay twice which results in all headers_out be present twice.
>
> I will remove the filter again before installing it. Someday, I need to understand the double invocation of the hook...
>
> //Stefan
>
> > Am 07.12.2015 um 13:38 schrieb Jim Jagielski <ji...@jaguNET.com>:
> >
> > Can you retry w DEBUG level, to see if h2_response_trailers_filter()
> > is the issue?
> >
> >> On Dec 6, 2015, at 6:33 AM, Jan Ehrhardt <ph...@ehrhardt.nl> wrote:
> >>
> >> Stefan Eissing in gmane.comp.apache.devel (Sun, 6 Dec 2015 09:26:56 +0100):
> >>> Jan, thanks for tracking this down to the duplicate headers. I will look into it.
> >>>
> >>> Just to be sure I got everything right: you tested against the mod_h2 github versions?
> >>
> >> I used the sources from the ASF repos. This one worked OK:
> >>
> >> svn co -r 1715371 http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
> >>
> >> But this one had the duplicate tokens:
> >>
> >> svn co -r 1716210 http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
> >> --
> >> Jan
> >>
> >
>
>
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
Within internal_internal_redirect;
83750 fielding /*
83750 fielding * XXX: hmm. This is because mod_setenvif and
mod_unique_id really need
83750 fielding * to do their thing on internal redirects as well.
Perhaps this is a
83750 fielding * misnamed function.
83750 fielding */
83750 fielding if ((access_status = ap_run_post_read_request(new)))
{
83750 fielding ap_die(access_status, new);
83750 fielding return NULL;
83750 fielding }
That's some ancient mojo there :)
On Mon, Dec 7, 2015 at 6:46 AM, Stefan Eissing <stefan.eissing@greenbytes.de
> wrote:
> I think I just found the problem:
>
> I thought that h2_h2.c, line 678 was no longer needed. But under some,
> still unknown, conditions the post_read_req hook is called twice. That
> makes a table overlay twice which results in all headers_out be present
> twice.
>
> I will remove the filter again before installing it. Someday, I need to
> understand the double invocation of the hook...
>
> //Stefan
>
> > Am 07.12.2015 um 13:38 schrieb Jim Jagielski <ji...@jaguNET.com>:
> >
> > Can you retry w DEBUG level, to see if h2_response_trailers_filter()
> > is the issue?
> >
> >> On Dec 6, 2015, at 6:33 AM, Jan Ehrhardt <ph...@ehrhardt.nl> wrote:
> >>
> >> Stefan Eissing in gmane.comp.apache.devel (Sun, 6 Dec 2015 09:26:56
> +0100):
> >>> Jan, thanks for tracking this down to the duplicate headers. I will
> look into it.
> >>>
> >>> Just to be sure I got everything right: you tested against the mod_h2
> github versions?
> >>
> >> I used the sources from the ASF repos. This one worked OK:
> >>
> >> svn co -r 1715371
> http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
> >>
> >> But this one had the duplicate tokens:
> >>
> >> svn co -r 1716210
> http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
> >> --
> >> Jan
> >>
> >
>
>
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by Stefan Eissing <st...@greenbytes.de>.
I think I just found the problem:
I thought that h2_h2.c, line 678 was no longer needed. But under some, still unknown, conditions the post_read_req hook is called twice. That makes a table overlay twice which results in all headers_out be present twice.
I will remove the filter again before installing it. Someday, I need to understand the double invocation of the hook...
//Stefan
> Am 07.12.2015 um 13:38 schrieb Jim Jagielski <ji...@jaguNET.com>:
>
> Can you retry w DEBUG level, to see if h2_response_trailers_filter()
> is the issue?
>
>> On Dec 6, 2015, at 6:33 AM, Jan Ehrhardt <ph...@ehrhardt.nl> wrote:
>>
>> Stefan Eissing in gmane.comp.apache.devel (Sun, 6 Dec 2015 09:26:56 +0100):
>>> Jan, thanks for tracking this down to the duplicate headers. I will look into it.
>>>
>>> Just to be sure I got everything right: you tested against the mod_h2 github versions?
>>
>> I used the sources from the ASF repos. This one worked OK:
>>
>> svn co -r 1715371 http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
>>
>> But this one had the duplicate tokens:
>>
>> svn co -r 1716210 http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
>> --
>> Jan
>>
>
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by Jim Jagielski <ji...@jaguNET.com>.
Can you retry w DEBUG level, to see if h2_response_trailers_filter()
is the issue?
> On Dec 6, 2015, at 6:33 AM, Jan Ehrhardt <ph...@ehrhardt.nl> wrote:
>
> Stefan Eissing in gmane.comp.apache.devel (Sun, 6 Dec 2015 09:26:56 +0100):
>> Jan, thanks for tracking this down to the duplicate headers. I will look into it.
>>
>> Just to be sure I got everything right: you tested against the mod_h2 github versions?
>
> I used the sources from the ASF repos. This one worked OK:
>
> svn co -r 1715371 http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
>
> But this one had the duplicate tokens:
>
> svn co -r 1716210 http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
> --
> Jan
>
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by Jan Ehrhardt <ph...@ehrhardt.nl>.
Stefan Eissing in gmane.comp.apache.devel (Sun, 6 Dec 2015 09:26:56 +0100):
>Jan, thanks for tracking this down to the duplicate headers. I will look into it.
>
>Just to be sure I got everything right: you tested against the mod_h2 github versions?
I used the sources from the ASF repos. This one worked OK:
svn co -r 1715371 http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
But this one had the duplicate tokens:
svn co -r 1716210 http://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x/modules/http2
--
Jan
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by Stefan Eissing <st...@greenbytes.de>.
Jan, thanks for tracking this down to the duplicate headers. I will look into it.
Just to be sure I got everything right: you tested against the mod_h2 github versions?
> Am 06.12.2015 um 07:53 schrieb Jan Ehrhardt <ph...@ehrhardt.nl>:
>
> Jan Ehrhardt in gmane.comp.apache.devel (Sun, 06 Dec 2015 05:47:43 +0100):
>> Content-Encoding: gzip, gzip
>
> Comment in h2_from_h1.c:
> /*
> * Since some clients choke violently on multiple Vary fields, or
> * Vary fields with duplicate tokens, combine any multiples and remove
> * any duplicates.
> */
>
> Well, some clients choke violently on duplicate tokens in the
> Content-Encoding field. And on duplicate tokens in the
> Strict-Transport-Security field. You should remove duplicates in all
> fields.
> --
> Jan
>
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by Jan Ehrhardt <ph...@ehrhardt.nl>.
Jan Ehrhardt in gmane.comp.apache.devel (Sun, 06 Dec 2015 05:47:43 +0100):
>Content-Encoding: gzip, gzip
Comment in h2_from_h1.c:
/*
* Since some clients choke violently on multiple Vary fields, or
* Vary fields with duplicate tokens, combine any multiples and remove
* any duplicates.
*/
Well, some clients choke violently on duplicate tokens in the
Content-Encoding field. And on duplicate tokens in the
Strict-Transport-Security field. You should remove duplicates in all
fields.
--
Jan
Re: mod_http2 1.0.7+ and Drupal (Was: No H2 Window updates!)
Posted by Jan Ehrhardt <ph...@ehrhardt.nl>.
Jan Ehrhardt in gmane.comp.apache.devel (Sun, 06 Dec 2015 04:44:23 +0100):
>So far the analysis at this moment. I do not know what I could investigate
>now. Suggestions welcome.
Found the difference!
mod_http2 1.0.5 response headers
Cache-Control: private, max-age=31536000
Content-Encoding: gzip
Content-Language: en
Content-Type: text/html; charset=utf-8
Date: Sun, 06 Dec 2015 04:25:44 GMT
Etag: "1449375944-1"
Expires: Mon, 05 Dec 2016 04:25:44 +0000
Last-Modified: Sun, 06 Dec 2015 04:25:44 GMT
Server: Apache/2.4.18-dev (Win32) OpenSSL/1.0.2e-fips mod_fcgid/2.3.9
Strict-Transport-Security: max-age=31536000
Vary: Cookie,Accept-Encoding
X-Content-Type-Options: nosniff
X-Drupal-Cache: MISS
X-Firefox-Spdy: h2
x-powered-by: PHP/5.6.16
mod_http2 1.0.7 response headers
Cache-Control: private, max-age=31536000, private, max-age=31536000
Content-Encoding: gzip, gzip
Content-Language: en, en
Content-Type: text/html; charset=utf-8
Date: Sun, 06 Dec 2015 04:29:09 GMT
Etag: "1449376149-1"
Expires: Mon, 05 Dec 2016 04:29:09 +0000, Mon, 05 Dec 2016 04:29:09 +0000
Last-Modified: Sun, 06 Dec 2015 04:29:09 GMT
Server: Apache/2.4.18-dev (Win32) OpenSSL/1.0.2e-fips mod_fcgid/2.3.9
Strict-Transport-Security: max-age=31536000, max-age=31536000
Vary: Cookie,Accept-Encoding
X-Content-Type-Options: nosniff, nosniff
X-Drupal-Cache: MISS, MISS
X-Firefox-Spdy: h2
x-powered-by: PHP/5.6.16, PHP/5.6.16
Some arguments are duplicated. And 'Content-Encoding: gzip, gzip' is
causing the content decoding error.
Something in mod_http2 1.0.7+ is duplicating the arguments in the headers.
But not always, because it does not duplicate them in
https://tfrtst.sessionportal.net/admin_menu_encoded.php
Jan
BTW. Firefox was also complaining about the double argument in the
Strict-Transport-Security header:
>Strict-Transport-Security: The site specified a header that could
>not be parsed successfully.