You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Stefan Eissing <st...@eissing.org> on 2022/01/24 14:28:35 UTC

http and http/1.x separation

FYI: I am busy hacking away at the separation between our HTTP and HTTP/1.x handling code. I'll make a PR once all tests have passed, so we can talk about the changes. 

The goals are pretty simple:

1. have mod_http focus on HTTP semantics, header checks, ap_die conversions,
   not chunking and CRLFs.
2. have a mod_http1 that installs TRANSCODE filters when the connection is h1.
   those do the chunking and the header serialization.
3. Have interim responses that send META buckets, not DATA to the transcode filters.
   No content filter will be confused by those.
4. Pass FOOTERS in/out as META buckets in a similar way.

The main idea is to introduce a META "HEADERS" bucket that is used for
final/interim responses and footers as well. This will safely pass through
all filters that do not know about it. This is similar to the ERROR bucket 
type we already have.

It will eliminate the remaining code duplications in mod_http2, make
it leaner and use the standard HTTP filters just like for HTTP/1.x requests.
Also, h2 will no longer need to parse interim responses back into header tables
or deal with chunked encodings.

Kind Regards,
Stefan

Re: http and http/1.x separation

Posted by Graham Leggett <mi...@sharp.fm>.
On 24 Jan 2022, at 16:28, Stefan Eissing <st...@eissing.org> wrote:

> The main idea is to introduce a META "HEADERS" bucket that is used for
> final/interim responses and footers as well. This will safely pass through
> all filters that do not know about it. This is similar to the ERROR bucket 
> type we already have.

Big +1.

We don’t use metadata buckets enough, and life is made significantly easier when used.

Regards,
Graham
β€”


Re: http and http/1.x separation

Posted by Stefan Eissing <st...@eissing.org>.

> Am 25.01.2022 um 17:42 schrieb Roy T. Fielding <fi...@gbiv.com>:
> 
>> On Jan 24, 2022, at 6:53 AM, Stefan Eissing <st...@eissing.org> wrote:
>> 
>> 
>> 
>>> Am 24.01.2022 um 15:40 schrieb Yann Ylavic <yl...@gmail.com>:
>>> 
>>> On Mon, Jan 24, 2022 at 3:28 PM Stefan Eissing <st...@eissing.org> wrote:
>>>> 
>>>> FYI: I am busy hacking away at the separation between our HTTP and HTTP/1.x handling code. I'll make a PR once all tests have passed, so we can talk about the changes.
>>> 
>>> Looking forward to seeing it ;)
>> 
>> πŸ™ˆ
>> 
>>> 
>>>> 
>>>> The goals are pretty simple:
>>>> 
>>>> 1. have mod_http focus on HTTP semantics, header checks, ap_die conversions,
>>>>  not chunking and CRLFs.
>>>> 2. have a mod_http1 that installs TRANSCODE filters when the connection is h1.
>>>>  those do the chunking and the header serialization.
>>>> 3. Have interim responses that send META buckets, not DATA to the transcode filters.
>>>>  No content filter will be confused by those.
>>>> 4. Pass FOOTERS in/out as META buckets in a similar way.
>>>> 
>>>> The main idea is to introduce a META "HEADERS" bucket that is used for
>>>> final/interim responses and footers as well. This will safely pass through
>>>> all filters that do not know about it. This is similar to the ERROR bucket
>>>> type we already have.
>>> 
>>> There are some assumptions in httpd (and possibly third-party modules)
>>> that meta buckets have a no length, so if "HEADERS" buckets have one
>>> you might need to embed it in the bucket->data struct..
>> 
>> Yes, it's 0 length like ERROR and has a struct as that holds:
>> - int status, opt 0 
>> - const char *reason, opt NULL
>> - apr_table_t *headers
>> - apr_table_t *notes
>> 
>> on responses, shallow copied from the request_rec, etc.
>> 
>> The nice thing is that the HTTP/1.x transcode out filter can
>> now check in one place that
>> - there is indeed a HEADERS before data buckets
>> - there is only one HEADERS with status >= 200
>> 
>> And "above" the transcode layer, a data bucket is content, period.
> 
> And trailers can be implemented this way too, right?

Yes, exactly. On h2 always, on h1 when the response is chunked. Otherwise the footers HEADERS bucket will we a NOP in h1 transcoding.

> ....Roy


Re: http and http/1.x separation

Posted by "Roy T. Fielding" <fi...@gbiv.com>.
> On Jan 24, 2022, at 6:53 AM, Stefan Eissing <st...@eissing.org> wrote:
> 
> 
> 
>> Am 24.01.2022 um 15:40 schrieb Yann Ylavic <ylavic.dev@gmail.com <ma...@gmail.com>>:
>> 
>> On Mon, Jan 24, 2022 at 3:28 PM Stefan Eissing <stefan@eissing.org <ma...@eissing.org>> wrote:
>>> 
>>> FYI: I am busy hacking away at the separation between our HTTP and HTTP/1.x handling code. I'll make a PR once all tests have passed, so we can talk about the changes.
>> 
>> Looking forward to seeing it ;)
> 
> πŸ™ˆ
> 
>> 
>>> 
>>> The goals are pretty simple:
>>> 
>>> 1. have mod_http focus on HTTP semantics, header checks, ap_die conversions,
>>>  not chunking and CRLFs.
>>> 2. have a mod_http1 that installs TRANSCODE filters when the connection is h1.
>>>  those do the chunking and the header serialization.
>>> 3. Have interim responses that send META buckets, not DATA to the transcode filters.
>>>  No content filter will be confused by those.
>>> 4. Pass FOOTERS in/out as META buckets in a similar way.
>>> 
>>> The main idea is to introduce a META "HEADERS" bucket that is used for
>>> final/interim responses and footers as well. This will safely pass through
>>> all filters that do not know about it. This is similar to the ERROR bucket
>>> type we already have.
>> 
>> There are some assumptions in httpd (and possibly third-party modules)
>> that meta buckets have a no length, so if "HEADERS" buckets have one
>> you might need to embed it in the bucket->data struct..
> 
> Yes, it's 0 length like ERROR and has a struct as that holds:
> - int status, opt 0 
> - const char *reason, opt NULL
> - apr_table_t *headers
> - apr_table_t *notes
> 
> on responses, shallow copied from the request_rec, etc.
> 
> The nice thing is that the HTTP/1.x transcode out filter can
> now check in one place that
> - there is indeed a HEADERS before data buckets
> - there is only one HEADERS with status >= 200
> 
> And "above" the transcode layer, a data bucket is content, period.

And trailers can be implemented this way too, right?

....Roy


Re: http and http/1.x separation

Posted by Graham Leggett <mi...@sharp.fm>.
On 24 Jan 2022, at 16:53, Stefan Eissing <st...@eissing.org> wrote:

> Yes, it's 0 length like ERROR and has a struct as that holds:
> - int status, opt 0 
> - const char *reason, opt NULL
> - apr_table_t *headers
> - apr_table_t *notes
> 
> on responses, shallow copied from the request_rec, etc.
> 
> The nice thing is that the HTTP/1.x transcode out filter can
> now check in one place that
> - there is indeed a HEADERS before data buckets
> - there is only one HEADERS with status >= 200
> 
> And "above" the transcode layer, a data bucket is content, period.

I have a module that parses multipart/mime headers and puts them into a metadata bucket, that has a lot of similarity to what you’re after:

https://github.com/minfrin/mod_multipart/blob/main/mod_multipart.c#L514

Some of the functions above would be useful in httpd.

Regards,
Graham
β€”


Re: http and http/1.x separation

Posted by Stefan Eissing <st...@eissing.org>.

> Am 24.01.2022 um 15:40 schrieb Yann Ylavic <yl...@gmail.com>:
> 
> On Mon, Jan 24, 2022 at 3:28 PM Stefan Eissing <st...@eissing.org> wrote:
>> 
>> FYI: I am busy hacking away at the separation between our HTTP and HTTP/1.x handling code. I'll make a PR once all tests have passed, so we can talk about the changes.
> 
> Looking forward to seeing it ;)

πŸ™ˆ

> 
>> 
>> The goals are pretty simple:
>> 
>> 1. have mod_http focus on HTTP semantics, header checks, ap_die conversions,
>>   not chunking and CRLFs.
>> 2. have a mod_http1 that installs TRANSCODE filters when the connection is h1.
>>   those do the chunking and the header serialization.
>> 3. Have interim responses that send META buckets, not DATA to the transcode filters.
>>   No content filter will be confused by those.
>> 4. Pass FOOTERS in/out as META buckets in a similar way.
>> 
>> The main idea is to introduce a META "HEADERS" bucket that is used for
>> final/interim responses and footers as well. This will safely pass through
>> all filters that do not know about it. This is similar to the ERROR bucket
>> type we already have.
> 
> There are some assumptions in httpd (and possibly third-party modules)
> that meta buckets have a no length, so if "HEADERS" buckets have one
> you might need to embed it in the bucket->data struct..

Yes, it's 0 length like ERROR and has a struct as that holds:
- int status, opt 0 
- const char *reason, opt NULL
- apr_table_t *headers
- apr_table_t *notes

on responses, shallow copied from the request_rec, etc.

The nice thing is that the HTTP/1.x transcode out filter can
now check in one place that
- there is indeed a HEADERS before data buckets
- there is only one HEADERS with status >= 200

And "above" the transcode layer, a data bucket is content, period.

> 
> 
> Cheers;
> Yann.


Re: http and http/1.x separation

Posted by Yann Ylavic <yl...@gmail.com>.
On Mon, Jan 24, 2022 at 3:28 PM Stefan Eissing <st...@eissing.org> wrote:
>
> FYI: I am busy hacking away at the separation between our HTTP and HTTP/1.x handling code. I'll make a PR once all tests have passed, so we can talk about the changes.

Looking forward to seeing it ;)

>
> The goals are pretty simple:
>
> 1. have mod_http focus on HTTP semantics, header checks, ap_die conversions,
>    not chunking and CRLFs.
> 2. have a mod_http1 that installs TRANSCODE filters when the connection is h1.
>    those do the chunking and the header serialization.
> 3. Have interim responses that send META buckets, not DATA to the transcode filters.
>    No content filter will be confused by those.
> 4. Pass FOOTERS in/out as META buckets in a similar way.
>
> The main idea is to introduce a META "HEADERS" bucket that is used for
> final/interim responses and footers as well. This will safely pass through
> all filters that do not know about it. This is similar to the ERROR bucket
> type we already have.

There are some assumptions in httpd (and possibly third-party modules)
that meta buckets have a no length, so if "HEADERS" buckets have one
you might need to embed it in the bucket->data struct..


Cheers;
Yann.