You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by William A Rowe Jr <wr...@rowe-clan.net> on 2016/12/07 20:03:37 UTC

About Interim Response Headers (was: Content-Length header for 1xx status codes)

On Wed, Dec 7, 2016 at 10:50 AM, Jacob Champion <ch...@gmail.com>
wrote:

> On 12/07/2016 08:31 AM, William A Rowe Jr wrote:
>
>> Does anyone have pointers to legitimizing any 100 response
>> headers?
>>
>
> Date is called out explicitly (RFC 7231, sec. 7.1.1.2):
>
>    [...] An origin server MAY
>    send a Date header field if the response is in the 1xx
>    (Informational) or 5xx (Server Error) class of status codes.
>
> It was even more explicit in 2616 (sec. 14.18):
>
>     1. If the response status code is 100 (Continue) or 101 (Switching
>        Protocols), the response MAY include a Date header field, at
>        the server's option.
>
> What's your end goal? I don't think we can *prohibit* modules from sending
> extra headers in 100 responses, but it does make sense to limit what we
> send by default, especially if we're currently sending Content-Length
> (which IIUC is meaningless in that context).
>

When we send 100 CONTINUE from the http_filters.c core read
filter, we send no headers. Simple.

I'm looking at ap_send_interim_response and am somewhat confused
by core_upgrade handler, which I'll get to in a moment.

h2_h2.c check_push() applies only to HTTP/2, and notably does
not clear headers_out before adding the Link header and pushing
out a 103 response, but I'm not going to dig much further at this
exact moment.

mod_proxy_http shoves all existing Set-Cookie headers into the
interim response (e.g. those added by other modules), and it doesn't
appear that these headers are cleared after a 100 continue response.
The Via response is added, hop by hop headers are cleared, and
a whole bunch of other inappropriate processing is going on there.
That module looks the most problematic, send_interim_response
was not very well thought out and includes much internal httpd
decoration that is only appropriate for the final response.

core_upgrade_handler (in server/core.c) *does* clear headers_out
before adding the two necessary Upgrade and Connection headers,
so in this context we are good. But it handles 101 UPGRADE
interim responses without first testing that expecting-100 was
satisfied, per spec. There appears to be no way to process more
than one upgrade (and we know from spec, that multiple upgrades
may be necessary; crypto vs framing, for example).

Prior to core_upgrade_handler calling ap_send_interim_response
to the upgrade, The 100 continue interim response is required,
and in response to the client's reaction to the 100 continue, the
request body must be read off the wire. Only then may the upgrade
interim response, and ap_switch_protocol handling occur.

We also seem to be failing to send Upgrade and Connection
headers after the first response (c.f. the c->keepalives), in the
core_upgrade_handler. Is this some "stateful" reading of the HTTP
protocol? I had thought we discussed this on list once before.

It appears to run in the map_to_storage hook for OPTIONS *
requests, and is deferred to VERY_FIRST in the handler
phase otherwise. This seems awfully late for the intermediate
phases, e.g. auth requiring a TLS connection, but it doesn't
seem possible to resolve this with respect to 100 continue,
which the spec insists can be avoided with an error response,
and they use an auth failure in that example :)


> *Are* we currently sending Content-Length in 100 responses? Luca's bug
> mentions 204 only.
>

I may have misread the top line complaint here.

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Wed, Dec 7, 2016 at 4:19 PM, William A Rowe Jr <wr...@rowe-clan.net>
wrote:

> On Wed, Dec 7, 2016 at 2:23 PM, Jacob Champion <ch...@gmail.com>
> wrote:
>
>> On 12/07/2016 12:03 PM, William A Rowe Jr wrote:
>>
>>> On Wed, Dec 7, 2016 at 10:50 AM, Jacob Champion <champion.p@gmail.com
>>> <ma...@gmail.com>> wrote:
>>>
>>>     What's your end goal? I don't think we can *prohibit* modules from
>>>     sending extra headers in 100 responses, but it does make sense to
>>>     limit what we send by default, especially if we're currently sending
>>>     Content-Length (which IIUC is meaningless in that context).
>>>
>>> When we send 100 CONTINUE from the http_filters.c core read
>>> filter, we send no headers. Simple.
>>>
>>
>> 1) Are 1xx responses handled by the http_filters code? I was under the
>> impression that they were not.
>>
>
> that code generates 100 CONTINUE responses, yes. Although not
> for mod_proxy_http in RFC mode, that case is a pass through of
> the various 1xx responses, except for 100 if it is not expected still.
>
>
>
>> 2) This is allowed by the spec, I guess, but it seems punitively strict
>> to me. Some deployments might be using something interesting in the
>> headers, perhaps a precursor to the upcoming 103?
>
>
> From a mod_proxy perspective, other 1xx's could be sent with the
> headers that accompanied them, but what headers of value can
> httpd add? None, IMO. mod_proxy_http should read the headers
> from 1XX interims into an empty headers array and send them on
> using ap_send_interim_response without extra processing, until
> we finally arrive at a final status which httpd should decorate using
> whatever happened to the headers_out array during pre-processing,
> including mod_headers activity.
>
>
>> mod_proxy_http shoves all existing Set-Cookie headers into the
>>>
>> interim response (e.g. those added by other modules), and it doesn't
>>> appear that these headers are cleared after a 100 continue response.
>>> The Via response is added, hop by hop headers are cleared, and
>>> a whole bunch of other inappropriate processing is going on there.
>>> That module looks the most problematic, send_interim_response
>>> was not very well thought out and includes much internal httpd
>>> decoration that is only appropriate for the final response.
>>>
>>
I mean the logic between reading interim headers and the call to
send_interim_response was inappropriate... and note that we really
can't relay very many 'unexpected' 1XX codes, in fact all 1xx codes
must be negotiated (even 100 with the 100-continue expectation),
something that we will be ill-posed to do as a proxy intermediary.
For example, the protocol negotiation itself is hop-by-hop and the
mod_proxy_http can only speak protocols it knows. We may want
to use 100-continue expectations to the backend even if our client
is HTTP/1.0 and we cannot forward them, provide them to our
client when asked even if the backend won't support them, etc.

This item in particular can't be resolved for 2.4.24 IMO, it is sufficiently
borked as to merit no attention until those items with concensus are
completed and 2.4.24 is tagged.

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by Stefan Eissing <st...@greenbytes.de>.
> Am 08.12.2016 um 11:25 schrieb Yann Ylavic <yl...@gmail.com>:
> 
> On Thu, Dec 8, 2016 at 3:12 AM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>> On Dec 7, 2016 6:23 PM, "Jacob Champion" <ch...@gmail.com> wrote:
>> 
>> On 12/07/2016 04:00 PM, William A Rowe Jr wrote:
>>> 
>>> Consider for a moment the case of an HTTP/1.1 upgrade request
>>> unrecognized by a proxy agent.
>> 
>> 
>> It was my understanding that this is an impossible situation for a
>> conforming proxy, since Upgrade is hop-by-hop. What am I missing?
>> 
>> 
>> The fact that there is no way for us to predict what new headers we are
>> passed in the future are defined in the future to be hop-by-hop, which
>> result in a 105 response code with a similarly imponderable conundrum. No
>> way for RFC2068 servers to know 101 became a hop by hop unhandleable
>> response.
> 
> "Connection: Upgrade, ..." is a MUST when the Upgrade header is used,
> so servers/proxies do know.

Exactly.

There are three things mixed in this dicussion:
1. what the HTTP/1.1 protocol allows and asks us to do
2. what we need to change for 2.4.24
3. what our implementation should be in the future

As to 1: 
basically https://tools.ietf.org/html/rfc7231#section-6.2 says it all. HTTP/1.1 defines 1xx interim responses and requires clients, on receiving them, to act on the known and ignore the unknown ones and continue processing. 

For HTTP/1.1, as a server, we should not generate 1xx interims on HTTP/1.1 connections without a really good reason. There is high probability on unasked and especially unknown 1xx breakage in HTTP/1.1 clients and proxies. 

In HTTP/2, we are pushing to change that behaviour, due to the opportunities that 1xx responses present and the relatively small set of implementations. mod_http2 does fully support it in the upcoming 2.4.24.

As to 2:
I agree that our interim handling and generation could do with some love. But it is working well enough, from my point of view, that I would say we can release a 2.4.24 on it. Even the one item that William mentioned, a HTTP/1.1 client talking via mod_proxy_http2 to a backend, works correctly. mod_proxy_http2 does not forward 1xx responses to a HTTP/1.1 main connection.

As to 3:
Possibilities are endless. From my PoV it was not simple to track down the working of our implementation when I worked on mod_http2 and mod_proxy_http2 and on the 1xx dependencies between them. I agree with the comments made that it is not clear which header are send out under which processing phase. However I think the possibility of modules/hooks to add headers to 1xx response should be there. The protocol allows it and we cannot peak into the future and see all uses cases. Just look at 103 and the Link header.

What is unsatisfying from HTTP/2 implementation PoV is that interim responses need to be serialized and parsed again. That is a course for breakage and, since mod_http2 had to make its own parser, needs to be fixed. In the beginning, I though I could intercept the 1xx generation and skip the parsing. However some modules like mod_proxy_http sends down a HTTP/1.1 serialized format.

-Stefan





Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by Yann Ylavic <yl...@gmail.com>.
On Thu, Dec 8, 2016 at 3:12 AM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Dec 7, 2016 6:23 PM, "Jacob Champion" <ch...@gmail.com> wrote:
>
> On 12/07/2016 04:00 PM, William A Rowe Jr wrote:
>>
>> Consider for a moment the case of an HTTP/1.1 upgrade request
>> unrecognized by a proxy agent.
>
>
> It was my understanding that this is an impossible situation for a
> conforming proxy, since Upgrade is hop-by-hop. What am I missing?
>
>
> The fact that there is no way for us to predict what new headers we are
> passed in the future are defined in the future to be hop-by-hop, which
> result in a 105 response code with a similarly imponderable conundrum. No
> way for RFC2068 servers to know 101 became a hop by hop unhandleable
> response.

"Connection: Upgrade, ..." is a MUST when the Upgrade header is used,
so servers/proxies do know.

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Dec 7, 2016 6:23 PM, "Jacob Champion" <ch...@gmail.com> wrote:

On 12/07/2016 04:00 PM, William A Rowe Jr wrote:

> Consider for a moment the case of an HTTP/1.1 upgrade request
> unrecognized by a proxy agent.
>

It was my understanding that this is an impossible situation for a
conforming proxy, since Upgrade is hop-by-hop. What am I missing?


The fact that there is no way for us to predict what new headers we are
passed in the future are defined in the future to be hop-by-hop, which
result in a 105 response code with a similarly imponderable conundrum. No
way for RFC2068 servers to know 101 became a hop by hop unhandleable
response.

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by Jacob Champion <ch...@gmail.com>.
On 12/07/2016 04:00 PM, William A Rowe Jr wrote:
> Consider for a moment the case of an HTTP/1.1 upgrade request
> unrecognized by a proxy agent.

It was my understanding that this is an impossible situation for a 
conforming proxy, since Upgrade is hop-by-hop. What am I missing?

--Jacob

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Dec 7, 2016 5:04 PM, "Jacob Champion" <ch...@gmail.com> wrote:

[combining replies to your last two emails]


>
No, not reject -- conforming clients parse (and may discard) 1xx's they
don't understand, treating them like 100 Continue, and proxies must forward
as-is (unless the proxy was the agent requesting the 1xx response). Right?

RFC 7231, sec. 6.2:

   A client MUST be able to parse one or more 1xx responses received
   prior to a final response, even if the client does not expect one.  A
   user agent MAY ignore unexpected 1xx responses.

   A proxy MUST forward 1xx responses unless the proxy itself requested
   the generation of the 1xx response.  For example, if a proxy adds an
   "Expect: 100-continue" field when it forwards a request, then it need
   not forward the corresponding 100 (Continue) response(s).


Consider for a moment the case of an HTTP/1.1 upgrade request unrecognized
by a proxy agent. The 101 switching protocols response is garbage to the
proxy agent, but passed along. The proxy agent attempts to read the
response and gets nothing meaningful, the originating user agent gets the
101 response and is busy in the new protocol and reads the 500 error as
garbage input.

The entire idea of forwarding unrecognized/in handled 1xx responses is
madness.

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by Jacob Champion <ch...@gmail.com>.
[combining replies to your last two emails]

On 12/07/2016 02:19 PM, William A Rowe Jr wrote:
> On Wed, Dec 7, 2016 at 2:23 PM, Jacob Champion <champion.p@gmail.com
> <ma...@gmail.com>> wrote:
>
>     On 12/07/2016 12:03 PM, William A Rowe Jr wrote:
>
>         On Wed, Dec 7, 2016 at 10:50 AM, Jacob Champion
>         <champion.p@gmail.com <ma...@gmail.com>
>         <mailto:champion.p@gmail.com <ma...@gmail.com>>> wrote:
>
>             What's your end goal? I don't think we can *prohibit*
>         modules from
>             sending extra headers in 100 responses, but it does make
>         sense to
>             limit what we send by default, especially if we're currently
>         sending
>             Content-Length (which IIUC is meaningless in that context).
>
>         When we send 100 CONTINUE from the http_filters.c core read
>         filter, we send no headers. Simple.

I misunderstood this sentence to be your answer to my question (i.e. I 
thought you were saying "we should never send headers in 100 Continue"). 
I see now that you were simply describing the current behavior 
("ap_http_filter() currently sends no headers when dealing with 100 
Continue").

In that light, my next two questions didn't make much sense.

>     1) Are 1xx responses handled by the http_filters code? I was under
>     the impression that they were not.
>
> that code generates 100 CONTINUE responses, yes. Although not
> for mod_proxy_http in RFC mode, that case is a pass through of
> the various 1xx responses, except for 100 if it is not expected still.
>
>     2) This is allowed by the spec, I guess, but it seems punitively
>     strict to me. Some deployments might be using something interesting
>     in the headers, perhaps a precursor to the upcoming 103?
>
> From a mod_proxy perspective, other 1xx's could be sent with the
> headers that accompanied them, but what headers of value can
> httpd add? None, IMO. mod_proxy_http should read the headers
> from 1XX interims into an empty headers array and send them on
> using ap_send_interim_response without extra processing, until
> we finally arrive at a final status which httpd should decorate using
> whatever happened to the headers_out array during pre-processing,
> including mod_headers activity.

I can't tell if we're discussing just the proxy behavior, or all 1xx 
processing in general. Again, I thought the original bug report was 
related to 204 processing only, and then Luca asked if his patch should 
also include informational-status checks as well.

>         I'm looking at ap_send_interim_response and am somewhat confused
>         by core_upgrade handler, which I'll get to in a moment.
>
>         h2_h2.c check_push() applies only to HTTP/2, and notably does
>         not clear headers_out before adding the Link header and pushing
>         out a 103 response, but I'm not going to dig much further at this
>         exact moment.
>
>     Well, in that case the entire point of 103 is to send headers. That
>     is, if I'm understanding the spec correctly; Stefan, can you confirm?
>
> 102 PROCESSING is defined here, per the registry,
> https://tools.ietf.org/html/rfc2518#page-59
> It doesn't seem to call for additional headers (concieveable one would
> want to add X-Delay: sec or something).
>
> 103 is not defined here;
> http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

103 Early Hints is in discussion on the IETF lists. The client 
interoperability angle -- which I assume is what you're referring to 
below -- is part of that conversation, and so far there seems to be 
significant interest in fixing up clients so that they properly deal 
with unsolicited 1xx codes, as required by the spec.

> In fact there doesn't seem to be any canonical definition. HTTP/1.1
> should surely reject such

No, not reject -- conforming clients parse (and may discard) 1xx's they 
don't understand, treating them like 100 Continue, and proxies must 
forward as-is (unless the proxy was the agent requesting the 1xx 
response). Right?

RFC 7231, sec. 6.2:

    A client MUST be able to parse one or more 1xx responses received
    prior to a final response, even if the client does not expect one.  A
    user agent MAY ignore unexpected 1xx responses.

    A proxy MUST forward 1xx responses unless the proxy itself requested
    the generation of the 1xx response.  For example, if a proxy adds an
    "Expect: 100-continue" field when it forwards a request, then it need
    not forward the corresponding 100 (Continue) response(s).

>, HTTP/2, ENOCLUE (but mod_proxy_http2
> should not let it dribble back into HTTP/1.1 requests).
>
>         mod_proxy_http shoves all existing Set-Cookie headers into the
>         interim response (e.g. those added by other modules), and it doesn't
>         appear that these headers are cleared after a 100 continue response.
>         The Via response is added, hop by hop headers are cleared, and
>         a whole bunch of other inappropriate processing is going on there.
>         That module looks the most problematic, send_interim_response
>         was not very well thought out and includes much internal httpd
>         decoration that is only appropriate for the final response.
>
>         core_upgrade_handler (in server/core.c) *does* clear headers_out
>         before adding the two necessary Upgrade and Connection headers,
>         so in this context we are good. But it handles 101 UPGRADE
>         interim responses without first testing that expecting-100 was
>         satisfied, per spec. There appears to be no way to process more
>         than one upgrade (and we know from spec, that multiple upgrades
>         may be necessary; crypto vs framing, for example).
>
>     Expect: 100-continue doesn't have anything to do with 101 Upgrade...
>     and more than one upgrade doesn't make sense IMO, as the HTTP
>     protocol is replaced wholesale. I think a couple of protocol pieces
>     are being confused here.
>
> Uhm, they can be combined and the spec is clear on how must happen.

They can be combined, yes, but you said "it handles 101 UPGRADE interim 
responses without first testing that expecting-100 was satisfied, per 
spec". That test is *not* required; the two features are orthogonal to 
each other.

On 12/07/2016 02:26 PM, William A Rowe Jr wrote:
> I mean the logic between reading interim headers and the call to
> send_interim_response was inappropriate... and note that we really
> can't relay very many 'unexpected' 1XX codes, in fact all 1xx codes
> must be negotiated (even 100 with the 100-continue expectation),

Not per spec. Practically speaking, it is probably *wise* not to 
generate unsolicited 1xx's at this time, considering how many clients 
have historically seen fit to ignore that MUST I referenced above. But 
like I said, there's currently some interest in fixing that on the 
modern web, and we should make sure we evolve alongside everyone else.

> something that we will be ill-posed to do as a proxy intermediary.
> For example, the protocol negotiation itself is hop-by-hop and the
> mod_proxy_http can only speak protocols it knows. We may want
> to use 100-continue expectations to the backend even if our client
> is HTTP/1.0 and we cannot forward them, provide them to our
> client when asked even if the backend won't support them, etc.

--Jacob

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Wed, Dec 7, 2016 at 2:23 PM, Jacob Champion <ch...@gmail.com> wrote:

> On 12/07/2016 12:03 PM, William A Rowe Jr wrote:
>
>> On Wed, Dec 7, 2016 at 10:50 AM, Jacob Champion <champion.p@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>>     What's your end goal? I don't think we can *prohibit* modules from
>>     sending extra headers in 100 responses, but it does make sense to
>>     limit what we send by default, especially if we're currently sending
>>     Content-Length (which IIUC is meaningless in that context).
>>
>> When we send 100 CONTINUE from the http_filters.c core read
>> filter, we send no headers. Simple.
>>
>
> 1) Are 1xx responses handled by the http_filters code? I was under the
> impression that they were not.
>

that code generates 100 CONTINUE responses, yes. Although not
for mod_proxy_http in RFC mode, that case is a pass through of
the various 1xx responses, except for 100 if it is not expected still.



> 2) This is allowed by the spec, I guess, but it seems punitively strict to
> me. Some deployments might be using something interesting in the headers,
> perhaps a precursor to the upcoming 103?


From a mod_proxy perspective, other 1xx's could be sent with the
headers that accompanied them, but what headers of value can
httpd add? None, IMO. mod_proxy_http should read the headers
from 1XX interims into an empty headers array and send them on
using ap_send_interim_response without extra processing, until
we finally arrive at a final status which httpd should decorate using
whatever happened to the headers_out array during pre-processing,
including mod_headers activity.


> I'm looking at ap_send_interim_response and am somewhat confused
>> by core_upgrade handler, which I'll get to in a moment.
>>
>> h2_h2.c check_push() applies only to HTTP/2, and notably does
>> not clear headers_out before adding the Link header and pushing
>> out a 103 response, but I'm not going to dig much further at this
>> exact moment.
>>
>
> Well, in that case the entire point of 103 is to send headers. That is, if
> I'm understanding the spec correctly; Stefan, can you confirm?


102 PROCESSING is defined here, per the registry,
https://tools.ietf.org/html/rfc2518#page-59
It doesn't seem to call for additional headers (concieveable one would
want to add X-Delay: sec or something).

103 is not defined here;
http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

In fact there doesn't seem to be any canonical definition. HTTP/1.1
should surely reject such, HTTP/2, ENOCLUE (but mod_proxy_http2
should not let it dribble back into HTTP/1.1 requests).


> mod_proxy_http shoves all existing Set-Cookie headers into the
>> interim response (e.g. those added by other modules), and it doesn't
>> appear that these headers are cleared after a 100 continue response.
>> The Via response is added, hop by hop headers are cleared, and
>> a whole bunch of other inappropriate processing is going on there.
>> That module looks the most problematic, send_interim_response
>> was not very well thought out and includes much internal httpd
>> decoration that is only appropriate for the final response.
>>
>> core_upgrade_handler (in server/core.c) *does* clear headers_out
>> before adding the two necessary Upgrade and Connection headers,
>> so in this context we are good. But it handles 101 UPGRADE
>> interim responses without first testing that expecting-100 was
>> satisfied, per spec. There appears to be no way to process more
>> than one upgrade (and we know from spec, that multiple upgrades
>> may be necessary; crypto vs framing, for example).
>>
>
> Expect: 100-continue doesn't have anything to do with 101 Upgrade... and
> more than one upgrade doesn't make sense IMO, as the HTTP protocol is
> replaced wholesale. I think a couple of protocol pieces are being confused
> here.


Uhm, they can be combined and the spec is clear on how must happen.

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

Posted by Jacob Champion <ch...@gmail.com>.
On 12/07/2016 12:03 PM, William A Rowe Jr wrote:
> On Wed, Dec 7, 2016 at 10:50 AM, Jacob Champion <champion.p@gmail.com
> <ma...@gmail.com>> wrote:
>
>     What's your end goal? I don't think we can *prohibit* modules from
>     sending extra headers in 100 responses, but it does make sense to
>     limit what we send by default, especially if we're currently sending
>     Content-Length (which IIUC is meaningless in that context).
>
>
> When we send 100 CONTINUE from the http_filters.c core read
> filter, we send no headers. Simple.

1) Are 1xx responses handled by the http_filters code? I was under the 
impression that they were not.
2) This is allowed by the spec, I guess, but it seems punitively strict 
to me. Some deployments might be using something interesting in the 
headers, perhaps a precursor to the upcoming 103?

> I'm looking at ap_send_interim_response and am somewhat confused
> by core_upgrade handler, which I'll get to in a moment.
>
> h2_h2.c check_push() applies only to HTTP/2, and notably does
> not clear headers_out before adding the Link header and pushing
> out a 103 response, but I'm not going to dig much further at this
> exact moment.

Well, in that case the entire point of 103 is to send headers. That is, 
if I'm understanding the spec correctly; Stefan, can you confirm?

> mod_proxy_http shoves all existing Set-Cookie headers into the
> interim response (e.g. those added by other modules), and it doesn't
> appear that these headers are cleared after a 100 continue response.
> The Via response is added, hop by hop headers are cleared, and
> a whole bunch of other inappropriate processing is going on there.
> That module looks the most problematic, send_interim_response
> was not very well thought out and includes much internal httpd
> decoration that is only appropriate for the final response.
>
> core_upgrade_handler (in server/core.c) *does* clear headers_out
> before adding the two necessary Upgrade and Connection headers,
> so in this context we are good. But it handles 101 UPGRADE
> interim responses without first testing that expecting-100 was
> satisfied, per spec. There appears to be no way to process more
> than one upgrade (and we know from spec, that multiple upgrades
> may be necessary; crypto vs framing, for example).

Expect: 100-continue doesn't have anything to do with 101 Upgrade... and 
more than one upgrade doesn't make sense IMO, as the HTTP protocol is 
replaced wholesale. I think a couple of protocol pieces are being 
confused here.

--Jacob