You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Ed Korthof <ed...@organic.com> on 1998/02/05 02:20:59 UTC
case insensitive, multi-line headers
I was looking for some unrelated information (safe characters w/ which to
identify the end of each header for internal communication between
different processes), and noticed a couple of things which seem incorrect
about how Apache handles headers for HTTP/1.1.
It looks as if we should be treating all header names in a case
insensitive manner. It also looks like we're not dealing correctly with
headers which extend past a single line by placing a space or horizontal
tab at the begining of the second (or third, etc.) line. If these aren't
correct in Apache currently, it's clearly not that big a deal (since it
works w/ all major clients) but if not, it'd be good to be HTTP/1.1
compliant. Consider the following:
-----
4.2 Message Headers
HTTP header fields, which include general-header (section 4.5), request-
header (section 5.3), response-header (section 6.2), and entity-header
(section 7.1) fields, follow the same generic format as that given in
Section 3.1 of RFC 822 [9]. Each header field consists of a name
followed by a colon (":") and the field value. Field names are case-
insensitive. The field value may be preceded by any amount of LWS,
though a single SP is preferred. Header fields can be extended over
multiple lines by preceding each extra line with at least one SP or HT.
Applications SHOULD follow "common form" when generating HTTP
constructs, since there might exist some implementations that fail to
accept anything beyond the common forms.
-----
(SP = space, HT = horizontal tab)
Am I missing something? I haven't studied the document all that
carefully... I was looking at
http://www.ics.uci.edu/pub/ietf/http/draft-ietf-http-v11-spec-rev-01.txt
which is described as valid till May 21st of this year.
Btw -- I did do a patch to limit connections by IP; it is, as Dean noted,
rather ugly. It's also an operation with linear running time, for all
that it's relatively fast (there's no way out of the O(n) running time
which I can see which isn't uglier and slower). It does work on 1.2
servers (except Stronghold -- I'm sure I can fix that w/ a little work,
though), and I can probably adapt it for 1.3. I'll see about putting it
in the patches directory of the Apache site... it worked wonderfully for
us for preventing the kind of DoS attack described a bit earlier. We
generally set the number of possible connections fairly high -- around 30
-- but that fixed it for us, since the DoS was coming from a broken piece
of software (shockwave on NT, for whatever reason). Any limit can be
better than none. It returns a 503 error, which makes it easy to track.
-- Ed Korthof | Web Server Engineer --
-- ed@organic.com | Organic Online, Inc --
-- (415) 278-5676 | Fax: (415) 284-6891 --
Re: case insensitive, multi-line headers
Posted by Ed Korthof <ed...@organic.com>.
eh, duh. Sorry -- we do case insensitive matching already. But I'm not
so sure about the multi-line headers...
-- Ed Korthof | Web Server Engineer --
-- ed@organic.com | Organic Online, Inc --
-- (415) 278-5676 | Fax: (415) 284-6891 --
On Wed, 4 Feb 1998, Ed Korthof wrote:
> It looks as if we should be treating all header names in a case
> insensitive manner. It also looks like we're not dealing correctly with
> headers which extend past a single line by placing a space or horizontal
> tab at the begining of the second (or third, etc.) line. If these aren't
> correct in Apache currently, it's clearly not that big a deal (since it
> works w/ all major clients) but if not, it'd be good to be HTTP/1.1
> compliant. Consider the following:
[PATCH] http_protocol (was Re: case insensitive, multi-line headers)
Posted by Ed Korthof <ed...@organic.com>.
Sorry 'bout missing the case-insensitivity & the way multi-line headers
are handled. <rueful>
I'd like to fix the actual bugs Marc and Dean described, as I have some
free time on my hands.
So far as fixing the 408 -> 414 problem, we could set the status to
HTTP_OK if it's currently 408 (we'd just do this in the sections where
we're reading headers and we're about to return a 414 error to the client
anyway, so this wouldn't mess up other parts of the code & vice versa),
and then die() with an appropriate error. So far as I can see, it really
is true that other than timeouts and the potential to return an error
while reading the headers, there's no way we'll have to worry about
r->status being set to HTTP_REQUEST_TIME_OUT.
There are, of course, other ways to do the HTTP_REQUEST_TIME_OUT thing,
and perhaps it'd be worthwhile to change it. I dunno. Each of the other
solutions I heard when we last went over this had it's own set of
problems...
Anyway, I put together a patch to fix some the problems Marc described
(without changing how we handle HTTP_REQUEST_TIME_OUT) and the one Dean
described. So far as the latter problem is concerned, I wrote the patch
to keep pulling data and discard it. It'd certainly also be possible to
have it return a 400 error or pull in all the data and then combine it;
I'd be happy to reimplement this either of those ways.
In this patch, I have set it up to log failures in the access log as well
as in the error log. Also, if run_post_read_request() returned something,
the request would be killed (via die()), but it wouldn't be logged. I've
changed that as well... it seems we should log such stuff.
Clients who trigger 400, 408, and 414 errors generally don't see the error
itself, though it will be logged. That might be worth fixing...
Comments are welcome, as always.
This is only lightly tested, though it does appear to work fine.
-- Ed Korthof | Web Server Engineer --
-- ed@organic.com | Organic Online, Inc --
-- (415) 278-5676 | Fax: (415) 284-6891 --
On Wed, 4 Feb 1998, Marc Slemko wrote:
> Yes. This is mixed in with a bunch of other brokenness in this area (eg.
> logging 408 (timeout) for a 414 (URI too large) that I ranted on a bit
> ago. Unfortunately, I'm swamped right now. Sigh. Perhaps next year.
>
> On Wed, 4 Feb 1998, Dean Gaudet wrote:
> > On Wed, 4 Feb 1998, Dean Gaudet wrote:
> > > Look at getline() we handle multiline headers.
> >
> > Although there's an obvious bug. If there's a line that exceeds the
> > buffer size then we bail without reading the rest of the line and any
> > possible continuation lines. So we'll treat the rest of the line as if
> > it's a new line. Bad.
> >
> > Dean
Re: case insensitive, multi-line headers
Posted by Marc Slemko <ma...@worldgate.com>.
Yes. This is mixed in with a bunch of other brokenness in this area (eg.
logging 408 (timeout) for a 414 (URI too large) that I ranted on a bit
ago. Unfortunately, I'm swamped right now. Sigh. Perhaps next year.
On Wed, 4 Feb 1998, Dean Gaudet wrote:
>
>
> On Wed, 4 Feb 1998, Dean Gaudet wrote:
>
> > Look at getline() we handle multiline headers.
>
> Although there's an obvious bug. If there's a line that exceeds the
> buffer size then we bail without reading the rest of the line and any
> possible continuation lines. So we'll treat the rest of the line as if
> it's a new line. Bad.
>
> Dean
>
Re: case insensitive, multi-line headers
Posted by Dean Gaudet <dg...@arctic.org>.
On Wed, 4 Feb 1998, Dean Gaudet wrote:
> Look at getline() we handle multiline headers.
Although there's an obvious bug. If there's a line that exceeds the
buffer size then we bail without reading the rest of the line and any
possible continuation lines. So we'll treat the rest of the line as if
it's a new line. Bad.
Dean
Re: case insensitive, multi-line headers
Posted by Dean Gaudet <dg...@arctic.org>.
On Wed, 4 Feb 1998, Ed Korthof wrote:
> I was looking for some unrelated information (safe characters w/ which to
> identify the end of each header for internal communication between
> different processes), and noticed a couple of things which seem incorrect
> about how Apache handles headers for HTTP/1.1.
>
> It looks as if we should be treating all header names in a case
> insensitive manner. It also looks like we're not dealing correctly with
> headers which extend past a single line by placing a space or horizontal
> tab at the begining of the second (or third, etc.) line. If these aren't
> correct in Apache currently, it's clearly not that big a deal (since it
> works w/ all major clients) but if not, it'd be good to be HTTP/1.1
> compliant. Consider the following:
Do you have any examples where we aren't treating them case-insensitive?
recall that tables are case-insensitive.
Look at getline() we handle multiline headers.
Dean