You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Nilesh Bansal <ni...@gmail.com> on 2009/11/28 00:48:52 UTC
AJP with HTTPD - Buffer Size on long URLs
Hi,
I have recently configured the apache httpd to use AJP for connecting to
backend Tomcat servers over AJP (ajp://tomcat.server/8009).
When I try to query the frontend httpd server with a long url (over 8000
characters), the request fails. The following is printed in error_log of
httpd
[Fri Nov 27 18:30:13 2009] [error] ajp_msg_append_cvt_string():
BufferOverflowException 4 6117
[Fri Nov 27 18:30:13 2009] [error] ajp_marshal_into_msgb: Error appending
the query string
[Fri Nov 27 18:30:13 2009] [error] ajp_send_header: ajp_marshal_into_msgb
failed
[Fri Nov 27 18:30:13 2009] [error] (120001)APR does not understand this
error code: proxy: AJP: request failed to (null) (tomcat.server)
Is there a way the buffer size can be increased?
thanks,
Nilesh
Re: AJP with HTTPD - Buffer Size on long URLs
Posted by wygnesh <vi...@realsoftinc.com>.
Nilesh / Others,
I'm facing the same problem too - I was trying to find about where the
"ProxyIOBufferSize" directive has to be added to. Is it the "httpd.h" file
(for the max value) & then subsequently the "httpd.conf" to set any limit
within the max value, if required.
The mod_proxy says "mod_proxy.c" is the module file, but I'm not certain on
it. Could you please let me know on where you added the directive, also did
you recompile Apache after changing the ProxyIOBufferSize ?
Thanks,
WYG
--
View this message in context: http://tomcat.10.n6.nabble.com/AJP-with-HTTPD-Buffer-Size-on-long-URLs-tp2165638p4985037.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Mike,
On 11/30/2009 3:10 AM, Looijmans, Mike wrote:
> The RFC specs a maximum URL size of 4k. That should be enough for everybody.
...along with 640k of regular memory.
I'll let you read André's and Chuck's harangues about your dubious
recollection of the HTTP specification.
On a related note, Apache httpd (used by the OP, so definitely relevant,
here) has a configuration option for limiting the length of the
first-line of the request from a client:
http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestline
The default limit in 2.0 and 2.2 is 8190 (an seemingly strange number
unless you account for the CR LF end-of-line marker required by the
specification (in section 2.2, since you asked). Presumably, httpd uses
fgets and the default buffer size of 8190 gives them a round-numbered
buffer size... though for no particular reason.
Since Apache httpd will choke after 8177 characters of URI (8190 - 13
required characters for "GET" and the HTTP version identifier), the OP
would be wise to change this setting in httpd.conf.
Or switch to POST, which is probably the right answer, here.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAksUItcACgkQ9CaO5/Lv0PB6oQCfdpF6kZpqyrglITbfEisLK4cO
MDcAoJE5HOrvzVuQpTOFNGXHT40RiQt/
=PIjv
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Posted by André Warnier <aw...@ice-sa.com>.
Caldarale, Charles R wrote:
>> From: Looijmans, Mike [mailto:mike.looijmans@oce.com]
>> Subject: RE: AJP with HTTPD - Buffer Size on long URLs
>>
>>> Looijmans, Mike wrote:
>>>> The RFC specs a maximum URL size of 4k.
>>> Where precisely did you find that ?
>> RFC2068 (old HTTP/1.1 spec)
>
> Citing an obsoleted RFC is a bit odd. Regardless, the actual wording from section 3.2.1 of 2068 and 2616 (the superseding document) is:
>
> "The HTTP protocol does not place any a priori limit on the length of a URI."
>
> Followed shortly by:
>
> "A server SHOULD return 414 (Request-URI Too Long) status if a URI is longer than the server can handle (see section 10.4.15)."
>
> (Note the SHOULD, not MUST.)
>
> There is also a warning note:
>
> "Note: Servers should be cautious about depending on URI lengths above 255 bytes, because some older client or proxy implementations may not properly support these lengths."
>
> No mention of a 4K limit anywhere that I can find.
>
Right. +1.
My point here (toward Mike) was that one should avoid propagating rumors
or incorrect information, on a list that is read by unsuspecting users
which may then believe that this is the ultimate truth.
This being said, the specs do not set a specific limit to a URI length,
but it is certain that any server software has a practical one, if only
to avoid some types of DoS attacks.
So my point to the original poster, was to recommend the use of a POST
rather than a GET, if the application is such that it already now
exceeds 8K for a URI.
In addition, even if one knows how many individual input fields there
may be in a form which sends such a URI, and how long each field is in
principle, it is much harder to predict how long a URI this will
actually generate once URI-escaping has taken place, and each non-ASCII
character has been replaced by a triplet of bytes.
There is no such arbitrary limit (or if there is, it is MUCH higher) for
the body of a POST.
In addition, at least for the body of a POST, there is a possibility of
indicating the character set of the data, which in fact there is not for
data contained in a URI.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: AJP with HTTPD - Buffer Size on long URLs
Posted by "Looijmans, Mike" <mi...@oce.com>.
I stand corrected.
What I do recall is that in the 1999's I was forced to build an HTTP/1.1
server from scratch (in objective-C) and, when faced with the question
"at what point in reading the URI should I give up and decide this is
not a HTTP request?", I found 4k to be the 'correct' answer. Since
RFC2068 was the basis for that server, I was lazy and assumef that
that's where it originated.
Anyway, when creating arbitrary long URIs, you can be sure that at some
point any HTTP server will give up, because it is more or less forced to
store the URI in precious RAM. Probably the 4k limit was intended as
"the maximum size you can expect a HTTP server to accept", anything
beyond that is "at your own peril".
The "SHOULD return 414" is easily explained: If it stops reading the
URL, it has no knowledge of the client's intended protocol yet, it is
not aware of the other headers in the request, and as such the server
may not be able to determine whether the client really expects a HTTP
response at all. So the safe thing to do is just close the connection
and give up.
Having said that, there is a very clear distinction between GET and POST
requests. The main difference is that POST requests in general have a
side-effect, and cannot be expected to return the same result twice. For
example, "POST /mything" might return "created a file" the first time,
and "file already exists" the second time.
M.
<!--
> -----Original Message-----
> From: Caldarale, Charles R [mailto:Chuck.Caldarale@unisys.com]
> Sent: maandag 30 november 2009 15:54
> To: Tomcat Users List
> Subject: RE: AJP with HTTPD - Buffer Size on long URLs
>
> > From: Looijmans, Mike [mailto:mike.looijmans@oce.com]
> > Subject: RE: AJP with HTTPD - Buffer Size on long URLs
> >
> > > Looijmans, Mike wrote:
> > > > The RFC specs a maximum URL size of 4k.
> > >
> > > Where precisely did you find that ?
> >
> > RFC2068 (old HTTP/1.1 spec)
>
> Citing an obsoleted RFC is a bit odd. Regardless, the actual
> wording from section 3.2.1 of 2068 and 2616 (the superseding
> document) is:
>
> "The HTTP protocol does not place any a priori limit on the
> length of a URI."
>
> Followed shortly by:
>
> "A server SHOULD return 414 (Request-URI Too Long) status if
> a URI is longer than the server can handle (see section 10.4.15)."
>
> (Note the SHOULD, not MUST.)
>
> There is also a warning note:
>
> "Note: Servers should be cautious about depending on URI
> lengths above 255 bytes, because some older client or proxy
> implementations may not properly support these lengths."
>
> No mention of a 4K limit anywhere that I can find.
>
> - Chuck
This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.
If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message.
Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: AJP with HTTPD - Buffer Size on long URLs
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Looijmans, Mike [mailto:mike.looijmans@oce.com]
> Subject: RE: AJP with HTTPD - Buffer Size on long URLs
>
> > Looijmans, Mike wrote:
> > > The RFC specs a maximum URL size of 4k.
> >
> > Where precisely did you find that ?
>
> RFC2068 (old HTTP/1.1 spec)
Citing an obsoleted RFC is a bit odd. Regardless, the actual wording from section 3.2.1 of 2068 and 2616 (the superseding document) is:
"The HTTP protocol does not place any a priori limit on the length of a URI."
Followed shortly by:
"A server SHOULD return 414 (Request-URI Too Long) status if a URI is longer than the server can handle (see section 10.4.15)."
(Note the SHOULD, not MUST.)
There is also a warning note:
"Note: Servers should be cautious about depending on URI lengths above 255 bytes, because some older client or proxy implementations may not properly support these lengths."
No mention of a 4K limit anywhere that I can find.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: AJP with HTTPD - Buffer Size on long URLs
Posted by "Looijmans, Mike" <mi...@oce.com>.
> Looijmans, Mike wrote:
> > The RFC specs a maximum URL size of 4k.
>
> Where precisely did you find that ?
RFC2068 (old HTTP/1.1 spec)
This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.
If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message.
Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Posted by André Warnier <aw...@ice-sa.com>.
Looijmans, Mike wrote:
> The RFC specs a maximum URL size of 4k.
Where precisely did you find that ?
As per my own memory, it is not as clear as that.
But in various places, it warns against "too long" URLs, not so much
because of the httpd server itself, but because intermediate agents may
have lower limits (proxies, firewalls, connectors..)
The Java Servlet Spec may also have something to say about this.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: AJP with HTTPD - Buffer Size on long URLs
Posted by "Looijmans, Mike" <mi...@oce.com>.
The RFC specs a maximum URL size of 4k. That should be enough for everybody.
Note that you can mix and match as required: Use the URL portion of your request to identify the target of the request, and put the additional data in the POST body.
> -----Original Message-----
> From: André Warnier [mailto:aw@ice-sa.com]
> Sent: zaterdag 28 november 2009 13:11
> To: Tomcat Users List
> Subject: Re: AJP with HTTPD - Buffer Size on long URLs
>
> Nilesh Bansal wrote:
> > Using ProxyIOBufferSize as 32192 totally worked even though the
> > documentation suggests otherwise. I am using httpd 2.2.14
> with Tomcat
> > 6.0.16. Thank you for the tip, now I can again use my long urls.
> >
> This may work for now, but someone should tell you that
> sending large amounts of data in a HTTP GET request is not
> such a good idea. It will get you in trouble sooner or later,
> for various reasons.
> You should use a POST request for that kind of thing.
>
This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.
If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message.
Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Posted by André Warnier <aw...@ice-sa.com>.
Nilesh Bansal wrote:
> Using ProxyIOBufferSize as 32192 totally worked even though the
> documentation suggests otherwise. I am using httpd 2.2.14 with Tomcat
> 6.0.16. Thank you for the tip, now I can again use my long urls.
>
This may work for now, but someone should tell you that sending large
amounts of data in a HTTP GET request is not such a good idea. It will
get you in trouble sooner or later, for various reasons.
You should use a POST request for that kind of thing.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: AJP with HTTPD - Buffer Size on long URLs
Posted by Nilesh Bansal <ni...@gmail.com>.
Using ProxyIOBufferSize as 32192 totally worked even though the
documentation suggests otherwise. I am using httpd 2.2.14 with Tomcat
6.0.16. Thank you for the tip, now I can again use my long urls.
thanks,
Nilesh
On Fri, Nov 27, 2009 at 11:32 PM, Caldarale, Charles R <
Chuck.Caldarale@unisys.com> wrote:
> > From: Nilesh Bansal [mailto:nileshbansal@gmail.com]
> > Subject: Re: AJP with HTTPD - Buffer Size on long URLs
> >
> > I am using mod_proxy_ajp (and not mod_jk)
>
> There's the ProxyIOBufferSize parameter; the 2.2 doc says the max value is
> 8192, but the 2.3 doc says the minimum is 512 with no mention of a maximum.
> Perhaps there's a change in trunk that allows the larger values - or you
> might just try it and see what happens.
>
> Or switch to mod_jk...
>
> - Chuck
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is thus for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all computers.
>
>
RE: AJP with HTTPD - Buffer Size on long URLs
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Nilesh Bansal [mailto:nileshbansal@gmail.com]
> Subject: Re: AJP with HTTPD - Buffer Size on long URLs
>
> I am using mod_proxy_ajp (and not mod_jk)
There's the ProxyIOBufferSize parameter; the 2.2 doc says the max value is 8192, but the 2.3 doc says the minimum is 512 with no mention of a maximum. Perhaps there's a change in trunk that allows the larger values - or you might just try it and see what happens.
Or switch to mod_jk...
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
Re: AJP with HTTPD - Buffer Size on long URLs
Posted by Nilesh Bansal <ni...@gmail.com>.
I am using mod_proxy_ajp (and not mod_jk) and I was not able to find any
suitable way of specifying the worker.properties file with this parameter on
max_packet_size with mod_proxy. Any pointers will be most appreciated. I am
still not sure where in httpd.conf to set the AJP specific parameters.
thanks,
Nilesh
On Fri, Nov 27, 2009 at 10:44 PM, Caldarale, Charles R <
Chuck.Caldarale@unisys.com> wrote:
> > From: Nilesh Bansal [mailto:nileshbansal@gmail.com]
> > Subject: AJP with HTTPD - Buffer Size on long URLs
> >
> > Is there a way the buffer size can be increased?
>
> As usual, it's in the docs:
>
> http://tomcat.apache.org/connectors-doc/reference/workers.html#Advanced%20Worker%20Directives
>
> Look at the max_packet_size attribute. Note that a corresponding setting
> must be made in the AJP <Connector> element in Tomcat's conf/server.xml
> file.
>
> - Chuck
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is thus for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all computers.
>
>
RE: AJP with HTTPD - Buffer Size on long URLs
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Nilesh Bansal [mailto:nileshbansal@gmail.com]
> Subject: AJP with HTTPD - Buffer Size on long URLs
>
> Is there a way the buffer size can be increased?
As usual, it's in the docs:
http://tomcat.apache.org/connectors-doc/reference/workers.html#Advanced%20Worker%20Directives
Look at the max_packet_size attribute. Note that a corresponding setting must be made in the AJP <Connector> element in Tomcat's conf/server.xml file.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.