You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Oleg Kalnichevski (JIRA)" <ji...@apache.org> on 2006/05/15 23:18:06 UTC
[jira] Resolved: (HTTPCORE-3) HttpParser triggers unfriendly
OutOfMemoryError on challenging input
[ http://issues.apache.org/jira/browse/HTTPCORE-3?page=all ]
Oleg Kalnichevski resolved HTTPCORE-3:
--------------------------------------
Resolution: Invalid
Feel free to re-open the issue if you think the problem has not been adequately resolved
Oleg
> HttpParser triggers unfriendly OutOfMemoryError on challenging input
> --------------------------------------------------------------------
>
> Key: HTTPCORE-3
> URL: http://issues.apache.org/jira/browse/HTTPCORE-3
> Project: Jakarta HttpCore
> Type: Bug
> Components: HttpCore
> Reporter: Gordon Mohr
>
> Many users of HttpClient use it to connect to servers which generate challenging HTTP responses, such as responses which include an arbitrarily large number of headers or headers of arbitrarily large size. (Sometimes such headers are conformant with the spec, in that they contain legal characters in plausible header formats; other times these are filled with binary content that is a violation of the relevant specs. Even when technically legal, often such giant headers are the inadvertent result of server-side bugs.)
> As a Java execution environment always has a hard cap on the available heap space, any parsing code which can use an arbitrary amount of memory risks triggering an OutOfMemoryError, either in its own thread or even another thread that happens to need memory after the parsing thread has exhausted it all.
> Such OutOfMemoryErrors are a particularly unfriendly way to indicate that a practical limit has been exceeded, compared to other options. They can hide the thread of execution which is most to blame. It is hard and awkward to set up handlers that catch and recover from OOMEs wherever they are most likely to occur. Even with such handlers, the actual allocation triggering an OOME may occur in another critical thread, even if that thread has minimal and well-controlled memory needs.
> HttpClient ought to provide one or more ways for a user to protect against such OOMEs, and instead receive a more convenient/recoverable indication of an HTTP response that is impossible to process with the HttpClient library within the available resources. Many approaches are possible; the easiest would be to allow a user of HttpClient to set their own optional, pragmatic limits on header sizes and number. Then, just as a user may already cleanly cancel the stream-reading of an arbitrarily-long content-body without fouling up their application state, they would be able to cancel the parsing of oversized response headers.
> Similar issues have been discussed before, for example in Bugzilla bug #25468 (http://issues.apache.org/bugzilla/show_bug.cgi?id=25468) which was to "Provide HttpParser plug-in mechanism." Though that issue is marked resolved/fixed, there is no such plug-in mechanism allowing an OOME workaround in the 3.x HttpClient, and it is not clear that a mechanism/work-around exists in whatever 4.0 work has been completed.
> So my suggestion is that this new issue be used to uniquely track the OOME risk in HttpParser, and only be considered "fixed" when some version of HttpClient offers an alternative to throwing OOMEs as a way of dealing with challenging HTTP responses. Alternatively, this could simply become the issue in the new system for collecting user-contributed workarounds/patches.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [jira] Resolved: (HTTPCORE-3) HttpParser triggers unfriendly
OutOfMemoryError on challenging input
Posted by Gordon Mohr <go...@archive.org>.
Ortwin Glück wrote:
>
>
> Gordon Mohr wrote:
>> If I understand the HttpCore code properly, there is no direct
>> facility for protecting against the OOME in the code -- just a chance
>> to hook in a theoretical alternate implementation that would address
>> the problem.
>>
>> Is that correct?
>
> Yes.
>
>> To use the HttpCore-4.0 facility, it appears I would create my own
>> HttpDataReceiver implementation which keeps a count of the bytes it
>> shovels & throws an IO or HTTP exception when some count is exceeded;
>> create a factory that makes such receivers; install that factory into
>> each HttpClientConnection instance before it begins receiving data.
>
> Correct.
OK, thanks.
>> This could work, but seems a roundabout and obscure approach. The
>> really valuable feature would be for OOME-resistance -- and friendly,
>> usable indicators that extreme content has been encountered -- to be
>> features of the library. It's require a switch or paramter to enable,
>> rather than patching in custom/third-party code.
>
> An OOM Exception is exactly that: a friendly usable indicator that an
> extreme condition has been encountered. I am personally against another
> mechanism that tries to somehow monitor heap memory.
I'd have to strongly disagree that OOMEs are friendly and usable. Among
the reasons I consider it unfriendly:
(1) OOMEs can occur anywhere, making it hard to catch and recover from
(2) pre-JDK 1.6, there's no stack trace info
(3) OOMEs only occasionally occur near the 'culpable' code
(3) is most troublesome for high-volume multi-threaded apps. If thread A
is behaving nicely, using 1MB and freeing it, endlessly -- but then
thread B comes along and grabs 1000MB, it's very possible that
well-behaved A, not gluttonous B, receives the OOME. Given that A can be
a crucial task whose unrecoverable error is fatal to your application,
this is a serious and "unfriendly" problem.
(It's bitten my project constantly -- from HttpClient most recently, but
several other open-source libraries as well.)
Thus it's nice to have libraries that at least have a mode where they
use a practically bounded amount of memory. Some libraries do this well;
I would like HttpClient(/HttpCore) to do whis well too, without
requiring custom alternate implementations to be hooked in. It would
make it a more robust library for many purposes.
> I agree that by sending large header information a malicious server
> could DOS a spider based on HttpCore. So it makes sense for this use
> case to include a protection parameter that sets an upper limit to the
> information in headers. A default of 100 KB should be enough for the
> real world. This is the easy solution.
I agree. It wouldn't even need to be a default -- the default could be
"take your chances", because the HTTP spec has no limits. But it'd be
nice if setting such a limit were a built-in option.
My application currently requires a fix for this in the 3.x line, so
I'll either use Christian's approach or work up my own. It'd be great to
see this capability in 3.1, especially if it's HttpClient-3.1 is still
pre-alpha and release ~4 months away.
If it can be opened as an RFE against 4.0, just to track that it's
pending and a priority, I'd also be happy to contribute an
implementation when my project gets around to using the 4.0 line (where
we'll also need this).
> Another possibility is to use a header stream like we do for the message
> body. This would mean a refactoring and probably cause painful client
> code. We would still have to protect ourselves against long header lines
> which is again not easy.
Thought about this too, but it'd be quite a contortion for a need --
truly handling arbitrarily long headers -- that's impractical if not
completely mythical.
> This all boils down to a missing feature: byte counting in the connection.
Agreed, that'd be one simple way to do it.
>> Is there a summary of expected dates of Core-4.0/Client-4.0 release
>> somewhere, or any assessments of how the 4.0 codebases match up
>> against 3.0 features? (Is it reasonable for an HttpClient-3.0-using
>> project to consider transitioning to the 4.0 codebase(s)?)
>
> This is the only plan there is:
> http://wiki.apache.org/jakarta-httpclient/HttpComponentsProjectRoadmap
Great, thanks.
- Gordon @ IA
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [jira] Resolved: (HTTPCORE-3) HttpParser triggers unfriendly
OutOfMemoryError on challenging input
Posted by Ortwin Glück <od...@odi.ch>.
Gordon Mohr wrote:
> If I understand the HttpCore code properly, there is no direct facility
> for protecting against the OOME in the code -- just a chance to hook in
> a theoretical alternate implementation that would address the problem.
>
> Is that correct?
Yes.
> To use the HttpCore-4.0 facility, it appears I would create my own
> HttpDataReceiver implementation which keeps a count of the bytes it
> shovels & throws an IO or HTTP exception when some count is exceeded;
> create a factory that makes such receivers; install that factory into
> each HttpClientConnection instance before it begins receiving data.
Correct.
> This could work, but seems a roundabout and obscure approach. The really
> valuable feature would be for OOME-resistance -- and friendly, usable
> indicators that extreme content has been encountered -- to be features
> of the library. It's require a switch or paramter to enable, rather than
> patching in custom/third-party code.
An OOM Exception is exactly that: a friendly usable indicator that an
extreme condition has been encountered. I am personally against another
mechanism that tries to somehow monitor heap memory.
I agree that by sending large header information a malicious server
could DOS a spider based on HttpCore. So it makes sense for this use
case to include a protection parameter that sets an upper limit to the
information in headers. A default of 100 KB should be enough for the
real world. This is the easy solution.
Another possibility is to use a header stream like we do for the message
body. This would mean a refactoring and probably cause painful client
code. We would still have to protect ourselves against long header lines
which is again not easy.
This all boils down to a missing feature: byte counting in the connection.
> Is there a summary of expected dates of Core-4.0/Client-4.0 release
> somewhere, or any assessments of how the 4.0 codebases match up against
> 3.0 features? (Is it reasonable for an HttpClient-3.0-using project to
> consider transitioning to the 4.0 codebase(s)?)
This is the only plan there is:
http://wiki.apache.org/jakarta-httpclient/HttpComponentsProjectRoadmap
> - Gordon @ IA
Ortwin
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [jira] Resolved: (HTTPCORE-3) HttpParser triggers unfriendly
OutOfMemoryError on challenging input
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2006-05-16 at 15:24 -0700, Gordon Mohr wrote:
> If I understand the HttpCore code properly, there is no direct facility
> for protecting against the OOME in the code -- just a chance to hook in
> a theoretical alternate implementation that would address the problem.
>
> Is that correct?
>
> To use the HttpCore-4.0 facility, it appears I would create my own
> HttpDataReceiver implementation which keeps a count of the bytes it
> shovels & throws an IO or HTTP exception when some count is exceeded;
> create a factory that makes such receivers; install that factory into
> each HttpClientConnection instance before it begins receiving data.
>
> This could work, but seems a roundabout and obscure approach. The really
> valuable feature would be for OOME-resistance -- and friendly, usable
> indicators that extreme content has been encountered -- to be features
> of the library. It's require a switch or paramter to enable, rather than
> patching in custom/third-party code.
>
Gordon,
We have to strike a delicate balance between at times conflicting
requirements of different user groups. A sizable proportion of our users
use HttpClient exclusively to interact with a limited number of
controlled HTTP servers that are known to be standards compliant. For
those who need to be prepared to deal with malformed or even malicious
content we provide mechanisms to inject custom built components intended
to address application (context) specific problems. There are hundreds
of CGI scripts out there that are broken in all sorts of wonderful ways.
The stock versions of HttpClient / HttpCore cannot be expected to be
able to deal with all these cases.
Having said that I do think an optional limit on the HTTP line length
should be reasonable provided it has a minimal impact on performance.
Please open a feature request and feel free to submit a patch against
HttpCore trunk.
> Is there a summary of expected dates of Core-4.0/Client-4.0 release
> somewhere, or any assessments of how the 4.0 codebases match up against
> 3.0 features? (Is it reasonable for an HttpClient-3.0-using project to
> consider transitioning to the 4.0 codebase(s)?)
>
HttpCore is known to be 30% to 40% faster than HttpClient 3.0 and to
generate _much_ less intermediate garbage on the heap per HTTP
request/response. And this is not the end of the story. I believe a
better performance and a much smaller footprint should alone be strong
incentives to upgrade. Feature wise, HttpClient 4.0 should be comparable
to HttpClient 3.1
Cheers,
Oleg
> - Gordon @ IA
>
> Oleg Kalnichevski (JIRA) wrote:
> > [ http://issues.apache.org/jira/browse/HTTPCORE-3?page=all ]
> >
> > Oleg Kalnichevski resolved HTTPCORE-3:
> > --------------------------------------
> >
> > Resolution: Invalid
> >
> > Feel free to re-open the issue if you think the problem has not been adequately resolved
> >
> > Oleg
> >
> >> HttpParser triggers unfriendly OutOfMemoryError on challenging input
> >> --------------------------------------------------------------------
> >>
> >> Key: HTTPCORE-3
> >> URL: http://issues.apache.org/jira/browse/HTTPCORE-3
> >> Project: Jakarta HttpCore
> >> Type: Bug
> >
> >> Components: HttpCore
> >> Reporter: Gordon Mohr
> >
> >> Many users of HttpClient use it to connect to servers which generate challenging HTTP responses, such as responses which include an arbitrarily large number of headers or headers of arbitrarily large size. (Sometimes such headers are conformant with the spec, in that they contain legal characters in plausible header formats; other times these are filled with binary content that is a violation of the relevant specs. Even when technically legal, often such giant headers are the inadvertent result of server-side bugs.)
> >> As a Java execution environment always has a hard cap on the available heap space, any parsing code which can use an arbitrary amount of memory risks triggering an OutOfMemoryError, either in its own thread or even another thread that happens to need memory after the parsing thread has exhausted it all.
> >> Such OutOfMemoryErrors are a particularly unfriendly way to indicate that a practical limit has been exceeded, compared to other options. They can hide the thread of execution which is most to blame. It is hard and awkward to set up handlers that catch and recover from OOMEs wherever they are most likely to occur. Even with such handlers, the actual allocation triggering an OOME may occur in another critical thread, even if that thread has minimal and well-controlled memory needs.
> >> HttpClient ought to provide one or more ways for a user to protect against such OOMEs, and instead receive a more convenient/recoverable indication of an HTTP response that is impossible to process with the HttpClient library within the available resources. Many approaches are possible; the easiest would be to allow a user of HttpClient to set their own optional, pragmatic limits on header sizes and number. Then, just as a user may already cleanly cancel the stream-reading of an arbitrarily-long content-body without fouling up their application state, they would be able to cancel the parsing of oversized response headers.
> >> Similar issues have been discussed before, for example in Bugzilla bug #25468 (http://issues.apache.org/bugzilla/show_bug.cgi?id=25468) which was to "Provide HttpParser plug-in mechanism." Though that issue is marked resolved/fixed, there is no such plug-in mechanism allowing an OOME workaround in the 3.x HttpClient, and it is not clear that a mechanism/work-around exists in whatever 4.0 work has been completed.
> >> So my suggestion is that this new issue be used to uniquely track the OOME risk in HttpParser, and only be considered "fixed" when some version of HttpClient offers an alternative to throwing OOMEs as a way of dealing with challenging HTTP responses. Alternatively, this could simply become the issue in the new system for collecting user-contributed workarounds/patches.
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [jira] Resolved: (HTTPCORE-3) HttpParser triggers unfriendly
OutOfMemoryError on challenging input
Posted by Gordon Mohr <go...@archive.org>.
If I understand the HttpCore code properly, there is no direct facility
for protecting against the OOME in the code -- just a chance to hook in
a theoretical alternate implementation that would address the problem.
Is that correct?
To use the HttpCore-4.0 facility, it appears I would create my own
HttpDataReceiver implementation which keeps a count of the bytes it
shovels & throws an IO or HTTP exception when some count is exceeded;
create a factory that makes such receivers; install that factory into
each HttpClientConnection instance before it begins receiving data.
This could work, but seems a roundabout and obscure approach. The really
valuable feature would be for OOME-resistance -- and friendly, usable
indicators that extreme content has been encountered -- to be features
of the library. It's require a switch or paramter to enable, rather than
patching in custom/third-party code.
Is there a summary of expected dates of Core-4.0/Client-4.0 release
somewhere, or any assessments of how the 4.0 codebases match up against
3.0 features? (Is it reasonable for an HttpClient-3.0-using project to
consider transitioning to the 4.0 codebase(s)?)
- Gordon @ IA
Oleg Kalnichevski (JIRA) wrote:
> [ http://issues.apache.org/jira/browse/HTTPCORE-3?page=all ]
>
> Oleg Kalnichevski resolved HTTPCORE-3:
> --------------------------------------
>
> Resolution: Invalid
>
> Feel free to re-open the issue if you think the problem has not been adequately resolved
>
> Oleg
>
>> HttpParser triggers unfriendly OutOfMemoryError on challenging input
>> --------------------------------------------------------------------
>>
>> Key: HTTPCORE-3
>> URL: http://issues.apache.org/jira/browse/HTTPCORE-3
>> Project: Jakarta HttpCore
>> Type: Bug
>
>> Components: HttpCore
>> Reporter: Gordon Mohr
>
>> Many users of HttpClient use it to connect to servers which generate challenging HTTP responses, such as responses which include an arbitrarily large number of headers or headers of arbitrarily large size. (Sometimes such headers are conformant with the spec, in that they contain legal characters in plausible header formats; other times these are filled with binary content that is a violation of the relevant specs. Even when technically legal, often such giant headers are the inadvertent result of server-side bugs.)
>> As a Java execution environment always has a hard cap on the available heap space, any parsing code which can use an arbitrary amount of memory risks triggering an OutOfMemoryError, either in its own thread or even another thread that happens to need memory after the parsing thread has exhausted it all.
>> Such OutOfMemoryErrors are a particularly unfriendly way to indicate that a practical limit has been exceeded, compared to other options. They can hide the thread of execution which is most to blame. It is hard and awkward to set up handlers that catch and recover from OOMEs wherever they are most likely to occur. Even with such handlers, the actual allocation triggering an OOME may occur in another critical thread, even if that thread has minimal and well-controlled memory needs.
>> HttpClient ought to provide one or more ways for a user to protect against such OOMEs, and instead receive a more convenient/recoverable indication of an HTTP response that is impossible to process with the HttpClient library within the available resources. Many approaches are possible; the easiest would be to allow a user of HttpClient to set their own optional, pragmatic limits on header sizes and number. Then, just as a user may already cleanly cancel the stream-reading of an arbitrarily-long content-body without fouling up their application state, they would be able to cancel the parsing of oversized response headers.
>> Similar issues have been discussed before, for example in Bugzilla bug #25468 (http://issues.apache.org/bugzilla/show_bug.cgi?id=25468) which was to "Provide HttpParser plug-in mechanism." Though that issue is marked resolved/fixed, there is no such plug-in mechanism allowing an OOME workaround in the 3.x HttpClient, and it is not clear that a mechanism/work-around exists in whatever 4.0 work has been completed.
>> So my suggestion is that this new issue be used to uniquely track the OOME risk in HttpParser, and only be considered "fixed" when some version of HttpClient offers an alternative to throwing OOMEs as a way of dealing with challenging HTTP responses. Alternatively, this could simply become the issue in the new system for collecting user-contributed workarounds/patches.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org