You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Alex Holmes (JIRA)" <ji...@apache.org> on 2008/01/24 22:34:37 UTC
[jira] Created: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
--------------------------------------------------------------------------------
Key: HTTPCLIENT-732
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
Project: HttpComponents HttpClient
Issue Type: Bug
Components: HttpClient
Affects Versions: 3.1.1
Reporter: Alex Holmes
The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
Thanks,
Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Holmes updated HTTPCLIENT-732:
-----------------------------------
Attachment: TestStreams.java.diff
HttpMethodParams.java.diff
ChunkedInputStream.java.diff
ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diff, HttpGet.java, HttpMethodParams.java.diff, TestStreams.java.diff
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Holmes updated HTTPCLIENT-732:
-----------------------------------
Attachment: (was: TestStreams.java.diff)
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diffu, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Ortwin Glück (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562596#action_12562596 ]
Ortwin Glück commented on HTTPCLIENT-732:
-----------------------------------------
"API compatible" means: if an application uses HttpClient 3.1 it should not break with your patch included (i.e. 3.2). Do not worry about 4.0 as it is a very different API anyway. For the 4.0 code base please see http://hc.apache.org/httpcomponents-client/httpclient/source-repository.html
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Final
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diffu, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Resolved: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Kalnichevski resolved HTTPCLIENT-732.
------------------------------------------
Resolution: Won't Fix
Alex,
It is still in alpha, but we expect it to get to the RC (release candidate) stage towards the end of the year.
I am closing the issue as WONTFIX. Anyone who is prepared to produce an API compatible patch against HttpClient 3.x trunk including reasonable test coverage for the new code is welcome to re-open it.
Oleg
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diffu, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Ortwin Glück (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ortwin Glück updated HTTPCLIENT-732:
------------------------------------
Affects Version/s: (was: 3.1.1)
3.1 Final
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Final
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diffu, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Ortwin Glück (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562450#action_12562450 ]
Ortwin Glück commented on HTTPCLIENT-732:
-----------------------------------------
Alex, it's a valid concern. Could you resubmit your patches in unidiff format (diff -u), please?
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diff, HttpGet.java, HttpMethodParams.java.diff, TestStreams.java.diff
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Reopened: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Holmes reopened HTTPCLIENT-732:
------------------------------------
Given that the 4.0 RC is towards the end of the year, wouldn't it be worthwhile creating the patch? My diff files above were created against the 3.0.1 codebase, but I'll be happy to put in the work to create an "API compatible patch" (I'm assuming you mean use the 4.0 version methods to be forward compatible), and I can modify my existing unit tests.
If I can get read access to the 4.0 codebase I'm willing to put in the work.
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diffu, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Holmes updated HTTPCLIENT-732:
-----------------------------------
Comment: was deleted
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diff, HttpGet.java, HttpMethodParams.java.diff, TestStreams.java.diff
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Resolved: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Kalnichevski resolved HTTPCLIENT-732.
------------------------------------------
Resolution: Won't Fix
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Final
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diffu, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Holmes updated HTTPCLIENT-732:
-----------------------------------
Attachment: TestStreams.java.diffu
HttpMethodParams.java.diffu
ChunkedInputStream.java.diffu
Attaching files using "diff -u" method
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diff, HttpMethodParams.java.diffu, TestStreams.java.diff, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562454#action_12562454 ]
Oleg Kalnichevski commented on HTTPCLIENT-732:
----------------------------------------------
Guys,
This problem has been fixed in the 4.0 codeline. I see no point in fixing it in HttpClient 3.x unless you insist.
Oleg
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diff, HttpGet.java, HttpMethodParams.java.diff, TestStreams.java.diff
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562605#action_12562605 ]
Oleg Kalnichevski commented on HTTPCLIENT-732:
----------------------------------------------
Alex, et al
With all due respect I do not see a point in fixing OME issues in chunk codec and not fixing them in the HTTP header parser. I also see little sense in adding more hacks to a codebase that is already full of ugly hacks. Anyways, I'll happily check in the patch (just one patch in unified diff format, not many) that
(1) retains full API compatibility with 3.1 API (including JRE 1.2.2 compatibility)
(2) has a decent test coverage of the new code
(3) does not break any of the existing test cases
Oleg
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1 Final
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diffu, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562566#action_12562566 ]
Alex Holmes commented on HTTPCLIENT-732:
----------------------------------------
I've patched the commons code locally so it's not a pressing concern. Can I ask when the 4.0 codeline will be out of beta?
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diff, HttpGet.java, HttpMethodParams.java.diff, TestStreams.java.diff
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Holmes updated HTTPCLIENT-732:
-----------------------------------
Attachment: HttpGet.java
Sample client demonstrating OutOfMemoryError (when pointed at location with corrupted chink-size line: http://www.pepoweb.com/gallery/)
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diff, HttpGet.java, HttpMethodParams.java.diff, TestStreams.java.diff
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Holmes updated HTTPCLIENT-732:
-----------------------------------
Attachment: (was: ChunkedInputStream.java.diff)
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diff, HttpMethodParams.java.diffu, TestStreams.java.diff, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Holmes updated HTTPCLIENT-732:
-----------------------------------
Attachment: (was: HttpMethodParams.java.diff)
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diffu, HttpGet.java, HttpMethodParams.java.diffu, TestStreams.java.diffu
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-732) Corrupted chunk-size field can
cause OutOfMemory exception on ChunkedInputStream
Posted by "Alex Holmes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562242#action_12562242 ]
Alex Holmes commented on HTTPCLIENT-732:
----------------------------------------
Another URL that exhibits this problem: http://erfolgreichepraxis.com/
Interestingly, sometimes I see the OutOfMemoryError and other times the connection is reset (on both links).
> Corrupted chunk-size field can cause OutOfMemory exception on ChunkedInputStream
> --------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-732
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-732
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 3.1.1
> Reporter: Alex Holmes
> Attachments: ChunkedInputStream.java.diff, HttpGet.java, HttpMethodParams.java.diff, TestStreams.java.diff
>
>
> The ChunkedInputStream.getChunkSizeFromInputStream method keeps reading the chunk-size+extension line until it reaches the end of the line. However with corrupted chunked lines of sufficient size, it keeps reading content into the ByteArrayOutputStream until an OutOfMemory exception occurs.
> I'm attaching a test client which demonstrates this behavior. An example of a URL with a corrupted chunk-size line that causes this exception is here: http://www.pepoweb.com/gallery/
> It would be useful to have a mechanism by which the max length for both the chunk-size and chunk-extension fields can be configured. I'm attaching diff patches that provide two additional configurable parameters enabling a max byte size for both fields, along with unit tests to test the changes.
> The patches are based off the 3.0.1 codebase - if there's interest in this fix, I'll be happy to generate diff's for newer codebases.
> ChunkedInputStream - now throws IOException if max limits on chunk-size/chunk-extension fields are supplied
> HttpMethodParams - two additional configurables to set max limits on chunk-size/chunk-extension fields (defaults are unlimited)
> TestStreams - two new methods to test the max chunk-size/chunk-extension behavior of the ChunkedInputStream
> A key point is that this code is backwards compatible; the default behavior of the ChunkedInputStream is unchanged (unlimited # of bytes read for chunk-size/chunk-extension fields).
> Thanks,
> Alex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org