[jira] [Resolved] (HTTPCLIENT-2430) Brotli decompression fails for large payloads

["Arturo Bernal (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arturo Bernal resolved HTTPCLIENT-2430.
---------------------------------------
    Resolution: Fixed

> Brotli decompression fails for large payloads
> ---------------------------------------------
>
>                 Key: HTTPCLIENT-2430
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2430
>             Project: HttpComponents HttpClient
>          Issue Type: Bug
>          Components: HttpClient (async)
>    Affects Versions: 5.6.1
>            Reporter: Jonny Carter
>            Assignee: Arturo Bernal
>            Priority: Major
>             Fix For: 5.6.5, 5.7-alpha2
>
>
> h2. Symptom
> With transparent async content decompression (new in 5.6) and brotli4j on the classpath, any response with {{Content-Encoding: br}} whose *compressed* body is larger than 8 KB never completes. {{InflatingBrotliDataConsumer.consume()}} stops making progress once the decoder's fixed 8 KB input buffer is full: the {{xfer == 0}} branch calls {{decoder.push(0)}} / {{pump()}} in a loop that can neither advance nor fail, so {{while (src.hasRemaining())}} never exits.
> In a real async client the consequences are severe and silent: the exchange never completes, no response timeout fires, and the pooled connection is never released. Under load the pool drains connection by connection until every request to that route hangs. There is no exception and no log line anywhere.
> We found this on 5.6.4; the code is unchanged on the 5.6.x branch head. HTTPCLIENT-2428 (fixed for 5.7-alpha1) touches the same class but only replaces the capacity plumbing; the {{consume()}} loop is untouched, so 5.7-alpha1 appears to be affected as well.
> A second, smaller defect in the same class: a single spurious byte after a complete, valid brotli stream fails the exchange with {{{}IOException: Brotli stream corrupted{}}}, although the content is fully recoverable and the trailer could be ignored.
> h2. How to reproduce
> Failing unit tests against {{InflatingBrotliDataConsumerTest}} are attached and available on a GitHub branch in my fork [https://github.com/jonnybot0/httpcomponents-client/tree/bugfix-brotli] (commit 402e689774fe76ae9385e9b2e7b603a3ac11f5bc).
>  * {{inflateBrotliLargerThanInputBufferSingleBuffer}} – well-formed brotli stream, ~64 KB of incompressible (random) payload so the compressed form exceeds 8 KB, delivered as one buffer. Fails with {{{}execution timed out after 5000 ms{}}}; the timeout stack pins the busy-loop at {{InflatingBrotliDataConsumer.consume}} ({{{}decoder.push{}}} in the {{xfer == 0}} branch).
>  * {{inflateBrotliLargerThanInputBufferChunked}} – the same stream in 1 KB chunks, i.e. identical in shape to the existing passing test; the only difference is the compressed size. Same 5 s timeout.
>  * {{inflateBrotliWithTrailingByte}} – valid small stream plus one trailing byte; fails with {{{}IOException: Brotli stream corrupted{}}}.
> Note the existing happy-path test passes only because its highly compressible payload stays well under the 8 KB buffer.
> Core of the repro (the harness drives the consumer directly, no network):
> {code:java}
> final byte[] original = new byte[64 * 1024];
> new java.util.Random(42).nextBytes(original);          // incompressible => compressed > 8 KB
> final byte[] compressed = Encoder.compress(original,
>         new Encoder.Parameters().setQuality(6).setWindow(22));
> final InflatingBrotliDataConsumer inflating = new InflatingBrotliDataConsumer(rawByteCollector);
> inflating.consume(ByteBuffer.wrap(compressed));        // never returns
> inflating.streamEnd(Collections.emptyList());
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org