You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2026/05/27 09:34:39 UTC
[PR] Core: Retry transient REST response-body read failures [iceberg]
wombatu-kun opened a new pull request, #16578:
URL: https://github.com/apache/iceberg/pull/16578
Closes #15030.
## Summary
The Iceberg REST catalog client occasionally fails with an unrecoverable `RESTException: Failed to convert HTTP response body to string` caused by `MalformedChunkCodingException` while reading a chunked/gzip response body. The failure is transient (restarting the affected process — e.g. a Kafka Connect sink — recovers), so it should be retried rather than surfaced as fatal.
The client already retries idempotent requests on transient `IOException`s via `ExponentialHttpRequestRetryStrategy`, and `MalformedChunkCodingException` is not in its non-retriable set. The gap is structural: Apache HttpClient5's retry runs in the `HttpRequestRetryExec` chain element, which only covers connecting and reading the response headers. The response body is read later, in the response handler (outside the exec chain), so a body-read failure bypasses the retry entirely.
## What changed
Added an exec-chain interceptor, registered immediately inside `HttpRequestRetryExec`, that fully buffers the response entity (`BufferedHttpEntity`). Because the body is now read inside the retry-aware part of the chain, a transient body-read `IOException` is surfaced to the existing retry strategy and retried like any other transient failure — same idempotency check, same non-retriable classification, same exponential backoff, and a fresh connection per attempt. The entity is wrapped transparently (content type and encoding preserved), so the body is still parsed exactly once by the response handler and no other code paths change.
Non-idempotent requests (e.g. commits) are not retried on a body-read failure: the retry strategy only retries idempotent methods on transient `IOException`s. This change makes idempotent body-read failures retryable while leaving non-idempotent requests unaffected — matching how the strategy already treats connection- and header-level `IOException`s.
## Tests
Added a test in `TestHTTPClient` that uses a small raw-socket server (MockServer cannot emit a malformed chunked response) returning a malformed chunked body on the first connection and a valid response on the second, asserting that an idempotent GET transparently retries on a fresh connection and succeeds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Retry transient REST response-body read failures [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16578:
URL: https://github.com/apache/iceberg/pull/16578#issuecomment-5610711585
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Retry transient REST response-body read failures [iceberg]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun closed pull request #16578: Core: Retry transient REST response-body read failures
URL: https://github.com/apache/iceberg/pull/16578
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Retry transient REST response-body read failures [iceberg]
Posted by "wombatu-kun (via GitHub) via issues" <is...@iceberg.apache.org>.
wombatu-kun closed pull request #16578: Core: Retry transient REST response-body read failures
URL: https://github.com/apache/iceberg/pull/16578
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Retry transient REST response-body read failures [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16578:
URL: https://github.com/apache/iceberg/pull/16578#issuecomment-5234608728
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Retry transient REST response-body read failures [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16578:
URL: https://github.com/apache/iceberg/pull/16578#issuecomment-4814402015
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Retry transient REST response-body read failures [iceberg]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #16578:
URL: https://github.com/apache/iceberg/pull/16578#issuecomment-4828076178
any comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org