You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "gnehil (via GitHub)" <gi...@apache.org> on 2026/07/04 02:38:36 UTC

[PR] [Fix][Connector-V2] Fix Doris stream load waitForContinue timeout when FE redirect is slow [seatunnel]

gnehil opened a new pull request, #11330:
URL: https://github.com/apache/seatunnel/pull/11330

   ### Purpose of this pull request
   
   Fix a Doris sink stream load failure that occurs when FE is slow to return the 307 redirect.
   
   The stream load client sends `Expect: 100-continue` and relies on FE's 307 redirect to reach BE. `HttpUtil` builds the HttpClient without a custom `HttpRequestExecutor`, so the default `waitForContinue` timeout of **3s** applies. When FE is busy (heavy load / FullGC / network jitter) and the 307 arrives later than 3s:
   
   1. The client stops waiting and sends the non-repeatable `InputStreamEntity` body to FE;
   2. When the 307 finally arrives, `RedirectExec` detects the consumed entity (`RequestEntityProxy.isRepeatable == false`), silently refuses to follow the redirect, and returns the raw 307 response to the caller;
   3. The load fails with `STREAM_LOAD_FAILED ... HTTP/1.1 307 Temporary Redirect`.
   
   This PR raises the `waitForContinue` timeout to **60s** by setting `.setRequestExecutor(new HttpRequestExecutor(60_000))` on the client builder, aligned with the same fix in doris-flink-connector (apache/doris-flink-connector#522).
   
   `HttpUtil` is the single client-building choke point on the sink side, so this covers all FE 307 redirect paths: `DorisStreamLoad` (write / pre-commit / abort) and `DorisCommitter` (2PC commit / abort).
   
   ### Does this PR introduce _any_ user-facing change?
   
   No. No config option, public API, or dependency changes — only the internal wait-for-continue timeout is raised from 3s to 60s, matching doris-flink-connector's default behavior.
   
   ### How was this patch tested?
   
   Added `HttpUtilTest#testStreamLoadClientWaitsForSlowFeRedirect`, which reproduces the scenario end-to-end:
   
   - A raw-socket FE stub reads the request headers, delays the 307 redirect by 4s (longer than the old 3s default, far shorter than the new 60s) without sending `100 Continue`. (A JDK `HttpServer` cannot simulate this — it auto-replies `100 Continue` before the handler runs.)
   - A JDK `HttpServer` BE stub receives the redirected request and returns 200.
   - The client sends an `HttpPut` built by `HttpPutBuilder` with a non-repeatable `InputStreamEntity`.
   - Asserts the request follows the 307 to BE, returns 200, and BE receives the full body.
   
   Before the fix this test fails with `expected: <200> but was: <307>` (instrumentation confirmed the body was prematurely sent to FE and BE was never hit); after the fix it passes.
   
   Also ran the full connector-doris unit test suite (111 tests, all passing), including the existing `DorisStreamLoadTest` / `DorisCommitterTest` 307-handling regressions, plus `./mvnw spotless:apply` and `./mvnw -DskipTests verify`.
   
   ### Check list
   
   * [x] If any new Jar binary package adding in your PR, please add License Notice according
     [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md) — N/A, no new dependency
   * [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs — N/A, no user-facing change
   * [x] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR. — N/A, no incompatible change
   * [ ] If you are contributing the connector code, please check that the following files are updated: — N/A, bug fix in existing connector
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix Doris stream load waitForContinue timeout when FE redirect is slow [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo merged PR #11330:
URL: https://github.com/apache/seatunnel/pull/11330


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org