[I] [Bug] JDBC sink flush() recurses and throws StackOverflowError under sustained backlog [pulsar-connectors]

["darinspivey (via GitHub)" <gi...@apache.org>]

darinspivey opened a new issue, #38:
URL: https://github.com/apache/pulsar-connectors/issues/38

   **What happened**
   
   Under sustained catch-up (a backlog is present and message arrival outpaces the flush rate), `JdbcAbstractSink.flush()` recurses instead of looping and eventually throws `StackOverflowError`. The handler is `catch (Exception)`, so the `Error` is not caught. The `ScheduledThreadPoolExecutor` running the flush swallows it and cancels the periodic flush task. The sink then hangs silently: 1/1 Ready, no restart, no logged error, zero writes and zero acks, and the consumer blocks at its unacked cap, stalling the whole Shared subscription.
   
   **Where**
   
   `jdbc/core/src/main/java/org/apache/pulsar/io/jdbc/JdbcAbstractSink.java`, the `if (needAnotherRound) { flush(); }` tail call in `flush()`. Each queued full batch adds a stack frame, so the recursion depth grows with the backlog.
   
   **Affected versions**
   
   Present in v4.1.3, branch-4.1, and current `master`.
   
   **Impact**
   
   Any JDBC sink that falls behind (backlog recovery, traffic spike) can wedge permanently with no signal until throughput is noticed sitting at zero. Restarting the instance clears it until the next time it falls behind.
   
   **Fix**
   
   Drain iteratively with a `do { ... } while (needAnotherRound)` loop instead of recursing, and widen the handler to `catch (Throwable)` so an `Error` still reaches `fatal()` and restarts the instance instead of vanishing. PR to follow.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] JDBC sink flush() recurses and throws StackOverflowError under sustained backlog [pulsar-connectors]

["david-streamlio (via GitHub)" <gi...@apache.org>]

david-streamlio closed issue #38: [Bug] JDBC sink flush() recurses and throws StackOverflowError under sustained backlog
URL: https://github.com/apache/pulsar-connectors/issues/38


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org