[jira] [Created] (CAMEL-25299) Error handler - with asyncDelayedRedelivery, allowRedeliveryWhileStopping=false is ignored: a route stop waits for the redelivery delay and then redelivers

["shashank (Jira)" <ji...@apache.org>]

shashank created CAMEL-25299:
--------------------------------

             Summary: Error handler - with asyncDelayedRedelivery, allowRedeliveryWhileStopping=false is ignored: a route stop waits for the redelivery delay and then redelivers
                 Key: CAMEL-25299
                 URL: https://issues.apache.org/jira/browse/CAMEL-25299
             Project: Camel
          Issue Type: Bug
          Components: camel-core
            Reporter: shashank


The documentation of the dead letter channel says that with {{allowRedeliveryWhileStopping=false}} any new redelivery attempt is not allowed once the stop of the route (or Camel) has been triggered: a {{RejectedExecutionException}} is set and the exchange stops being routed (the dead letter channel still receives it).

That holds for the synchronous redelivery: {{RedeliveryTask.sleep()}} waits in chunks of 1 second and checks {{preparingShutdown}} after each chunk, then marks the exchange rejected and exhausted, so it goes to the dead letter channel within a second ({{NotAllowRedeliveryWhileStoppingDeadLetterChannelTest}}).

With {{asyncDelayedRedelivery}} the delay is a task on the error handler's scheduled thread pool:

{code:java}
executorService.schedule(() -> reactiveExecutor.schedule(this::redeliver), redeliveryDelay, TimeUnit.MILLISECONDS);
{code}

and {{redeliver()}} calls the output again without any check of the shutdown state (only {{run()}} checks {{isRunAllowed()}}, and the redelivery decision in {{doRun()}} was taken before the stop). So when the route is stopped while a redelivery waits for its delay:

* the graceful stop waits for the inflight exchange for the whole remaining delay, or until the shutdown timeout (45 s by default) if the delay is longer, then stops the consumer forcibly;
* when the delay ends, the message is redelivered anyway (also after the forced stop, into the stopped route), which is what {{allowRedeliveryWhileStopping=false}} is meant to prevent.

CAMEL-3364 (2.11.0), which added {{allowRedeliveryWhileStopping}}, already named this case in its discussion: the error handler keeps the delayed exchanges as tasks in its thread pool, and on shutdown those have to be dealt with too. The implementation only made {{sleep()}} check the shutdown state; nothing in the ticket or the code says the scheduled task was meant to be exempt.

h3. Reproduction

Test {{NotAllowRedeliveryWhileStoppingAsyncDelayedTest}} (camel-core): {{deadLetterChannel("mock:dead").maximumRedeliveries(5).redeliveryDelay(1 hour, capped to the 60 s maximumRedeliveryDelay).asyncDelayedRedelivery().allowRedeliveryWhileStopping(false)}}, a route {{seda:start -> mock:foo -> throwException}}, and a scheduled thread pool for the error handler that counts down a latch when the redelivery is scheduled. After the first attempt failed and the redelivery is scheduled, the test calls {{stopRoute("foo")}} (shutdown timeout 5 s). Expected: the dead letter channel gets the exchange with {{RejectedExecutionException("Redelivery not allowed while stopping")}}, no redelivery, nothing inflight. A second test does the same with {{context.stop()}}. On main both stops run into the 5 s timeout and the dead letter channel has nothing:

{noformat}
NotAllowRedeliveryWhileStoppingAsyncDelayedTest.testStopRouteRejectsScheduledRedelivery mock://dead Received message count. Expected: <1> but was: <0>
NotAllowRedeliveryWhileStoppingAsyncDelayedTest.testStopCamelContextRejectsScheduledRedelivery mock://dead Received message count. Expected: <1> but was: <0>
{noformat}

The defect was found with a TLA+ model of the redelivery task (delivery, {{run()}}/{{doRun()}}, synchronous {{sleep()}}, the scheduled task, {{redeliver()}}) and the graceful shutdown ({{prepareShutdown}}, waiting for inflight exchanges, timeout). The invariant "no redelivery starts after its delay ended while the error handler prepares to shut down (allowRedeliveryWhileStopping=false)" is violated with {{asyncDelayedRedelivery}} in 6 steps (deliver, fail, schedule, prepareShutdown, delay ends, redeliver); it holds for the synchronous redelivery, with and without a dead letter channel, and for the fixed task.

h3. Proposed fix

When the redelivery is not allowed while stopping, the scheduled task wakes up every second, like {{sleep()}}, and when the error handler is preparing to shut down it rejects the redelivery the same way ({{RejectedExecutionException("Redelivery not allowed while stopping")}}, redelivery exhausted, back to {{run()}}, which moves the exchange to the dead letter channel or fails it). When the delay is over it redelivers as before. With {{allowRedeliveryWhileStopping=true}} (the default) nothing changes: one task for the whole delay. If the thread pool rejects the next wake-up, the redelivery is rejected the same way instead of losing the callback.

With the fix the new tests pass (each in about 1.5 s), and {{*Redelivery*}}, {{NotAllowRedelivery*}} and {{DeadLetterChannel*}} tests in camel-core pass (95 tests).

Cost: one short task per second per waiting redelivery, only with {{allowRedeliveryWhileStopping=false}} (the same rate as the synchronous {{sleep()}} loop, but without a blocked thread). One difference to {{sleep()}}: {{sleep()}} does not check at all for a delay below 1 second, while the scheduled task checks once when such a delay ends, so a short redelivery whose delay ends while stopping is rejected too (which is what the option describes). The upgrade guide gets a note, as a stop now rejects these messages within a second instead of redelivering them.

Not addressed (low): with {{allowRedeliveryWhileStopping=true}} (the default), when Camel itself is stopped and the shutdown times out, the error handler thread pool is shut down with the delayed task still in it, so the exchange is never completed; Camel is stopping and the await manager releases waiting callers. (With {{false}} the fix rejects the task within a second, before the timeout.)

Affected: 4.14.x, 4.18.x and main (same code).

Duplicate check (2026-10-03): JIRA text "allowRedeliveryWhileStopping" (CAMEL-6033, CAMEL-20744, CAMEL-3364, CAMEL-19896, CAMEL-12603), "asyncDelayedRedelivery" with stop/shutdown (none), "redelivery" with "stopping" since 2022 (11 issues, none about the asynchronous delay). GitHub pull requests "asyncDelayedRedelivery" (#1771, a doc note, 2017), "allowRedeliveryWhileStopping" (#23179, CAMEL-23494).

_Filed with Claude Code on behalf of allthingssecurity._




--
This message was sent by Atlassian Jira
(v8.20.10#820010)