You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "oscerd (via GitHub)" <gi...@apache.org> on 2026/09/23 12:22:10 UTC
[PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
oscerd opened a new pull request, #26779:
URL: https://github.com/apache/camel/pull/26779
Three defects in the same two files, so one PR with a commit each.
## CAMEL-24933 — the consumer orphans a pooled exchange for every message
`PulsarMessageListener.received()` does:
```java
final Exchange exchange = PulsarMessageUtils.updateExchange(message, pulsarConsumer.createExchange(false));
```
and `updateExchange` opened with `input.copy()`. `DefaultPooledExchange.newCopy()` returns
`new DefaultExchange(this)`, so a copy is never pooled. With `camel.main.exchange-factory=pooled` that
means, per consumed message:
* the `DefaultPooledExchange` taken from the factory is never `done()` and never returned to the pool;
* the callback calls `releaseExchange(copy, false)`, and since the copy is not a `PooledExchange` the
`pooledExchange.done()` branch in `DefaultConsumer.releaseExchange` is skipped;
* `PooledExchangeFactory.release` opens with `PooledExchange ee = (PooledExchange) exchange;`, throws
`ClassCastException`, catches it, logs *"Error resetting exchange"* at DEBUG and counts a discard.
So the pool never refills, pooling costs more than it saves here, and a cast failure is thrown and
swallowed on every message. Without pooling it was "only" a second exchange allocation per message.
The exchange is now populated in place. `updateExchange` has a single caller. I left the unused
`updateExchangeWithException` alone — it is dead code, but removing a public method is an API change this
fix does not need.
## CAMEL-24934 — a failed acknowledgement was reported without its cause
```java
try {
acknowledge(consumer, message);
} catch (Exception e) {
pulsarConsumer.getExceptionHandler().handleException("Error processing exchange", exchange,
exchange.getException());
}
```
That catch sits in the `else` of `if (exchange.getException() != null)`, so `exchange.getException()` is
`null` there by construction and the caught `e` was never used. A closed consumer, an unreachable broker
or an acknowledgement timeout was logged with no cause and no stack trace, and the message came back
later with nothing in the log to explain why. Now it passes `e`, with its own wording.
## CAMEL-16073 — negatively acknowledge a failed exchange
Open since 2021. On a route failure the message was neither acked nor nacked, so it returned only when
the acknowledgement timeout expired. It is now `negativeAcknowledge`d, unless `allowManualAcknowledgement`
is enabled — in which case the route stays in charge, exactly as it already does for `acknowledge`.
**Please read this part before approving.** It changes redelivery timing, and not in the direction one
might assume. Disassembling `ConsumerImpl.negativeAcknowledge` in `pulsar-client` 4.2.4 shows it does:
```java
negativeAcksTracker.add(messageId);
unAckedMessageTracker.remove(discardBatch(messageId));
```
i.e. a nack **removes the message from the ack-timeout tracker**. `camel-pulsar` defaults
`ackTimeoutMillis` to 10 s (Pulsar's own default is 0, disabled) while `negativeAckRedeliveryDelayMicros`
defaults to 60 s, so with the component defaults a failed message now comes back after **60 s instead of
10 s**. The upgrade guide says so and gives the one-line remedy
(`negativeAckRedeliveryDelayMicros=10000000`).
The nack is the semantically correct signal — it is explicit, it feeds the nack metric and it honours
`negativeAckRedeliveryBackoff`, which this component already exposes. But if you would rather have this
behind an opt-in option, or have the default delay aligned, say so and I will change it.
## Tests
* `PulsarMessageUtilsTest.testUpdateExchangeUpdatesTheGivenExchangeInPlace` — asserts the same instance
comes back, plus headers and body.
* `PulsarMessageListenerAcknowledgementTest` — a mocked `PulsarClient` (Mockito `RETURNS_SELF` for the
fluent `ConsumerBuilder`) drives the real listener against a route that fails on one body and succeeds
on another; asserts ack on success, nack on failure, and that an acknowledgement failure reaches the
exception handler carrying its cause.
Each fix was verified by reverting it and re-running: the three reverts produce exactly the three
matching failures and nothing else. Module unit suite green (27 tests).
_Claude Code on behalf of oscerd_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26779:
URL: https://github.com/apache/camel/pull/26779#issuecomment-5794803579
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on code in PR #26779:
URL: https://github.com/apache/camel/pull/26779#discussion_r4083534919
##########
components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/PulsarMessageListener.java:
##########
@@ -73,4 +73,15 @@ private void acknowledge(final Consumer<byte[]> consumer, final Message<byte[]>
}
}
+ /**
+ * Tells the broker the message was not processed, so that it is redelivered after
+ * <tt>negativeAckRedeliveryDelayMicros</tt> instead of waiting for the acknowledgement timeout. Left to the route
+ * when manual acknowledgement is enabled, the same way {@link #acknowledge} is.
+ */
+ private void negativeAcknowledge(final Consumer<byte[]> consumer, final Message<byte[]> message) {
+ if (!endpoint.getPulsarConfiguration().isAllowManualAcknowledgement()) {
+ consumer.negativeAcknowledge(message.getMessageId());
Review Comment:
Please pass the message itself here. In pulsar-client, `negativeAcknowledge(MessageId)` records redelivery count 0, while `negativeAcknowledge(Message)` passes `message.getRedeliveryCount()` to `negativeAckRedeliveryBackoff.next(count)`. With the id only, a configured backoff always uses its first delay and never grows, which contradicts the upgrade guide.
```suggestion
consumer.negativeAcknowledge(message);
```
The verify in `PulsarMessageListenerAcknowledgementTest` (line 78) then needs to expect the message instead of `messageId`.
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -2580,3 +2580,17 @@ Routes that reference the constants (for example `setHeader(MustacheConstants.MU
are unaffected. Routes that set the header by its literal string name, or that use
`allowTemplateFromHeader=true` with the old header names, must switch to the new `Camel`-prefixed
names.
+
+=== camel-pulsar - a failed exchange is negatively acknowledged
+
+When a route fails, the consumer now calls `negativeAcknowledge` on the Pulsar consumer instead of
+leaving the message unacknowledged. This only applies when `allowManualAcknowledgement` is `false`
+(the default); with manual acknowledgement the route stays in charge, as before.
+
+This changes when the message comes back. Previously it was redelivered once the acknowledgement
+timeout expired, which `camel-pulsar` sets to 10 seconds by default through `ackTimeoutMillis`. A
+negative acknowledgement removes the message from the client's unacknowledged-message tracker, so
+redelivery now follows `negativeAckRedeliveryDelayMicros`, which defaults to 60 seconds, and honours
Review Comment:
This sentence is only true once the listener passes the `Message` to `negativeAcknowledge` (see the comment in `PulsarMessageListener`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on code in PR #26779:
URL: https://github.com/apache/camel/pull/26779#discussion_r4090410592
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -2567,3 +2567,17 @@ Routes that reference the constants (for example `setHeader(MustacheConstants.MU
are unaffected. Routes that set the header by its literal string name, or that use
`allowTemplateFromHeader=true` with the old header names, must switch to the new `Camel`-prefixed
names.
+
+=== camel-pulsar - a failed exchange is negatively acknowledged
+
+When a route fails, the consumer now calls `negativeAcknowledge` on the Pulsar consumer instead of
+leaving the message unacknowledged. This only applies when `allowManualAcknowledgement` is `false`
+(the default); with manual acknowledgement the route stays in charge, as before.
+
+This changes when the message comes back. Previously it was redelivered once the acknowledgement
+timeout expired, which `camel-pulsar` sets to 10 seconds by default through `ackTimeoutMillis`. A
+negative acknowledgement removes the message from the client's unacknowledged-message tracker, so
+redelivery now follows `negativeAckRedeliveryDelayMicros`, which defaults to 60 seconds, and honours
+`negativeAckRedeliveryBackoff` when one is configured.
+
+A route that wants the previous timing can set `negativeAckRedeliveryDelayMicros=10000000`.
Review Comment:
Fair point, and I had missed it — the PR body noted the semantic change but the guide did not, and the method is on the public API surface even though the only caller is inside the component. Added in 736f68d:
```
=== camel-pulsar - PulsarMessageUtils.updateExchange returns the exchange it was given
```
covering that it now populates and returns the instance it received instead of a copy, why (the copy orphaned the pooled exchange), and that external callers relying on copy semantics must make their own.
I put it before the negative-acknowledgement entry rather than at the end of the file: three open PRs each add an entry here, and appending them all guarantees a conflict for whichever merges second.
_Claude Code on behalf of oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "gnodet-bot (via GitHub)" <gi...@apache.org>.
gnodet-bot commented on code in PR #26779:
URL: https://github.com/apache/camel/pull/26779#discussion_r4102926537
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -2839,6 +2839,29 @@ reply of a discarded exchange (`waitForTaskToComplete`) is released with that ex
`timeout`, or forever when the timeout is disabled. On completions handed over to a discarded InOnly exchange, such as
the commit or rollback of the consumer that received the message, now run as a failure, where previously they never ran.
+=== camel-pulsar - PulsarMessageUtils.updateExchange returns the exchange it was given
+
+`PulsarMessageUtils.updateExchange(message, exchange)` used to return a *copy* of the exchange passed to
+it. It now populates and returns that same instance.
+
+The copy orphaned the exchange the consumer had taken from the exchange factory, so with
+`camel.main.exchange-factory=pooled` every consumed message leaked one pooled exchange and the pool never
+refilled. Code outside the component that called this method and relied on getting an independent copy
+must make its own copy instead.
+
+=== camel-pulsar - a failed exchange is negatively acknowledged
+
+When a route fails, the consumer now calls `negativeAcknowledge` on the Pulsar consumer instead of
+leaving the message unacknowledged. This only applies when `allowManualAcknowledgement` is `false`
+(the default); with manual acknowledgement the route stays in charge, as before.
+
+This changes when the message comes back. Previously it was redelivered once the acknowledgement
+timeout expired, which `camel-pulsar` sets to 10 seconds by default through `ackTimeoutMillis`. A
+negative acknowledgement removes the message from the client's unacknowledged-message tracker, so
+redelivery now follows `negativeAckRedeliveryDelayMicros`, which defaults to 60 seconds, and honours
+`negativeAckRedeliveryBackoff` when one is configured.
+
+A route that wants the previous timing can set `negativeAckRedeliveryDelayMicros=10000000`.
=== camel-seda - multipleConsumers broadcasts to consumers with different uri options
With `multipleConsumers=true` every consumer of a SEDA queue now receives a copy of each message, also when the consumers
Review Comment:
💡 **Nit (non-blocking):** AsciiDoc needs a blank line before the next `===` heading (`camel-seda`) or it won't render as a section title. Add a blank line after this one.
```suggestion
A route that wants the previous timing can set `negativeAckRedeliveryDelayMicros=10000000`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on code in PR #26779:
URL: https://github.com/apache/camel/pull/26779#discussion_r4103084178
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -2839,6 +2839,29 @@ reply of a discarded exchange (`waitForTaskToComplete`) is released with that ex
`timeout`, or forever when the timeout is disabled. On completions handed over to a discarded InOnly exchange, such as
the commit or rollback of the consumer that received the message, now run as a failure, where previously they never ran.
+=== camel-pulsar - PulsarMessageUtils.updateExchange returns the exchange it was given
+
+`PulsarMessageUtils.updateExchange(message, exchange)` used to return a *copy* of the exchange passed to
+it. It now populates and returns that same instance.
+
+The copy orphaned the exchange the consumer had taken from the exchange factory, so with
+`camel.main.exchange-factory=pooled` every consumed message leaked one pooled exchange and the pool never
+refilled. Code outside the component that called this method and relied on getting an independent copy
+must make its own copy instead.
+
+=== camel-pulsar - a failed exchange is negatively acknowledged
+
+When a route fails, the consumer now calls `negativeAcknowledge` on the Pulsar consumer instead of
+leaving the message unacknowledged. This only applies when `allowManualAcknowledgement` is `false`
+(the default); with manual acknowledgement the route stays in charge, as before.
+
+This changes when the message comes back. Previously it was redelivered once the acknowledgement
+timeout expired, which `camel-pulsar` sets to 10 seconds by default through `ackTimeoutMillis`. A
+negative acknowledgement removes the message from the client's unacknowledged-message tracker, so
+redelivery now follows `negativeAckRedeliveryDelayMicros`, which defaults to 60 seconds, and honours
+`negativeAckRedeliveryBackoff` when one is configured.
+
+A route that wants the previous timing can set `negativeAckRedeliveryDelayMicros=10000000`.
=== camel-seda - multipleConsumers broadcasts to consumers with different uri options
With `multipleConsumers=true` every consumer of a SEDA queue now receives a copy of each message, also when the consumers
Review Comment:
Fixed in 9d9e71f. The merge of `main` into this branch dropped that blank line — it was not in the original entry.
I also swept the whole file: the only other `===` headings missing one are `camel-azure-eventgrid` (921), `camel-langchain4j` (1357) and `camel-salesforce` (1401), and all three are pre-existing on `main`, so I left them rather than widen this PR.
_Claude Code on behalf of oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on code in PR #26779:
URL: https://github.com/apache/camel/pull/26779#discussion_r4084256876
##########
components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/PulsarMessageListener.java:
##########
@@ -73,4 +73,15 @@ private void acknowledge(final Consumer<byte[]> consumer, final Message<byte[]>
}
}
+ /**
+ * Tells the broker the message was not processed, so that it is redelivered after
+ * <tt>negativeAckRedeliveryDelayMicros</tt> instead of waiting for the acknowledgement timeout. Left to the route
+ * when manual acknowledgement is enabled, the same way {@link #acknowledge} is.
+ */
+ private void negativeAcknowledge(final Consumer<byte[]> consumer, final Message<byte[]> message) {
+ if (!endpoint.getPulsarConfiguration().isAllowManualAcknowledgement()) {
+ consumer.negativeAcknowledge(message.getMessageId());
Review Comment:
Applied in a46191 — and you were right for a reason I had not checked. Disassembling `NegativeAcksTracker` in pulsar-client 4.2.4:
```
add(MessageId) -> add(messageId, 0)
add(Message) -> add(message.getMessageId(), message.getRedeliveryCount())
add(MessageId, int redeliveryCount) -> negativeAckRedeliveryBackoff.next(redeliveryCount)
```
So with the id overload the backoff is consulted but always asked for the delay of attempt **zero**, and never escalates. The upgrade guide claim only holds with the `Message` overload. The test now verifies `negativeAcknowledge(message)` and asserts the id overload is *not* used, and reverting to the id overload fails it.
Both optional items are in too: the nack is sent **before** the exception handler, so a throwing custom handler cannot cost us the nack, and there is a new test that `allowManualAcknowledgement=true` plus a failed exchange sends nothing to the broker. Removing the manual-ack guard fails that test.
On the 10 s vs 60 s question — thanks, keeping it as is with the upgrade-guide note, no opt-in option.
The branch is rebased past the `CatalogDocExamplesTest` breakage, so CI should be clean now.
_Claude Code on behalf of oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "apupier (via GitHub)" <gi...@apache.org>.
apupier commented on PR #26779:
URL: https://github.com/apache/camel/pull/26779#issuecomment-5796221330
```
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.16.0:testCompile (default-testCompile) on project camel-jbang-core: Compilation failure
Error: [ERROR] /home/runner/work/camel/camel/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/CatalogDocExamplesTest.java:[100,49] cannot find symbol
```
problem was on main branch and is fixed by https://github.com/apache/camel/commit/fa8ab2a7a4b28e3d3f7bc8fffada15bf12003921, please rebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #26779:
URL: https://github.com/apache/camel/pull/26779
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on PR #26779:
URL: https://github.com/apache/camel/pull/26779#issuecomment-5810867267
Rebased onto current `main` (`760cf484`) to clear a conflict with the `camel-seda` entry that landed at the end of the 4.23 upgrade guide. All six commits replayed; the four reviewed source files are byte-identical to the pre-rebase head and the module suite is green (28 tests).
@davsclaus this also carries everything from the review round — the `Message` overload for `negativeAcknowledge`, the nack ahead of the exception handler, the `allowManualAcknowledgement` test, and the `updateExchange` upgrade-guide entry @gnodet-bot asked for.
_Claude Code on behalf of oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "gnodet-bot (via GitHub)" <gi...@apache.org>.
gnodet-bot commented on code in PR #26779:
URL: https://github.com/apache/camel/pull/26779#discussion_r4082393806
##########
components/camel-pulsar/src/test/java/org/apache/camel/component/pulsar/PulsarMessageListenerAcknowledgementTest.java:
##########
@@ -0,0 +1,168 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.pulsar;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collections;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.camel.CamelContext;
+import org.apache.camel.Exchange;
+import org.apache.camel.RoutesBuilder;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.spi.ExceptionHandler;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.apache.pulsar.client.api.Consumer;
+import org.apache.pulsar.client.api.ConsumerBuilder;
+import org.apache.pulsar.client.api.Message;
+import org.apache.pulsar.client.api.MessageId;
+import org.apache.pulsar.client.api.PulsarClient;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Answers.RETURNS_SELF;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.timeout;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+/**
+ * A route failure used to leave the message neither acknowledged nor negatively acknowledged, so it was only
+ * redelivered once the acknowledgement timeout expired (CAMEL-16073), and a failing acknowledgement was reported
+ * without its cause.
+ */
+public class PulsarMessageListenerAcknowledgementTest extends CamelTestSupport {
+
+ private static final String ENDPOINT_URI
+ = "pulsar:persistent://public/default/camel-ack-test?subscriptionName=camel-subscription";
+
+ private final MessageId messageId = mock(MessageId.class);
+
+ @Test
+ public void testSuccessfulExchangeIsAcknowledged() throws Exception {
+ final Consumer<byte[]> pulsarConsumer = mock(Consumer.class);
+
+ listener().received(pulsarConsumer, message("ok"));
+
+ verify(pulsarConsumer, timeout(5000)).acknowledge(messageId);
Review Comment:
⚠️ **Missing test: `allowManualAcknowledgement=true` guard is untested**
The three tests here all use the default endpoint URI (no `allowManualAcknowledgement=true`), so they only cover the `isAllowManualAcknowledgement() == false` path. The new `negativeAcknowledge()` private method contains a guard:
```java
if (!endpoint.getPulsarConfiguration().isAllowManualAcknowledgement()) {
consumer.negativeAcknowledge(message.getMessageId());
}
```
If that guard is accidentally removed, no existing test catches it. The PR body says the route stays in charge when manual acknowledgement is enabled — that guarantee needs a test:
```java
@Test
public void testFailedExchangeWithManualAckDoesNotNegativelyAcknowledge() throws Exception {
// reconfigure the endpoint URI to include allowManualAcknowledgement=true
final Consumer<byte[]> pulsarConsumer = mock(Consumer.class);
// use an endpoint with allowManualAcknowledgement=true in its URI
// then trigger a route failure and verify:
verify(pulsarConsumer, never()).negativeAcknowledge(messageId);
verify(pulsarConsumer, never()).acknowledge(messageId);
}
```
(The exact wiring depends on how you expose the endpoint URI with the parameter — you may need a second CamelContext or a `@ParameterizedTest` if the current context is shared.)
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -2567,3 +2567,17 @@ Routes that reference the constants (for example `setHeader(MustacheConstants.MU
are unaffected. Routes that set the header by its literal string name, or that use
`allowTemplateFromHeader=true` with the old header names, must switch to the new `Camel`-prefixed
names.
+
+=== camel-pulsar - a failed exchange is negatively acknowledged
+
+When a route fails, the consumer now calls `negativeAcknowledge` on the Pulsar consumer instead of
+leaving the message unacknowledged. This only applies when `allowManualAcknowledgement` is `false`
+(the default); with manual acknowledgement the route stays in charge, as before.
+
+This changes when the message comes back. Previously it was redelivered once the acknowledgement
+timeout expired, which `camel-pulsar` sets to 10 seconds by default through `ackTimeoutMillis`. A
+negative acknowledgement removes the message from the client's unacknowledged-message tracker, so
+redelivery now follows `negativeAckRedeliveryDelayMicros`, which defaults to 60 seconds, and honours
+`negativeAckRedeliveryBackoff` when one is configured.
+
+A route that wants the previous timing can set `negativeAckRedeliveryDelayMicros=10000000`.
Review Comment:
⚠️ **CAMEL-24933 behavior change also needs an upgrade guide entry**
The nack redelivery change (CAMEL-16073) is documented here, but the fix to `PulsarMessageUtils.updateExchange()` (CAMEL-24933) is not. That method is `public static` and its return contract changed: it now returns the **same exchange instance** passed in, instead of a copy.
Any external code that did:
```java
Exchange copy = PulsarMessageUtils.updateExchange(msg, original);
// uses both copy and original expecting them to be independent
```
will silently break — both variables now point to the same object.
While the only known caller is internal, the method is part of the public API surface. Please add a brief upgrade guide note, e.g.:
```
=== camel-pulsar - PulsarMessageUtils.updateExchange() now returns the input exchange
Previously, `PulsarMessageUtils.updateExchange(message, exchange)` returned a copy of
the exchange. It now populates and returns the same instance it received, to prevent
orphan pooled exchanges. Custom extensions that relied on copy semantics must be updated.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26779:
URL: https://github.com/apache/camel/pull/26779#issuecomment-5799098701
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `components/camel-pulsar`
- `docs`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 of 693 tested, 26 compile-only — current: 10 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **9 affected modules** (current approach: 10).
**Skip-tests mode would test 9 modules** (2 direct + 8 downstream), **skip tests for 26** (generated code, meta-modules)
<details><summary>Modules only in current approach (1)</summary>
- `camel-catalog-suggest`
</details>
<details><summary>Modules Scalpel would test (9)</summary>
- `camel-jbang-mcp` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-mcp` ← downstream of org.apache.camel:camel-jbang-core
- `camel-jbang-plugin-route-parser` ← downstream of org.apache.camel:camel-route-parser
- `camel-jbang-plugin-tui` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-validate` ← downstream of org.apache.camel:camel-yaml-dsl-validator
- `camel-launcher-container` ← downstream of org.apache.camel:camel-launcher
- `camel-pulsar` ← components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/PulsarMessageListener.java, components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/utils/message/PulsarMessageUtils.java, components/camel-pulsar/src/test/java/org/apache/camel/component/pulsar/PulsarMessageListenerAcknowledgementTest.java, components/camel-pulsar/src/test/java/org/apache/camel/component/pulsar/utils/message/PulsarMessageUtilsTest.java
- `camel-yaml-dsl-validator` ← downstream of org.apache.camel:camel-catalog
- `camel-yaml-dsl-validator-maven-plugin` ← downstream of org.apache.camel:camel-yaml-dsl-validator
</details>
<details><summary>Modules with tests skipped (26)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
<details><summary>All tested modules (36 modules, 4m 59s total)</summary>
**Total reactor time:** 4m 59s
| Module | Duration | Status |
| --- | --- | --- |
| Camel :: Launcher | 51.8s | SUCCESS |
| Camel :: JBang :: MCP | 39.4s | SUCCESS |
| Camel :: JBang :: Plugin :: TUI | 30.1s | SUCCESS |
| Camel :: Catalog :: Camel Catalog | 22.9s | SUCCESS |
| Camel :: Component DSL | 19.7s | SUCCESS |
| Camel :: YAML DSL | 16.6s | SUCCESS |
| Camel :: Docs | 14.6s | SUCCESS |
| Camel :: JBang :: Plugin :: Kubernetes | 13.9s | SUCCESS |
| Camel :: YAML DSL :: Validator | 10.0s | SUCCESS |
| Camel :: Kamelet Main | 9.5s | SUCCESS |
| Camel :: YAML DSL :: Deserializers | 9.2s | SUCCESS |
| Camel :: Catalog :: Camel Route Parser | 7.8s | SUCCESS |
| Camel :: JBang :: Plugin :: Testing | 7.5s | SUCCESS |
| Camel :: Catalog :: Camel Report Maven Plugin | 7.5s | SUCCESS |
| Camel :: YAML DSL :: Validator Maven Plugin | 7.3s | SUCCESS |
| Camel :: All Components Sync point | 5.3s | SUCCESS |
| Camel :: JBang :: Plugin :: Validate | 5.0s | SUCCESS |
| Camel :: YAML DSL :: Maven Plugins | 3.2s | SUCCESS |
| Camel :: Catalog :: Maven | 3.1s | SUCCESS |
| Camel :: Catalog :: Suggest (deprecated) | 2.1s | SUCCESS |
| Camel :: Assembly | 1.7s | SUCCESS |
| Camel :: JBang :: Plugin :: Edit | 1.5s | SUCCESS |
| Camel :: Coverage | 1.5s | SUCCESS |
| Camel :: Catalog :: Dummy Component | 1.4s | SUCCESS |
| Camel :: Catalog :: Console | 1.2s | SUCCESS |
| Camel :: JBang :: Integration tests | 1.1s | SUCCESS |
| Camel :: JBang :: Plugin :: Generate | 1.1s | SUCCESS |
| Camel :: JBang :: Main | 0.8s | SUCCESS |
| Camel :: JBang :: Plugin :: MCP | 0.8s | SUCCESS |
| Camel :: JBang :: Plugin :: Route Parser | 0.8s | SUCCESS |
| Camel :: Endpoint DSL :: Support | 0.6s | SUCCESS |
| Camel :: Launcher :: Container | 0.6s | SUCCESS |
| Camel :: Endpoint DSL | n/a | |
| Camel :: Integration Tests | n/a | |
| Camel :: JBang :: Core | n/a | |
| Camel :: Pulsar | n/a | |
**Top 20 slowest modules:**
- `Camel :: Launcher` (51.8s)
- `Camel :: JBang :: MCP` (39.4s)
- `Camel :: JBang :: Plugin :: TUI` (30.1s)
- `Camel :: Catalog :: Camel Catalog` (22.9s)
- `Camel :: Component DSL` (19.7s)
- `Camel :: YAML DSL` (16.6s)
- `Camel :: Docs` (14.6s)
- `Camel :: JBang :: Plugin :: Kubernetes` (13.9s)
- `Camel :: YAML DSL :: Validator` (10.0s)
- `Camel :: Kamelet Main` (9.5s)
- `Camel :: YAML DSL :: Deserializers` (9.2s)
- `Camel :: Catalog :: Camel Route Parser` (7.8s)
- `Camel :: JBang :: Plugin :: Testing` (7.5s)
- `Camel :: Catalog :: Camel Report Maven Plugin` (7.5s)
- `Camel :: YAML DSL :: Validator Maven Plugin` (7.3s)
- `Camel :: All Components Sync point` (5.3s)
- `Camel :: JBang :: Plugin :: Validate` (5.0s)
- `Camel :: YAML DSL :: Maven Plugins` (3.2s)
- `Camel :: Catalog :: Maven` (3.1s)
- `Camel :: Catalog :: Suggest (deprecated)` (2.1s)
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/35882563855)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on code in PR #26779:
URL: https://github.com/apache/camel/pull/26779#discussion_r4090290903
##########
components/camel-pulsar/src/test/java/org/apache/camel/component/pulsar/PulsarMessageListenerAcknowledgementTest.java:
##########
@@ -0,0 +1,168 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.pulsar;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collections;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.camel.CamelContext;
+import org.apache.camel.Exchange;
+import org.apache.camel.RoutesBuilder;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.spi.ExceptionHandler;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.apache.pulsar.client.api.Consumer;
+import org.apache.pulsar.client.api.ConsumerBuilder;
+import org.apache.pulsar.client.api.Message;
+import org.apache.pulsar.client.api.MessageId;
+import org.apache.pulsar.client.api.PulsarClient;
+import org.apache.pulsar.client.api.PulsarClientException;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Answers.RETURNS_SELF;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.timeout;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+/**
+ * A route failure used to leave the message neither acknowledged nor negatively acknowledged, so it was only
+ * redelivered once the acknowledgement timeout expired (CAMEL-16073), and a failing acknowledgement was reported
+ * without its cause.
+ */
+public class PulsarMessageListenerAcknowledgementTest extends CamelTestSupport {
+
+ private static final String ENDPOINT_URI
+ = "pulsar:persistent://public/default/camel-ack-test?subscriptionName=camel-subscription";
+
+ private final MessageId messageId = mock(MessageId.class);
+
+ @Test
+ public void testSuccessfulExchangeIsAcknowledged() throws Exception {
+ final Consumer<byte[]> pulsarConsumer = mock(Consumer.class);
+
+ listener().received(pulsarConsumer, message("ok"));
+
+ verify(pulsarConsumer, timeout(5000)).acknowledge(messageId);
Review Comment:
Agreed, and added in a46191 — `testManualAcknowledgementLeavesTheFailedMessageToTheRoute` flips `allowManualAcknowledgement` on the endpoint configuration, drives a failing exchange through the listener, and asserts nothing at all is sent to the broker (neither overload of `negativeAcknowledge`, nor `acknowledge`). It waits on the exception handler first so the callback has demonstrably run before asserting the absence.
Removing the guard from `negativeAcknowledge` fails that test, so it is load-bearing rather than passing by construction.
_Claude Code on behalf of oscerd_
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -2580,3 +2580,17 @@ Routes that reference the constants (for example `setHeader(MustacheConstants.MU
are unaffected. Routes that set the header by its literal string name, or that use
`allowTemplateFromHeader=true` with the old header names, must switch to the new `Camel`-prefixed
names.
+
+=== camel-pulsar - a failed exchange is negatively acknowledged
+
+When a route fails, the consumer now calls `negativeAcknowledge` on the Pulsar consumer instead of
+leaving the message unacknowledged. This only applies when `allowManualAcknowledgement` is `false`
+(the default); with manual acknowledgement the route stays in charge, as before.
+
+This changes when the message comes back. Previously it was redelivered once the acknowledgement
+timeout expired, which `camel-pulsar` sets to 10 seconds by default through `ackTimeoutMillis`. A
+negative acknowledgement removes the message from the client's unacknowledged-message tracker, so
+redelivery now follows `negativeAckRedeliveryDelayMicros`, which defaults to 60 seconds, and honours
Review Comment:
Correct — and the sentence is now true: the listener passes the `Message` as of a46191.
For the record of why it matters, from the `NegativeAcksTracker` bytecode in pulsar-client 4.2.4:
```
add(MessageId) -> add(messageId, 0)
add(Message) -> add(message.getMessageId(), message.getRedeliveryCount())
add(MessageId, int redeliveryCount) -> negativeAckRedeliveryBackoff.next(redeliveryCount)
```
With the id overload the backoff is consulted but always asked for the delay of attempt zero, so it never escalates.
_Claude Code on behalf of oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24933, CAMEL-24934, CAMEL-16073: camel-pulsar - consumer acknowledgement and exchange handling [camel]
Posted by "apupier (via GitHub)" <gi...@apache.org>.
apupier commented on PR #26779:
URL: https://github.com/apache/camel/pull/26779#issuecomment-5809916448
conflict on the upgrade guide to resolve.
To limit conflict with other PRs in upgrade guide, I recommend to insert in alphabetical order (more or less because it wasn't done since the start) instead of adding at the end. This way it will reduce collision
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org