You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@camel.apache.org by "oscerd (via GitHub)" <gi...@apache.org> on 2026/09/23 12:42:02 UTC

[PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

oscerd opened a new pull request, #26783:
URL: https://github.com/apache/camel/pull/26783

   ## What
   
   `PulsarProducer.process` did:
   
   ```java
   messageBuilder.sendAsync()
           .thenAccept(r -> exchange.getIn().setBody(r))
   ```
   
   `r` is the `MessageId` the broker assigned, and it replaced the IN body. So after a `to("pulsar:...")`:
   
   * every later step in the route saw a `MessageId` instead of the payload it had just sent;
   * `.to("pulsar:a").to("pulsar:b")` published a **Java-serialized `MessageId`** to the second topic,
     because `PulsarProducer.serialize` falls back to `ObjectOutputStream` when no type converter applies.
   
   Nothing in `pulsar-component.adoc` mentioned it and no test covered it, so it does not read as a
   deliberate contract — but it has been the behaviour for a long time, hence the upgrade-guide entry.
   
   ## Change
   
   Leave the body untouched and report the assigned id on a new header,
   `CamelPulsarProducerMessageId`, named the same way as the producer headers this component already has
   (`CamelPulsarProducerMessageKey`, `CamelPulsarProducerMessageEventTime`, …).
   
   I deliberately did **not** reuse the existing consumer-side `MESSAGE_ID` constant: its value is the
   un-prefixed `message_id`, which `DefaultHeaderFilterStrategy` does not filter, and adding a producer
   meaning to it would spread that name rather than contain it.
   
   ## Compatibility
   
   A route that today reads the `MessageId` from the body has to read the header instead. That is the whole
   cost, and it is in the guide. If you would rather keep the body assignment and only add the header, say
   so — but then `.to("pulsar:a").to("pulsar:b")` keeps publishing a serialized `MessageId`, which seems
   worse than the break.
   
   ## Test
   
   `PulsarProducerMessageIdHeaderTest` sends through a mocked `PulsarClient` (Mockito `RETURNS_SELF` for the
   fluent `ProducerBuilder` and `TypedMessageBuilder`) and asserts the body survives and the header carries
   the id. Verified by reverting the fix: it then fails with *"the producer should leave the body alone …
   but was: Mock for MessageId"*.
   
   Module unit suite green (24 tests); full reactor `mvn clean install -DskipTests -DskipITs` green, which is
   what regenerated the catalog and the endpoint DSL for the new header.
   
   _Claude Code on behalf of oscerd_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

Posted by "oscerd (via GitHub)" <gi...@apache.org>.

oscerd commented on code in PR #26783:
URL: https://github.com/apache/camel/pull/26783#discussion_r4084317681


##########
components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/utils/message/PulsarMessageHeaders.java:
##########
@@ -60,4 +60,7 @@ public interface PulsarMessageHeaders {
     @Metadata(label = "producer", description = "Deliver the message after a given delayed time (millis).",
               javaType = "Long")
     String DELIVER_AFTER = "CamelPulsarProducerMessageDeliverAfter";
+    @Metadata(label = "producer", description = "The message id the broker assigned to the published message.",
+              javaType = "org.apache.pulsar.client.api.MessageId")
+    String MESSAGE_ID_OUT = "CamelPulsarProducerMessageId";

Review Comment:
   Good distinction, renamed to `PRODUCER_MESSAGE_ID`. The header value is unchanged (`CamelPulsarProducerMessageId`); only the Java constant moves, so the catalog picks up the new `constantName`. Regenerating now.
   
   _Claude Code on behalf of oscerd_



##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -2534,6 +2534,17 @@ was published as a second, undocumented spelling of the same setting. Routes tha
 `databaseOutServerName` should use `xstreamOutServerName` instead; it configures the same XStream outbound
 server and is unchanged.
 
+=== camel-pulsar - the producer no longer replaces the body with the message id

Review Comment:
   The 3.20 provenance is a good addition — verified and added: CAMEL-16030 ("camel-pulsar - Add async send to producer", fixed 3.20.0) is where the `thenAccept(r -> setBody(r))` came in.
   
   On the position: that one is deliberate rather than base drift. I have three open PRs that each add an entry to this guide, and appending all of them at the end guarantees a conflict for whichever merges second — it already cost #26733 a rebase today. Putting this one before an existing section keeps the hunks apart so git merges them cleanly. Happy to move it to the end if you would rather the file stayed strictly append-ordered; it is a one-line change either way.
   
   _Claude Code on behalf of oscerd_



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on PR #26783:
URL: https://github.com/apache/camel/pull/26783#issuecomment-5795079439

   :star2: Thank you for your contribution to the Apache Camel project! :star2:
     :robot: CI automation will test this PR automatically.
   
     :camel: Apache Camel Committers, please review the following items:
   
     * First-time contributors **require MANUAL approval** for the GitHub Actions to run
     * You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
     * You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
     * Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
     
     :warning: Be careful when sharing logs. Review their contents before sharing them publicly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on PR #26783:
URL: https://github.com/apache/camel/pull/26783#issuecomment-5798201250

   <!-- ci-tested-modules -->
   :test_tube: **CI tested the following changed modules:**
   
   - `catalog/camel-catalog`
   - `components/camel-pulsar`
   - `docs`
   - `dsl/camel-endpointdsl`
   
   ---
   
   <details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 of 693 tested, 24 compile-only — current: 9 all tested</summary>
   
   [Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **9 affected modules** (current approach: 9).
   
   **Skip-tests mode would test 9 modules** (4 direct + 8 downstream), **skip tests for 24** (generated code, meta-modules)
   
   <details><summary>Modules Scalpel would test (9)</summary>
   
   - `camel-jbang-mcp` ← downstream of org.apache.camel:camel-catalog
   - `camel-jbang-plugin-mcp` ← downstream of org.apache.camel:camel-jbang-core
   - `camel-jbang-plugin-route-parser` ← downstream of org.apache.camel:camel-route-parser
   - `camel-jbang-plugin-tui` ← downstream of org.apache.camel:camel-catalog
   - `camel-jbang-plugin-validate` ← downstream of org.apache.camel:camel-yaml-dsl-validator
   - `camel-launcher-container` ← downstream of org.apache.camel:camel-launcher
   - `camel-pulsar` ← components/camel-pulsar/src/generated/resources/META-INF/org/apache/camel/component/pulsar/pulsar.json, components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/PulsarProducer.java, components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/utils/message/PulsarMessageHeaders.java, components/camel-pulsar/src/test/java/org/apache/camel/component/pulsar/PulsarProducerMessageIdHeaderTest.java
   - `camel-yaml-dsl-validator` ← downstream of org.apache.camel:camel-catalog
   - `camel-yaml-dsl-validator-maven-plugin` ← downstream of org.apache.camel:camel-yaml-dsl-validator
   
   </details>
   
   <details><summary>Modules with tests skipped (24)</summary>
   
   - `apache-camel`
   - `camel-allcomponents`
   - `camel-catalog-console`
   - `camel-catalog-maven`
   - `camel-catalog-suggest`
   - `camel-componentdsl`
   - `camel-endpointdsl-support`
   - `camel-itest`
   - `camel-jbang-core`
   - `camel-jbang-it`
   - `camel-jbang-main`
   - `camel-jbang-plugin-edit`
   - `camel-jbang-plugin-generate`
   - `camel-jbang-plugin-kubernetes`
   - `camel-jbang-plugin-test`
   - `camel-kamelet-main`
   - `camel-launcher`
   - `camel-report-maven-plugin`
   - `camel-route-parser`
   - `camel-yaml-dsl`
   - `camel-yaml-dsl-deserializers`
   - `camel-yaml-dsl-maven-plugin`
   - `coverage`
   - `dummy-component`
   
   </details>
   
   > :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
   
   </details>
   
   <details><summary>All tested modules (36 modules, 5m 16s total)</summary>
   
   **Total reactor time:** 5m 16s
   
   | Module | Duration | Status |
   | --- | --- | --- |
   | Camel :: Launcher | 51.1s | SUCCESS |
   | Camel :: JBang :: MCP | 41.3s | SUCCESS |
   | Camel :: JBang :: Plugin :: TUI | 33.4s | SUCCESS |
   | Camel :: Component DSL | 26.6s | SUCCESS |
   | Camel :: Catalog :: Camel Catalog | 24.2s | SUCCESS |
   | Camel :: YAML DSL | 18.9s | SUCCESS |
   | Camel :: Docs | 15.6s | SUCCESS |
   | Camel :: JBang :: Plugin :: Kubernetes | 15.3s | SUCCESS |
   | Camel :: Kamelet Main | 12.3s | SUCCESS |
   | Camel :: YAML DSL :: Validator | 10.8s | SUCCESS |
   | Camel :: Catalog :: Camel Route Parser | 8.2s | SUCCESS |
   | Camel :: YAML DSL :: Deserializers | 8.1s | SUCCESS |
   | Camel :: JBang :: Plugin :: Testing | 7.4s | SUCCESS |
   | Camel :: Catalog :: Camel Report Maven Plugin | 6.7s | SUCCESS |
   | Camel :: All Components Sync point | 5.2s | SUCCESS |
   | Camel :: JBang :: Plugin :: Validate | 5.1s | SUCCESS |
   | Camel :: YAML DSL :: Validator Maven Plugin | 4.1s | SUCCESS |
   | Camel :: YAML DSL :: Maven Plugins | 3.1s | SUCCESS |
   | Camel :: Catalog :: Maven | 3.1s | SUCCESS |
   | Camel :: Catalog :: Suggest (deprecated) | 2.7s | SUCCESS |
   | Camel :: Coverage | 1.9s | SUCCESS |
   | Camel :: Assembly | 1.9s | SUCCESS |
   | Camel :: JBang :: Plugin :: Edit | 1.3s | SUCCESS |
   | Camel :: Catalog :: Dummy Component | 1.1s | SUCCESS |
   | Camel :: JBang :: Integration tests | 1.1s | SUCCESS |
   | Camel :: JBang :: Plugin :: Generate | 0.9s | SUCCESS |
   | Camel :: Catalog :: Console | 0.8s | SUCCESS |
   | Camel :: Endpoint DSL :: Support | 0.7s | SUCCESS |
   | Camel :: JBang :: Main | 0.7s | SUCCESS |
   | Camel :: Launcher :: Container | 0.7s | SUCCESS |
   | Camel :: JBang :: Plugin :: MCP | 0.5s | SUCCESS |
   | Camel :: JBang :: Plugin :: Route Parser | 0.5s | SUCCESS |
   | Camel :: Endpoint DSL | n/a |  |
   | Camel :: Integration Tests | n/a |  |
   | Camel :: JBang :: Core | n/a |  |
   | Camel :: Pulsar | n/a |  |
   
   **Top 20 slowest modules:**
   - `Camel :: Launcher` (51.1s)
   - `Camel :: JBang :: MCP` (41.3s)
   - `Camel :: JBang :: Plugin :: TUI` (33.4s)
   - `Camel :: Component DSL` (26.6s)
   - `Camel :: Catalog :: Camel Catalog` (24.2s)
   - `Camel :: YAML DSL` (18.9s)
   - `Camel :: Docs` (15.6s)
   - `Camel :: JBang :: Plugin :: Kubernetes` (15.3s)
   - `Camel :: Kamelet Main` (12.3s)
   - `Camel :: YAML DSL :: Validator` (10.8s)
   - `Camel :: Catalog :: Camel Route Parser` (8.2s)
   - `Camel :: YAML DSL :: Deserializers` (8.1s)
   - `Camel :: JBang :: Plugin :: Testing` (7.4s)
   - `Camel :: Catalog :: Camel Report Maven Plugin` (6.7s)
   - `Camel :: All Components Sync point` (5.2s)
   - `Camel :: JBang :: Plugin :: Validate` (5.1s)
   - `Camel :: YAML DSL :: Validator Maven Plugin` (4.1s)
   - `Camel :: YAML DSL :: Maven Plugins` (3.1s)
   - `Camel :: Catalog :: Maven` (3.1s)
   - `Camel :: Catalog :: Suggest (deprecated)` (2.7s)
   
   </details>
   
   ---
   :gear: [View full build and test results](https://github.com/apache/camel/actions/runs/35873430044)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.

davsclaus commented on code in PR #26783:
URL: https://github.com/apache/camel/pull/26783#discussion_r4083550912


##########
components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/utils/message/PulsarMessageHeaders.java:
##########
@@ -60,4 +60,7 @@ public interface PulsarMessageHeaders {
     @Metadata(label = "producer", description = "Deliver the message after a given delayed time (millis).",
               javaType = "Long")
     String DELIVER_AFTER = "CamelPulsarProducerMessageDeliverAfter";
+    @Metadata(label = "producer", description = "The message id the broker assigned to the published message.",
+              javaType = "org.apache.pulsar.client.api.MessageId")
+    String MESSAGE_ID_OUT = "CamelPulsarProducerMessageId";

Review Comment:
   Nit: in this file the other `*_OUT` constants (`KEY_OUT`, `EVENT_TIME_OUT`, ...) are values the producer *reads*, while this one is a result the producer *writes*. A name like `PRODUCER_MESSAGE_ID` would make that clearer. Not blocking.



##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc:
##########
@@ -2534,6 +2534,17 @@ was published as a second, undocumented spelling of the same setting. Routes tha
 `databaseOutServerName` should use `xstreamOutServerName` instead; it configures the same XStream outbound
 server and is unchanged.
 
+=== camel-pulsar - the producer no longer replaces the body with the message id

Review Comment:
   Nit: this entry lands in the middle of the file (the branch was based on an older main). Optionally, move it to the end and mention that the old behaviour dates from 3.20 (CAMEL-16030).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

Posted by "apupier (via GitHub)" <gi...@apache.org>.

apupier commented on PR #26783:
URL: https://github.com/apache/camel/pull/26783#issuecomment-5796229534

   ```
   [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.16.0:testCompile (default-testCompile) on project camel-jbang-core: Compilation failure
   Error: [ERROR] /home/runner/work/camel/camel/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/CatalogDocExamplesTest.java:[100,49] cannot find symbol
   ```
   problem was on main branch and is fixed by https://github.com/apache/camel/commit/fa8ab2a7a4b28e3d3f7bc8fffada15bf12003921, please rebase


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

Posted by "gnodet (via GitHub)" <gi...@apache.org>.

gnodet commented on PR #26783:
URL: https://github.com/apache/camel/pull/26783#issuecomment-5829691206

   <!-- camel-backport-bot -->
   
   ### 🔄 Backport Bot
   
   **Conflicts (🤖 agent dispatched to resolve):**
   - ⚠️ `camel-4.22.x` — Cherry-pick conflict in: docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc
   - ⚠️ `camel-4.18.x` — Cherry-pick conflict in: docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc
   
   ℹ️ If you push additional commits after `/port`, use `/port` again to update the port PRs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

Posted by "oscerd (via GitHub)" <gi...@apache.org>.

oscerd merged PR #26783:
URL: https://github.com/apache/camel/pull/26783


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] CAMEL-24935: camel-pulsar - report the message id as a header, not as the body [camel]

Posted by "oscerd (via GitHub)" <gi...@apache.org>.

oscerd commented on PR #26783:
URL: https://github.com/apache/camel/pull/26783#issuecomment-5830077017

Post-merge housekeeping done: milestone 4.23.0, `bug` label, assigned, CAMEL-24935 Resolved/Fixed.

**One thing I would like a committer to confirm before the bot ports this.** The PR carries `port/camel-4.22.x` and `port/camel-4.18.x`, but this is a deliberate behaviour break, not a pure bug fix: a route that reads the `MessageId` from the body after `to("pulsar:...")` stops working and has to read `CamelPulsarProducerMessageId` instead. On `main` that is fine — it is in the 4.23 upgrade guide. On a patch release of an LTS line it means someone upgrading 4.22.1 → 4.22.2 can have a working route stop working.

Two ways to square it:

1. **Drop the port labels** and let it ship in 4.23.0 only, which is the usual home for a break of this kind. The bug it fixes — `.to("pulsar:a").to("pulsar:b")` publishing a Java-serialized `MessageId` to the second topic — has been there since 3.20, so it is not a regression that needs urgent patching.
2. **Keep the ports** and add matching entries to `camel-4x-upgrade-guide-4_22.adoc` and `camel-4x-upgrade-guide-4_18.adoc` on `main`, per the convention that the per-line guides live there. I am happy to open that doc PR.

I have left the labels as they are rather than deciding this myself. The fixVersions on CAMEL-24935 currently say 4.18.5/4.22.2/4.23.0, which matches option 2; if we take option 1 they should come back to 4.23.0 alone.

_Claude Code on behalf of oscerd_

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org