You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "oscerd (via GitHub)" <gi...@apache.org> on 2026/10/01 13:44:36 UTC
[PR] CAMEL-25227: camel-azure-eventhubs - partition headers and default header filter [camel]
oscerd opened a new pull request, #27213:
URL: https://github.com/apache/camel/pull/27213
_Claude Code on behalf of oscerd_
Jira: [CAMEL-25227](https://issues.apache.org/jira/browse/CAMEL-25227), a follow-up to CAMEL-24158.
## Problem
A route that consumes from one Event Hub and produces to another, `from("azure-eventhubs:...").to("azure-eventhubs:...")`, trips over the `CamelAzureEventHubsPartitionKey` and `CamelAzureEventHubsPartitionId` headers that the consumer sets. The producer prefers both headers over its endpoint options and rejects a partition key together with a partition id, so:
- every event with a partition key failed with `Both partitionKey and partitionId are set`, whatever the producer configuration;
- with `partitionKey` configured on the producer, every event failed;
- with `partitionId` configured, the configured value was replaced by the partition the event was received from, which may not exist on the target hub.
CAMEL-24158 only stopped setting a null `PartitionKey` header, which never triggered the error.
The same change made `EventHubsComponent` extend `HeaderFilterStrategyComponent`, but no default strategy is ever installed. `getHeaderFilterStrategy()` is therefore null and the producer still copies every header onto the `EventData` application properties, unlike what the 4.14/4.18/4.22 upgrade guides say. The consumer's own `CamelAzureEventHubsEnqueuedTime` (a `java.time.Instant`) then cannot be encoded by the SDK, so a default bridging route failed for every event.
## Changes
- `EventHubsConsumer` records the partition id and key of the received event as (internal) exchange properties.
- `EventHubsConfigurationOptionsProxy` does not take a partition header that still holds the received value as a partition chosen by the route. `EventHubsProducerOperations` falls back to the received partition key only when neither the route nor the endpoint chose a partition, and never reuses the received partition id.
- The resulting order is: a header set by the route, then the `partitionKey`/`partitionId` option, then the received partition key, then Event Hubs' own choice.
- Exchanges that do not come from an azure-eventhubs consumer behave exactly as before: a header still overrides the endpoint option, and a key and an id set together still fail.
- `EventHubsComponent` installs a `DefaultHeaderFilterStrategy` in `doInit` when none is configured, following the same pattern as `SpringRabbitMQComponent`.
- Docs: the component page describes the partition selection order, and the 4.23 upgrade guide has an entry for both behaviour changes.
## Tests
`EventHubsConsumerToProducerTest` builds the exchange through the consumer's own mapping and sends it through the real producer endpoint with a mocked `EventHubProducerAsyncClient`. It covers:
- keyed and unkeyed events, each with no option, `partitionKey` and `partitionId`;
- a partition chosen by the route after consuming;
- unchanged behaviour for exchanges not coming from a consumer;
- `Camel*` headers not being sent as event properties.
7 of its 9 tests fail on the previous code. The other 2 cover the unchanged behaviour and pass on both. The module's unit tests pass (29/29), and the full reactor `mvn clean install -DskipTests -DskipITs` is green.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25227: camel-azure-eventhubs - partition headers and default header filter [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #27213:
URL: https://github.com/apache/camel/pull/27213
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25227: camel-azure-eventhubs - partition headers and default header filter [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27213:
URL: https://github.com/apache/camel/pull/27213#issuecomment-5932746220
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25227: camel-azure-eventhubs - partition headers and default header filter [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27213:
URL: https://github.com/apache/camel/pull/27213#issuecomment-5933287671
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `catalog/camel-catalog`
- `components/camel-azure/camel-azure-eventhubs`
- `docs`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 of 697 tested, 25 compile-only — current: 9 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **9 affected modules** (current approach: 9).
**Skip-tests mode would test 9 modules** (3 direct + 8 downstream), **skip tests for 25** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (9)</summary>
- `camel-azure-eventhubs` ← components/camel-azure/camel-azure-eventhubs/src/main/docs/azure-eventhubs-component.adoc, components/camel-azure/camel-azure-eventhubs/src/main/java/org/apache/camel/component/azure/eventhubs/EventHubsComponent.java, components/camel-azure/camel-azure-eventhubs/src/main/java/org/apache/camel/component/azure/eventhubs/EventHubsConfigurationOptionsProxy.java, components/camel-azure/camel-azure-eventhubs/src/main/java/org/apache/camel/component/azure/eventhubs/EventHubsConstants.java, components/camel-azure/camel-azure-eventhubs/src/main/java/org/apache/camel/component/azure/eventhubs/EventHubsConsumer.java, components/camel-azure/camel-azure-eventhubs/src/main/java/org/apache/camel/component/azure/eventhubs/operations/EventHubsProducerOperations.java, components/camel-azure/camel-azure-eventhubs/src/test/java/org/apache/camel/component/azure/eventhubs/EventHubsConsumerToProducerTest.java
- `camel-jbang-mcp` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-mcp` ← downstream of org.apache.camel:camel-jbang-core
- `camel-jbang-plugin-route-parser` ← downstream of org.apache.camel:camel-route-parser
- `camel-jbang-plugin-tui` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-validate` ← downstream of org.apache.camel:camel-yaml-dsl-validator
- `camel-launcher-container` ← downstream of org.apache.camel:camel-launcher
- `camel-yaml-dsl-validator` ← downstream of org.apache.camel:camel-catalog
- `camel-yaml-dsl-validator-maven-plugin` ← downstream of org.apache.camel:camel-yaml-dsl-validator
</details>
<details><summary>Modules with tests skipped (25)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
<details><summary>All tested modules (36 modules, 5m 31s total)</summary>
**Total reactor time:** 5m 31s
| Module | Duration | Status |
| --- | --- | --- |
| Camel :: Launcher | 49.7s | SUCCESS |
| Camel :: JBang :: MCP | 38.8s | SUCCESS |
| Camel :: JBang :: Plugin :: TUI | 34.4s | SUCCESS |
| Camel :: Component DSL | 32.2s | SUCCESS |
| Camel :: Catalog :: Camel Catalog | 21.7s | SUCCESS |
| Camel :: YAML DSL | 19.6s | SUCCESS |
| Camel :: JBang :: Plugin :: Kubernetes | 15.7s | SUCCESS |
| Camel :: YAML DSL :: Validator | 15.2s | SUCCESS |
| Camel :: Azure :: Event Hubs | 13.2s | SUCCESS |
| Camel :: Docs | 11.6s | SUCCESS |
| Camel :: Kamelet Main | 9.4s | SUCCESS |
| Camel :: YAML DSL :: Deserializers | 9.2s | SUCCESS |
| Camel :: Catalog :: Camel Route Parser | 8.0s | SUCCESS |
| Camel :: Catalog :: Camel Report Maven Plugin | 7.3s | SUCCESS |
| Camel :: JBang :: Plugin :: Testing | 7.1s | SUCCESS |
| Camel :: YAML DSL :: Validator Maven Plugin | 5.2s | SUCCESS |
| Camel :: All Components Sync point | 5.2s | SUCCESS |
| Camel :: JBang :: Plugin :: Validate | 4.9s | SUCCESS |
| Camel :: YAML DSL :: Maven Plugins | 3.5s | SUCCESS |
| Camel :: Catalog :: Maven | 3.1s | SUCCESS |
| Camel :: Catalog :: Suggest (deprecated) | 2.7s | SUCCESS |
| Camel :: Assembly | 1.9s | SUCCESS |
| Camel :: JBang :: Plugin :: Edit | 1.8s | SUCCESS |
| Camel :: Coverage | 1.7s | SUCCESS |
| Camel :: JBang :: Main | 1.2s | SUCCESS |
| Camel :: Catalog :: Console | 1.1s | SUCCESS |
| Camel :: JBang :: Integration tests | 1.1s | SUCCESS |
| Camel :: Catalog :: Dummy Component | 1.1s | SUCCESS |
| Camel :: JBang :: Plugin :: Generate | 0.9s | SUCCESS |
| Camel :: Endpoint DSL :: Support | 0.8s | SUCCESS |
| Camel :: JBang :: Plugin :: MCP | 0.6s | SUCCESS |
| Camel :: Launcher :: Container | 0.6s | SUCCESS |
| Camel :: JBang :: Plugin :: Route Parser | 0.5s | SUCCESS |
| Camel :: Endpoint DSL | n/a | |
| Camel :: Integration Tests | n/a | |
| Camel :: JBang :: Core | n/a | |
**Top 20 slowest modules:**
- `Camel :: Launcher` (49.7s)
- `Camel :: JBang :: MCP` (38.8s)
- `Camel :: JBang :: Plugin :: TUI` (34.4s)
- `Camel :: Component DSL` (32.2s)
- `Camel :: Catalog :: Camel Catalog` (21.7s)
- `Camel :: YAML DSL` (19.6s)
- `Camel :: JBang :: Plugin :: Kubernetes` (15.7s)
- `Camel :: YAML DSL :: Validator` (15.2s)
- `Camel :: Azure :: Event Hubs` (13.2s)
- `Camel :: Docs` (11.6s)
- `Camel :: Kamelet Main` (9.4s)
- `Camel :: YAML DSL :: Deserializers` (9.2s)
- `Camel :: Catalog :: Camel Route Parser` (8.0s)
- `Camel :: Catalog :: Camel Report Maven Plugin` (7.3s)
- `Camel :: JBang :: Plugin :: Testing` (7.1s)
- `Camel :: YAML DSL :: Validator Maven Plugin` (5.2s)
- `Camel :: All Components Sync point` (5.2s)
- `Camel :: JBang :: Plugin :: Validate` (4.9s)
- `Camel :: YAML DSL :: Maven Plugins` (3.5s)
- `Camel :: Catalog :: Maven` (3.1s)
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/36870944422)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25227: camel-azure-eventhubs - partition headers and default header filter [camel]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on PR #27213:
URL: https://github.com/apache/camel/pull/27213#issuecomment-5951686043
_Claude Code on behalf of oscerd_
Thanks for the review. I rebased the branch on current main, because the 4.23 upgrade guide conflicted with #27123, and pushed a follow-up commit for these points.
1. **Keeping the received partition.** Agreed, this needed a documented way. `EventHubsConstants.RECEIVED_PARTITION_ID` and `RECEIVED_PARTITION_KEY` are now public. The component page and the 4.23 upgrade guide explain that removing the property before the producer makes the unchanged header count as chosen by the route again. For example, `.removeProperty(EventHubsConstants.RECEIVED_PARTITION_ID)` keeps the partition when mirroring between Event Hubs that have the same partitions. `receivedPartitionIsKeptWhenTheRouteRemovesTheReceivedProperty` covers both the id and the key. I chose this over a new endpoint option so that the catalog and the DSLs stay unchanged.
2. **`org.apache.camel.*`.** I kept the plain `DefaultHeaderFilterStrategy`, so the component uses the project-wide default from CAMEL-23543 (#23652). That default leaves `org.apache.camel.` out on purpose. As you noted on #23652, it is a Camel 1.x convention and has been obsolete to filter since Camel 2. If we want it, it belongs in `DefaultHeaderFilterStrategy` for every component, not in eventhubs alone.
3. **Backport.** Yes. Once this is merged, I'll backport the whole change to camel-4.22.x and camel-4.18.x, because the bridging failure exists on those lines too. On 4.18.x, `DefaultHeaderFilterStrategy` doesn't filter `Camel*` by default (CAMEL-23543 is 4.21+), so that backport will configure the outbound filter explicitly and run the test on that branch. I'll then update the 4.14, 4.18 and 4.22 upgrade-guide entries on main so they stop claiming that the filter shipped in 4.14.9, 4.18.4 and 4.22.0, and name the releases that actually apply it.
4. **Reflection.** `createAzureEventHubExchange(EventContext)` is now package-private, and the test calls it directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org