You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "allthingssecurity (via GitHub)" <gi...@apache.org> on 2026/10/19 00:11:34 UTC
[PR] CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl [camel]
allthingssecurity opened a new pull request, #28074:
URL: https://github.com/apache/camel/pull/28074
# Description
[CAMEL-25759](https://issues.apache.org/jira/browse/CAMEL-25759)
The EventBridge consumer adds its SQS queue as a target of the rule when it starts, with a new random target id each time (`"camel-eb-" + UUID...`), and removes that target when it stops. With an auto-created queue each consumer has its own queue, so that is fine. With a queue given by the user (`queueUrl`), the rule gets one target per consumer for the same queue:
1. two consumers that share the queue (two instances of the application, or two routes) add two targets for it. EventBridge sends a matching event to every target of the rule, so every event is put on the queue twice and routed twice;
2. a consumer that is not stopped cleanly (JVM killed, container OOM) leaves its target on the rule, and the next start adds a second one. From then on every event is put on the queue twice, with nothing in the log. Each such restart adds one more copy, until the rule reaches the limit of 5 targets and the consumer fails to start.
This change gives the target of a user-provided queue an id derived from the queue ARN (`camel-eb-` + 32 hex characters of its SHA-256). PutTargets replaces a target with the same id, so every start and every consumer of the queue updates the same target. That target now stays on the rule when the consumer stops: other consumers may still use the queue, and the queue keeps collecting the events of the rule while no consumer runs (before, the events published during a restart never reached the queue).
An auto-created queue is not changed: random target id, target removed on stop, queue deleted with `deleteQueueOnShutdown=true`.
For review: keeping the target of a user-provided queue on stop reverses the removal that CAMEL-23207 added (the component page and `testUserProvidedQueueNotDeletedOnShutdown` said so). The two changes go together: with one shared id but removal on stop, the first consumer that stops would remove the target the others still use. If you prefer to keep the removal for a queue that only one consumer uses, it could go behind an option instead. Not changed: a target that the user added to the rule for the same queue under an id of their own (for example from infrastructure as code) is still a second target, as before.
The behaviour change for user-provided queues (the target is kept on stop) is in the 4.24 upgrade guide, with a note that a target an earlier version left behind (`camel-eb-` + 8 characters) has to be removed by hand. The component page says the same in "How It Works".
Tests:
- New `EventbridgeConsumerUserQueueTargetTest` (4 tests). The mocked EventBridge client keeps the rule as a map of target id to ARN, with the semantics of PutTargets (replace by id) and RemoveTargets. Without the fix `twoConsumersSharingAQueueAddOneTarget` and `restartAfterACrashAddsNoSecondTarget` fail with `expected: <1> but was: <2>` (two `camel-eb-xxxxxxxx` targets for the queue), and `userQueueKeepsCollectingEventsWhileTheConsumerIsStopped` fails because the target was removed. `autoCreatedQueueTargetIsStillRemovedOnStop` passes with and without the fix; it shows that the auto-created path is unchanged.
- `EventbridgeConsumerTest.testUserProvidedQueueNotDeletedOnShutdown` checked that the target of the user queue is removed on stop; it now checks that the target is kept and the queue is not deleted.
- camel-aws2-eventbridge unit tests: 30 run, 0 failures (formatter and import sort run in the same build).
- Not run: the LocalStack integration tests (they need Docker). The EventBridge semantics the tests rely on (every target of a rule receives the event; PutTargets with an existing id updates that target) come from the AWS API reference, not from a run against AWS.
How it was found: a small TLA+ model of the targets of the rule (consumers sharing a user queue that start, stop or crash, and events put on the queue once per target) found a duplicate in 4 steps for two consumers and for one consumer that crashed and started again. With the fix, no duplicate and no lost event for 2 and 3 consumers. The tests above reproduce it on the real consumer.
# Target
- [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch)
# Tracking
- [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it).
# Apache Camel coding standards and style
- [x] I checked that each commit in the pull request has a meaningful subject line and body.
- [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes.
(I built and tested camel-aws2-eventbridge, including the formatter and import-sort plugins, and updated the catalog copy of the component page by hand. I did not run the full root build.)
# AI-assisted contributions
- [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used.
This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer.
_Claude Code on behalf of allthingssecurity_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #28074:
URL: https://github.com/apache/camel/pull/28074
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on code in PR #28074:
URL: https://github.com/apache/camel/pull/28074#discussion_r4310711749
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc:
##########
@@ -1361,6 +1361,16 @@ Previously it returned only the first chunk of the event log (at most 500 events
any sign. A wide `after`/`before` window now returns every event in one list: that is one request per 500 events plus
one that finds the end of the log, and more memory. Narrow the window when the event log is large.
+=== camel-aws2-eventbridge - the consumer keeps one target for a queue given with queueUrl
+
+A consumer that uses its own queue (`queueUrl`) added a target with a new random id to the EventBridge rule at every
+start, and removed it on stop. Two consumers that share the queue, or a restart after a consumer was not stopped
+cleanly, left two targets for the queue, and EventBridge put every event on the queue twice. The target now has an id
+derived from the queue ARN, and stays on the rule when the consumer stops, so the queue also collects the events of the
+rule while no consumer runs. Remove the target from the rule yourself when you no longer use the queue, and remove
+targets with ids `camel-eb-` plus 8 characters that an earlier version left for the queue. An auto-created queue is
+not affected.
+
Review Comment:
Optional: during a rolling upgrade, an instance on an earlier version and one on this version add two different targets (`camel-eb-` + 8 random characters, and the hashed id), so events are put on the queue twice until the old instance stops and removes its target. It may be worth one sentence here, for example:
```suggestion
not affected. During a rolling upgrade from an earlier version, events can be put on the queue twice until the last
instance on the earlier version stops.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28074:
URL: https://github.com/apache/camel/pull/28074#issuecomment-6260814686
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `catalog/camel-catalog`
- `components/camel-aws/camel-aws2-eventbridge`
- `docs`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 8 of 706 tested, 25 compile-only — current: 8 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **8 affected modules** (current approach: 8).
**Skip-tests mode would test 8 modules** (3 direct + 7 downstream), **skip tests for 25** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (8)</summary>
- `camel-aws2-eventbridge` ← components/camel-aws/camel-aws2-eventbridge/src/main/docs/aws2-eventbridge-component.adoc, components/camel-aws/camel-aws2-eventbridge/src/main/java/org/apache/camel/component/aws2/eventbridge/EventbridgeConsumer.java, components/camel-aws/camel-aws2-eventbridge/src/test/java/org/apache/camel/component/aws2/eventbridge/EventbridgeConsumerTest.java, components/camel-aws/camel-aws2-eventbridge/src/test/java/org/apache/camel/component/aws2/eventbridge/EventbridgeConsumerUserQueueTargetTest.java
- `camel-jbang-mcp` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-mcp` ← downstream of org.apache.camel:camel-jbang-core
- `camel-jbang-plugin-tui` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-validate` ← downstream of org.apache.camel:camel-yaml-dsl-validator
- `camel-launcher-container` ← downstream of org.apache.camel:camel-launcher
- `camel-yaml-dsl-validator` ← downstream of org.apache.camel:camel-catalog
- `camel-yaml-dsl-validator-maven-plugin` ← downstream of org.apache.camel:camel-yaml-dsl-validator
</details>
<details><summary>Modules with tests skipped (25)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
<details><summary>All tested modules (35 modules, 22.6s total)</summary>
**Total reactor time:** 22.6s
| Module | Duration | Status |
| --- | --- | --- |
| Camel :: AWS2 Eventbridge | 22.6s | FAILURE |
| Camel :: All Components Sync point | n/a | |
| Camel :: Assembly | n/a | |
| Camel :: Catalog :: Camel Catalog | n/a | |
| Camel :: Catalog :: Camel Report Maven Plugin | n/a | |
| Camel :: Catalog :: Camel Route Parser | n/a | |
| Camel :: Catalog :: Console | n/a | |
| Camel :: Catalog :: Dummy Component | n/a | |
| Camel :: Catalog :: Maven | n/a | |
| Camel :: Catalog :: Suggest (deprecated) | n/a | |
| Camel :: Component DSL | n/a | |
| Camel :: Coverage | n/a | |
| Camel :: Docs | n/a | |
| Camel :: Endpoint DSL | n/a | |
| Camel :: Endpoint DSL :: Support | n/a | |
| Camel :: Integration Tests | n/a | |
| Camel :: JBang :: Core | n/a | |
| Camel :: JBang :: Integration tests | n/a | |
| Camel :: JBang :: MCP | n/a | |
| Camel :: JBang :: Main | n/a | |
| Camel :: JBang :: Plugin :: Edit | n/a | |
| Camel :: JBang :: Plugin :: Generate | n/a | |
| Camel :: JBang :: Plugin :: Kubernetes | n/a | |
| Camel :: JBang :: Plugin :: MCP | n/a | |
| Camel :: JBang :: Plugin :: TUI | n/a | |
| Camel :: JBang :: Plugin :: Testing | n/a | |
| Camel :: JBang :: Plugin :: Validate | n/a | |
| Camel :: Kamelet Main | n/a | |
| Camel :: Launcher | n/a | |
| Camel :: Launcher :: Container | n/a | |
| Camel :: YAML DSL | n/a | |
| Camel :: YAML DSL :: Deserializers | n/a | |
| Camel :: YAML DSL :: Maven Plugins | n/a | |
| Camel :: YAML DSL :: Validator | n/a | |
| Camel :: YAML DSL :: Validator Maven Plugin | n/a | |
**Top 20 slowest modules:**
- `Camel :: AWS2 Eventbridge` (22.6s)
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/39179661932)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #28074:
URL: https://github.com/apache/camel/pull/28074#issuecomment-6262178013
Closing and reopening to re-run CI against main: the `EventbridgePutEventsIT` failure came from main (floci 2.3.0 refuses a detail that is not JSON, as AWS does) and is fixed there by CAMEL-25780 (d0a4758ab404). Nothing to change in this PR.
_Claude Code on behalf of Claus Ibsen_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl [camel]
Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on code in PR #28074:
URL: https://github.com/apache/camel/pull/28074#discussion_r4310844026
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc:
##########
@@ -1361,6 +1361,16 @@ Previously it returned only the first chunk of the event log (at most 500 events
any sign. A wide `after`/`before` window now returns every event in one list: that is one request per 500 events plus
one that finds the end of the log, and more memory. Narrow the window when the event log is large.
+=== camel-aws2-eventbridge - the consumer keeps one target for a queue given with queueUrl
+
+A consumer that uses its own queue (`queueUrl`) added a target with a new random id to the EventBridge rule at every
+start, and removed it on stop. Two consumers that share the queue, or a restart after a consumer was not stopped
+cleanly, left two targets for the queue, and EventBridge put every event on the queue twice. The target now has an id
+derived from the queue ARN, and stays on the rule when the consumer stops, so the queue also collects the events of the
+rule while no consumer runs. Remove the target from the rule yourself when you no longer use the queue, and remove
+targets with ids `camel-eb-` plus 8 characters that an earlier version left for the queue. An auto-created queue is
+not affected.
+
Review Comment:
Good point, thanks. Added as suggested in 2400956ed200, and the PR description now mentions it.
_Claude Code on behalf of allthingssecurity_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28074:
URL: https://github.com/apache/camel/pull/28074#issuecomment-6260628957
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus closed pull request #28074: CAMEL-25759: camel-aws2-eventbridge - keep one target on the rule for a queue given with queueUrl
URL: https://github.com/apache/camel/pull/28074
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org