You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "allthingssecurity (via GitHub)" <gi...@apache.org> on 2026/10/01 02:01:32 UTC
[PR] CAMEL-25210: camel-cassandraql - keep completed exchanges for recovery in the aggregation repository [camel]
allthingssecurity opened a new pull request, #27159:
URL: https://github.com/apache/camel/pull/27159
# Description
[CAMEL-25210](https://issues.apache.org/jira/browse/CAMEL-25210)
`CassandraAggregationRepository` (and `NamedCassandraAggregationRepository`) implement `RecoverableAggregationRepository`, and recovery is on by default. But the table only holds the aggregations in progress, so the recovery methods work on the wrong rows. This is the defect CAMEL-24622 fixed for Infinispan and CAMEL-25153 for Caffeine and Ehcache:
- `remove` deletes the row of the correlation key, so nothing is kept for recovery.
- `scan` returns the exchange ids of the aggregations in progress, and `recover` loads such an aggregation.
- `confirm(exchangeId)` deletes the aggregation in progress that has this exchange id.
So the recover task of the Aggregate EIP sends an aggregation that is still open (the first run is one second after start, then every `recoveryInterval`) with `CamelRedelivered=true`. When that exchange is processed, `confirm` deletes the open aggregation, and the messages that arrive later start a new group. A completed exchange whose processing fails is never recovered.
This change uses the same design as for Infinispan, Caffeine and Ehcache. A completed exchange is kept in the same table under the aggregation key `camel-recovery:<exchange id>` until it is confirmed, so the table does not change. `remove` writes the exchange it is given (see CAMEL-24946) before it deletes the aggregation. `scan` reports the exchange ids of those keys (none when recovery is disabled), `recover` reads them, `confirm` deletes them, and `getKeys` reports only the aggregations in progress. The `DELETE ... IF EXCHANGE_ID = ?` statement is no longer used.
Tests:
- New `CassandraAggregationRepositoryRecoveryTest`, a unit test that runs the repository against an in-memory table behind a mocked `CqlSession`. An open group must not be sent by the recover task, and a completed group whose processing fails once must be recovered with all its messages and confirmed.
- Without the change the test fails: the open group is sent twice, and the failed completed group is not recovered. With it, the camel-cassandraql unit tests pass (8 tests).
- `CassandraAggregationRepositoryIT` and `NamedCassandraAggregationRepositoryIT` encoded the old key space (`testConfirmExist`, `testScan`, `testRecover`), and are reworked. They compile, but I could not run them: they need Cassandra in Docker, which was not available here.
The upgrade guide gets a note: `scan` no longer returns open groups, and the table now also holds completed exchanges until they are confirmed.
# Target
- [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch)
# Tracking
- [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it).
# Apache Camel coding standards and style
- [x] I checked that each commit in the pull request has a meaningful subject line and body.
- [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes.
(I built and tested the affected module, including the formatter and import-sort plugins. I did not run the full root build.)
# AI-assisted contributions
- [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used.
This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer.
_Claude Code on behalf of allthingssecurity_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25210: camel-cassandraql - keep completed exchanges for recovery in the aggregation repository [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27159:
URL: https://github.com/apache/camel/pull/27159#issuecomment-5925858083
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `components/camel-cassandraql`
- `docs`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 of 697 tested, 26 compile-only — current: 9 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **9 affected modules** (current approach: 9).
**Skip-tests mode would test 9 modules** (2 direct + 8 downstream), **skip tests for 26** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (9)</summary>
- `camel-cassandraql` ← components/camel-cassandraql/src/main/java/org/apache/camel/processor/aggregate/cassandra/CassandraAggregationRepository.java, components/camel-cassandraql/src/test/java/org/apache/camel/processor/aggregate/cassandra/CassandraAggregationRepositoryIT.java, components/camel-cassandraql/src/test/java/org/apache/camel/processor/aggregate/cassandra/CassandraAggregationRepositoryRecoveryTest.java, components/camel-cassandraql/src/test/java/org/apache/camel/processor/aggregate/cassandra/NamedCassandraAggregationRepositoryIT.java
- `camel-jbang-mcp` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-mcp` ← downstream of org.apache.camel:camel-jbang-core
- `camel-jbang-plugin-route-parser` ← downstream of org.apache.camel:camel-route-parser
- `camel-jbang-plugin-tui` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-validate` ← downstream of org.apache.camel:camel-yaml-dsl-validator
- `camel-launcher-container` ← downstream of org.apache.camel:camel-launcher
- `camel-yaml-dsl-validator` ← downstream of org.apache.camel:camel-catalog
- `camel-yaml-dsl-validator-maven-plugin` ← downstream of org.apache.camel:camel-yaml-dsl-validator
</details>
<details><summary>Modules with tests skipped (26)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
<details><summary>All tested modules (36 modules, 5m 32s total)</summary>
**Total reactor time:** 5m 32s
| Module | Duration | Status |
| --- | --- | --- |
| Camel :: Launcher | 53.5s | SUCCESS |
| Camel :: JBang :: MCP | 42.8s | SUCCESS |
| Camel :: Component DSL | 37.0s | SUCCESS |
| Camel :: JBang :: Plugin :: TUI | 35.7s | SUCCESS |
| Camel :: Catalog :: Camel Catalog | 22.8s | SUCCESS |
| Camel :: YAML DSL | 18.5s | SUCCESS |
| Camel :: JBang :: Plugin :: Kubernetes | 16.9s | SUCCESS |
| Camel :: Docs | 14.2s | SUCCESS |
| Camel :: Kamelet Main | 12.3s | SUCCESS |
| Camel :: YAML DSL :: Validator | 10.8s | SUCCESS |
| Camel :: YAML DSL :: Deserializers | 8.3s | SUCCESS |
| Camel :: Catalog :: Camel Route Parser | 7.8s | SUCCESS |
| Camel :: JBang :: Plugin :: Testing | 7.7s | SUCCESS |
| Camel :: Catalog :: Camel Report Maven Plugin | 6.8s | SUCCESS |
| Camel :: All Components Sync point | 5.2s | SUCCESS |
| Camel :: YAML DSL :: Validator Maven Plugin | 5.0s | SUCCESS |
| Camel :: JBang :: Plugin :: Validate | 4.9s | SUCCESS |
| Camel :: YAML DSL :: Maven Plugins | 3.3s | SUCCESS |
| Camel :: Catalog :: Maven | 3.0s | SUCCESS |
| Camel :: Catalog :: Suggest (deprecated) | 2.0s | SUCCESS |
| Camel :: Assembly | 1.8s | SUCCESS |
| Camel :: Coverage | 1.7s | SUCCESS |
| Camel :: JBang :: Plugin :: Edit | 1.6s | SUCCESS |
| Camel :: Catalog :: Dummy Component | 1.5s | SUCCESS |
| Camel :: JBang :: Plugin :: Generate | 1.1s | SUCCESS |
| Camel :: Endpoint DSL :: Support | 1.1s | SUCCESS |
| Camel :: JBang :: Integration tests | 1.1s | SUCCESS |
| Camel :: JBang :: Main | 1.0s | SUCCESS |
| Camel :: Catalog :: Console | 0.9s | SUCCESS |
| Camel :: Launcher :: Container | 0.7s | SUCCESS |
| Camel :: JBang :: Plugin :: Route Parser | 0.7s | SUCCESS |
| Camel :: JBang :: Plugin :: MCP | 0.6s | SUCCESS |
| Camel :: Cassandra CQL | n/a | |
| Camel :: Endpoint DSL | n/a | |
| Camel :: Integration Tests | n/a | |
| Camel :: JBang :: Core | n/a | |
**Top 20 slowest modules:**
- `Camel :: Launcher` (53.5s)
- `Camel :: JBang :: MCP` (42.8s)
- `Camel :: Component DSL` (37.0s)
- `Camel :: JBang :: Plugin :: TUI` (35.7s)
- `Camel :: Catalog :: Camel Catalog` (22.8s)
- `Camel :: YAML DSL` (18.5s)
- `Camel :: JBang :: Plugin :: Kubernetes` (16.9s)
- `Camel :: Docs` (14.2s)
- `Camel :: Kamelet Main` (12.3s)
- `Camel :: YAML DSL :: Validator` (10.8s)
- `Camel :: YAML DSL :: Deserializers` (8.3s)
- `Camel :: Catalog :: Camel Route Parser` (7.8s)
- `Camel :: JBang :: Plugin :: Testing` (7.7s)
- `Camel :: Catalog :: Camel Report Maven Plugin` (6.8s)
- `Camel :: All Components Sync point` (5.2s)
- `Camel :: YAML DSL :: Validator Maven Plugin` (5.0s)
- `Camel :: JBang :: Plugin :: Validate` (4.9s)
- `Camel :: YAML DSL :: Maven Plugins` (3.3s)
- `Camel :: Catalog :: Maven` (3.0s)
- `Camel :: Catalog :: Suggest (deprecated)` (2.0s)
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/36803869073)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25210: camel-cassandraql - keep completed exchanges for recovery in the aggregation repository [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #27159:
URL: https://github.com/apache/camel/pull/27159
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25210: camel-cassandraql - keep completed exchanges for recovery in the aggregation repository [camel]
Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on PR #27159:
URL: https://github.com/apache/camel/pull/27159#issuecomment-5927053862
Thanks for the review. Docker is not available on my machine, so I ran the ITs against a local Apache Cassandra 5.0.9 instead (the binary tarball from downloads.apache.org, the same version as the test-infra image `cassandra:5.0.9`), started with a 1 GB heap on `localhost:9042`, with the module's `initScript.cql` schema.
For these runs only (nothing of this is committed):
- `BaseCassandra` used the test-infra remote service (`-Dcassandra.instance.type=remote -Dcassandra.host=localhost -Dcassandra.cql3.port=9042`), with its Docker check (`ContainerTestService`) overridden. Otherwise every test is skipped with "Docker is NOT available", also for the remote service.
- `testConfirmExist` is `@DisabledOnOs(OS.MAC)`, and I removed that annotation so the reworked test runs here.
- The aggregation tables were truncated before each run, because unlike a fresh container the local instance keeps the rows of the previous run.
Results (`mvn verify` of camel-cassandraql with `-Dit.test='CassandraAggregation*IT,NamedCassandraAggregationRepositoryIT'`, JDK 21):
- With this PR: `CassandraAggregationRepositoryIT` 10/10, `NamedCassandraAggregationRepositoryIT` 10/10, `CassandraAggregationIT` 1/1, `CassandraAggregationSerializedHeadersIT` 1/1. That is 22 tests, 0 failures, 0 skipped. The 8 unit tests also pass. I ran it twice, on a new and on a truncated keyspace.
- With main's `CassandraAggregationRepository` and the reworked ITs (negative control): 6 of 22 fail, which are `testConfirmExist`, `testScan` and `testRecover` in both ITs. `testConfirmExist` fails because `remove` kept nothing under `camel-recovery:Exchange_2`. `testScan` fails because `scan` returns the open aggregations. `testRecover` fails because `recover` returns the open aggregation `Exchange-Recover1`. The other 16 pass.
I updated the testing section of the PR description to match. A run against the container in CI would still be good to have.
_Claude Code on behalf of allthingssecurity_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25210: camel-cassandraql - keep completed exchanges for recovery in the aggregation repository [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27159:
URL: https://github.com/apache/camel/pull/27159#issuecomment-5925490793
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org