You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "allthingssecurity (via GitHub)" <gi...@apache.org> on 2026/10/11 00:06:02 UTC
[PR] CAMEL-25525: camel-keycloak - Do not route the latest events of a poll a second time [camel]
allthingssecurity opened a new pull request, #27676:
URL: https://github.com/apache/camel/pull/27676
# Description
[CAMEL-25525](https://issues.apache.org/jira/browse/CAMEL-25525)
`KeycloakConsumer` queries the events of the realm without a lower time bound and skips those older than its checkpoint (`lastEventTime`) and those whose fingerprint it has already routed. When a poll moved the checkpoint to a newer event time, it cleared all the fingerprints, also those of the events at the new checkpoint, so the next poll got those events again from Keycloak and routed them a second time. Every poll that saw a newer event therefore delivered the latest event(s) twice, in the default configuration, for user events and admin events. The docs say the deduplication ensures "events are not processed multiple times".
Now, when the checkpoint moves, the consumer forgets only the fingerprints older than the new checkpoint and keeps those at it (a fingerprint starts with the event time), in both `pollEvents` and `pollAdminEvents`. After each poll the set still holds only the fingerprints of the latest event time. A poll routes the same events as before except the repeated ones, so there is no upgrade-guide note.
Tests:
- New `KeycloakConsumerLatestEventsTest` (mocked `Keycloak` client as in `KeycloakConsumerTest`, `startScheduler=false`, the test calls `poll()`): two events polled twice must be routed once; then a newer event and a second event at the previous latest time are added and polled twice; the same for admin events.
- Without the change both fail: `the events of the first poll must not be routed again ==> expected: <0> but was: <1>` (and the same for admin events). The existing `KeycloakConsumerTest` uses `expectedMinimumMessageCount(2)`, so it does not see the third delivery.
- With the change: camel-keycloak 141 unit tests, 0 failures (the integration tests need Docker and were skipped).
Not changed here: the size limit of 1000 fingerprints (not reached with the default `maxResults`); Keycloak returns at most `maxResults` (default 100) events per query, ordered by descending time, so a poll with more new events than that misses the older ones; that needs paging with `first` and is better handled in its own issue.
Found with a Lean 4 model of the poll: "a second poll of the same events routes nothing" fails for two events, and main is proved to route again every event at the time a poll moves the checkpoint to; the fix is proved to never route again an event the previous poll routed. I then reproduced it with the consumer.
# Target
- [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch)
# Tracking
- [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it).
# Apache Camel coding standards and style
- [x] I checked that each commit in the pull request has a meaningful subject line and body.
- [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes.
(I built and tested camel-keycloak, including the formatter and import-sort plugins. I did not run the full root build.)
# AI-assisted contributions
- [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used.
This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer.
_Claude Code on behalf of allthingssecurity_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25525: camel-keycloak - Do not route the latest events of a poll a second time [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27676:
URL: https://github.com/apache/camel/pull/27676#issuecomment-6105969808
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25525: camel-keycloak - Do not route the latest events of a poll a second time [camel]
Posted by "allthingssecurity (via GitHub)" <gi...@apache.org>.
allthingssecurity commented on PR #27676:
URL: https://github.com/apache/camel/pull/27676#issuecomment-6106205104
- `keycloak-consumer.adoc`: confirmed, the sentence "Fingerprints are cached per timestamp and cleared when moving to
newer timestamps" no longer matched the code. Fixed in 47d7a2f39c78: the fingerprints of the events with the latest
timestamp are kept between polls, older ones are dropped. The catalog copy is regenerated.
- Admin-events test: `routesEachAdminEventOnce` now adds a second admin event at the previous latest time, as the
user-events test does, and checks the resource paths of the 4 routed events (47d7a2f39c78).
- camel-keycloak suite: 141 tests, 0 failures (the 133 integration tests are skipped without a Keycloak container).
- `EventRepresentation.getId()` as the dedup key: not changed in this PR.
_Claude Code on behalf of allthingssecurity_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25525: camel-keycloak - Do not route the latest events of a poll a second time [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #27676:
URL: https://github.com/apache/camel/pull/27676
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25525: camel-keycloak - Do not route the latest events of a poll a second time [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27676:
URL: https://github.com/apache/camel/pull/27676#issuecomment-6106190095
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `components/camel-keycloak`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 of 704 tested, 27 compile-only — current: 9 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **9 affected modules** (current approach: 9).
**Skip-tests mode would test 9 modules** (1 direct + 8 downstream), **skip tests for 27** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (9)</summary>
- `camel-jbang-mcp` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-mcp` ← downstream of org.apache.camel:camel-jbang-core
- `camel-jbang-plugin-route-parser` ← downstream of org.apache.camel:camel-route-parser
- `camel-jbang-plugin-tui` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-validate` ← downstream of org.apache.camel:camel-yaml-dsl-validator
- `camel-keycloak` ← components/camel-keycloak/src/main/java/org/apache/camel/component/keycloak/KeycloakConsumer.java, components/camel-keycloak/src/test/java/org/apache/camel/component/keycloak/KeycloakConsumerLatestEventsTest.java
- `camel-launcher-container` ← downstream of org.apache.camel:camel-launcher
- `camel-yaml-dsl-validator` ← downstream of org.apache.camel:camel-catalog
- `camel-yaml-dsl-validator-maven-plugin` ← downstream of org.apache.camel:camel-yaml-dsl-validator
</details>
<details><summary>Modules with tests skipped (27)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
<details><summary>All tested modules (36 modules, 5m 8s total)</summary>
**Total reactor time:** 5m 8s
| Module | Duration | Status |
| --- | --- | --- |
| Camel :: Launcher | 50.7s | SUCCESS |
| Camel :: JBang :: MCP | 40.9s | SUCCESS |
| Camel :: Component DSL | 24.0s | SUCCESS |
| Camel :: JBang :: Plugin :: Kubernetes | 23.0s | SUCCESS |
| Camel :: Catalog :: Camel Catalog | 22.1s | SUCCESS |
| Camel :: YAML DSL | 18.8s | SUCCESS |
| Camel :: YAML DSL :: Validator | 17.9s | SUCCESS |
| Camel :: JBang :: Plugin :: Validate | 16.5s | SUCCESS |
| Camel :: Docs | 14.7s | SUCCESS |
| Camel :: JBang :: Plugin :: Testing | 13.7s | SUCCESS |
| Camel :: Kamelet Main | 11.9s | SUCCESS |
| Camel :: YAML DSL :: Deserializers | 8.1s | SUCCESS |
| Camel :: Catalog :: Camel Route Parser | 7.3s | SUCCESS |
| Camel :: Catalog :: Camel Report Maven Plugin | 6.9s | SUCCESS |
| Camel :: All Components Sync point | 5.2s | SUCCESS |
| Camel :: YAML DSL :: Validator Maven Plugin | 4.7s | SUCCESS |
| Camel :: YAML DSL :: Maven Plugins | 3.3s | SUCCESS |
| Camel :: Catalog :: Maven | 3.0s | SUCCESS |
| Camel :: Catalog :: Suggest (deprecated) | 2.6s | SUCCESS |
| Camel :: Assembly | 1.5s | SUCCESS |
| Camel :: JBang :: Plugin :: Edit | 1.4s | SUCCESS |
| Camel :: JBang :: Plugin :: MCP | 1.4s | SUCCESS |
| Camel :: Catalog :: Dummy Component | 1.3s | SUCCESS |
| Camel :: Coverage | 1.2s | SUCCESS |
| Camel :: JBang :: Integration tests | 1.1s | SUCCESS |
| Camel :: JBang :: Plugin :: Generate | 1.0s | SUCCESS |
| Camel :: Catalog :: Console | 0.9s | SUCCESS |
| Camel :: JBang :: Main | 0.8s | SUCCESS |
| Camel :: Endpoint DSL :: Support | 0.8s | SUCCESS |
| Camel :: Launcher :: Container | 0.7s | SUCCESS |
| Camel :: JBang :: Plugin :: Route Parser | 0.6s | SUCCESS |
| Camel :: Endpoint DSL | n/a | |
| Camel :: Integration Tests | n/a | |
| Camel :: JBang :: Core | n/a | |
| Camel :: JBang :: Plugin :: TUI | n/a | |
| Camel :: Keycloak | n/a | |
**Top 20 slowest modules:**
- `Camel :: Launcher` (50.7s)
- `Camel :: JBang :: MCP` (40.9s)
- `Camel :: Component DSL` (24.0s)
- `Camel :: JBang :: Plugin :: Kubernetes` (23.0s)
- `Camel :: Catalog :: Camel Catalog` (22.1s)
- `Camel :: YAML DSL` (18.8s)
- `Camel :: YAML DSL :: Validator` (17.9s)
- `Camel :: JBang :: Plugin :: Validate` (16.5s)
- `Camel :: Docs` (14.7s)
- `Camel :: JBang :: Plugin :: Testing` (13.7s)
- `Camel :: Kamelet Main` (11.9s)
- `Camel :: YAML DSL :: Deserializers` (8.1s)
- `Camel :: Catalog :: Camel Route Parser` (7.3s)
- `Camel :: Catalog :: Camel Report Maven Plugin` (6.9s)
- `Camel :: All Components Sync point` (5.2s)
- `Camel :: YAML DSL :: Validator Maven Plugin` (4.7s)
- `Camel :: YAML DSL :: Maven Plugins` (3.3s)
- `Camel :: Catalog :: Maven` (3.0s)
- `Camel :: Catalog :: Suggest (deprecated)` (2.6s)
- `Camel :: Assembly` (1.5s)
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/38097271648)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org