You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "davsclaus (via GitHub)" <gi...@apache.org> on 2026/07/14 14:41:14 UTC
[PR] CAMEL-23994: Fix batching manual async commit never reaching Kafka [camel]
davsclaus opened a new pull request, #24686:
URL: https://github.com/apache/camel/pull/24686
## Summary
_Claude Code on behalf of davsclaus_
Fixes the fifth bug reported in [CAMEL-23994](https://issues.apache.org/jira/browse/CAMEL-23994): in batching mode with manual async commit, calling `manual.commit()` only recorded the offset in the in-memory `OffsetCache` but the cache was never flushed to Kafka.
**Root cause:** `DefaultKafkaManualAsyncCommit.commit()` calls `commitManager.recordOffset()` which stores in `OffsetCache`. The cache is flushed to Kafka only when `commitManager.commit(partition)` is called. The streaming facade (`KafkaRecordStreamingProcessorFacade`) calls this after processing each partition's records, but the batching facade (`KafkaRecordBatchingProcessorFacade`) never did.
**Impact:** With `batching=true` + `allowManualCommit=true` + `DefaultKafkaManualAsyncCommitFactory` + `autoCommitEnable=false`, offsets reached Kafka only on graceful partition revocation. On crash, the group offset was still at the initial position, causing the entire history since startup to be redelivered.
**Fix:** The batching facade now flushes recorded offsets for each partition after `processExchange()`, mirroring the streaming facade's behavior.
Note: `DefaultKafkaManualSyncCommit` was not affected since it calls `forceCommit()` which bypasses the cache and directly calls `consumer.commitSync()`.
## Test plan
- [x] All 141 unit tests pass
- [ ] CI integration tests will validate the batching + manual commit path
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.6 <no...@anthropic.com>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-23994: Fix batching manual async commit never reaching Kafka [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #24686:
URL: https://github.com/apache/camel/pull/24686#issuecomment-4970466175
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-23994: Fix batching manual async commit never reaching Kafka [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #24686:
URL: https://github.com/apache/camel/pull/24686#issuecomment-4970871294
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `components/camel-kafka`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 tested, 29 compile-only — current: 10 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **38 affected modules** (current approach: 10).
<details><summary>:warning: Modules only in Scalpel (28)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-lucene`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-csimple-maven-plugin`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
**Skip-tests mode would test 9 modules** (1 direct + 8 downstream), **skip tests for 29** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (9)</summary>
- `camel-jbang-mcp`
- `camel-jbang-plugin-mcp`
- `camel-jbang-plugin-route-parser`
- `camel-jbang-plugin-tui`
- `camel-jbang-plugin-validate`
- `camel-kafka`
- `camel-launcher-container`
- `camel-yaml-dsl-validator`
- `camel-yaml-dsl-validator-maven-plugin`
</details>
<details><summary>Modules with tests skipped (29)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-lucene`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-csimple-maven-plugin`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
:warning: **Some tests are disabled on GitHub Actions** (`@DisabledIfSystemProperty(named = "ci.env.name")`) and require manual verification:
- `components/camel-kafka`: 2 test(s) disabled on GitHub Actions
<details><summary>All tested modules (38 modules)</summary>
- Camel :: All Components Sync point
- Camel :: Assembly
- Camel :: Catalog :: CSimple Maven Plugin (deprecated)
- Camel :: Catalog :: Camel Catalog
- Camel :: Catalog :: Camel Report Maven Plugin
- Camel :: Catalog :: Camel Route Parser
- Camel :: Catalog :: Console
- Camel :: Catalog :: Dummy Component
- Camel :: Catalog :: Lucene (deprecated)
- Camel :: Catalog :: Maven
- Camel :: Catalog :: Suggest
- Camel :: Component DSL
- Camel :: Coverage
- Camel :: Docs
- Camel :: Endpoint DSL
- Camel :: Endpoint DSL :: Support
- Camel :: Integration Tests
- Camel :: JBang :: Core
- Camel :: JBang :: Integration tests
- Camel :: JBang :: MCP
- Camel :: JBang :: Main
- Camel :: JBang :: Plugin :: Edit
- Camel :: JBang :: Plugin :: Generate
- Camel :: JBang :: Plugin :: Kubernetes
- Camel :: JBang :: Plugin :: MCP
- Camel :: JBang :: Plugin :: Route Parser
- Camel :: JBang :: Plugin :: TUI
- Camel :: JBang :: Plugin :: Testing
- Camel :: JBang :: Plugin :: Validate
- Camel :: Kafka
- Camel :: Kamelet Main
- Camel :: Launcher
- Camel :: Launcher :: Container
- Camel :: YAML DSL
- Camel :: YAML DSL :: Deserializers
- Camel :: YAML DSL :: Maven Plugins
- Camel :: YAML DSL :: Validator
- Camel :: YAML DSL :: Validator Maven Plugin
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/29341966312)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-23994: Fix batching manual async commit never reaching Kafka [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #24686:
URL: https://github.com/apache/camel/pull/24686
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org