You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "oscerd (via GitHub)" <gi...@apache.org> on 2026/09/16 16:15:04 UTC
[PR] CAMEL-24779: camel-kafka - reduce per-message allocations on the producer, consumer and transform hot paths [camel]
oscerd opened a new pull request, #26522:
URL: https://github.com/apache/camel/pull/26522
## What this does
Three small, **behaviour-preserving** performance clean-ups that remove per-message allocations on the camel-kafka hot paths. No public API or option changes.
### 1. Producer — redundant callback allocation on the single-message async path
`KafkaProducer.process(Exchange, AsyncCallback)` sent a single record via `doSend(exchange, record, cb)`. Because the key was non-null, `doSend` allocated a `KafkaProducerMetadataCallBack` **and** a `DelegatingCallback` for every message — even though the parent `KafkaProducerCallBack` already records the metadata and any exception on the same exchange. The single-message path now passes a `null` key, so it sends with the parent callback alone (two fewer short-lived allocations per message). The resulting `CamelKafkaRecordMeta` header is unchanged (a one-element `List<RecordMetadata>`). The iterator/batch path is untouched — it genuinely needs a per-element metadata callback.
### 2. Consumer — per-record `Stream`/lambda allocations in header propagation
`KafkaRecordProcessor.propagateHeaders` built a `Stream` + spliterator + two capturing lambdas for **every** consumed record and re-resolved `exchange.getIn()` per header. It now iterates `consumerRecord.headers()` with a plain loop and a hoisted `Message`. Behaviour is identical.
### 3. Transforms — `ObjectMapper` constructed per message
Six JSON transform beans (`HoistField`, `MaskField`, `ExtractField`, `ReplaceField`, `MessageTimestampRouter`, `ValueToKey`) built a `new ObjectMapper()` on every invocation. `ObjectMapper` is expensive to construct and thread-safe once configured; these back the corresponding Kamelet actions and run per message. They now reuse a single shared static `ObjectMapper`.
## Tests
These are behaviour-preserving optimizations covered by existing tests, all green in a full `camel-kafka` module build (unit + integration):
- `KafkaProducerTest` (23 tests) — including `processAsyncSendsMessage`, which exercises the single-message async path and asserts the `CamelKafkaRecordMeta` header.
- The transform unit tests (`HoistFieldTest`, `MaskFieldTest`, `MessageTimestampRouterTest`, `ReplaceFieldTest`, …).
- The consumer integration tests (`KafkaConsumerFullIT`, …) which cover header propagation.
JIRA: https://issues.apache.org/jira/browse/CAMEL-24779
_Claude Code on behalf of @oscerd_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24779: camel-kafka - reduce per-message allocations on the producer, consumer and transform hot paths [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26522:
URL: https://github.com/apache/camel/pull/26522#issuecomment-5701812086
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `components/camel-kafka`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 tested, 27 compile-only — current: 9 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **36 affected modules** (current approach: 9).
<details><summary>:warning: Modules only in Scalpel (27)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
**Skip-tests mode would test 9 modules** (1 direct + 8 downstream), **skip tests for 27** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (9)</summary>
- `camel-jbang-mcp`
- `camel-jbang-plugin-mcp`
- `camel-jbang-plugin-route-parser`
- `camel-jbang-plugin-tui`
- `camel-jbang-plugin-validate`
- `camel-kafka`
- `camel-launcher-container`
- `camel-yaml-dsl-validator`
- `camel-yaml-dsl-validator-maven-plugin`
</details>
<details><summary>Modules with tests skipped (27)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
:warning: **Some tests are disabled on GitHub Actions** (`@DisabledIfSystemProperty(named = "ci.env.name")`) and require manual verification:
- `components/camel-kafka`: 2 test(s) disabled on GitHub Actions
<details><summary>All tested modules (36 modules, 4m 38s total)</summary>
**Total reactor time:** 4m 38s
| Module | Duration | Status |
| --- | --- | --- |
| Camel :: Launcher | 50.7s | SUCCESS |
| Camel :: JBang :: MCP | 41.4s | SUCCESS |
| Camel :: JBang :: Plugin :: TUI | 34.3s | SUCCESS |
| Camel :: Catalog :: Camel Catalog | 20.0s | SUCCESS |
| Camel :: Component DSL | 19.6s | SUCCESS |
| Camel :: JBang :: Plugin :: Kubernetes | 15.9s | SUCCESS |
| Camel :: Docs | 15.3s | SUCCESS |
| Camel :: Catalog :: Camel Report Maven Plugin | 9.2s | SUCCESS |
| Camel :: YAML DSL :: Validator | 8.6s | SUCCESS |
| Camel :: Catalog :: Camel Route Parser | 8.1s | SUCCESS |
| Camel :: Kamelet Main | 7.4s | SUCCESS |
| Camel :: JBang :: Plugin :: Testing | 7.3s | SUCCESS |
| Camel :: All Components Sync point | 5.4s | SUCCESS |
| Camel :: YAML DSL :: Deserializers | 5.3s | SUCCESS |
| Camel :: JBang :: Plugin :: Validate | 5.2s | SUCCESS |
| Camel :: YAML DSL :: Validator Maven Plugin | 4.3s | SUCCESS |
| Camel :: Catalog :: Maven | 3.5s | SUCCESS |
| Camel :: YAML DSL :: Maven Plugins | 2.9s | SUCCESS |
| Camel :: Catalog :: Suggest (deprecated) | 2.4s | SUCCESS |
| Camel :: Assembly | 1.7s | SUCCESS |
| Camel :: Coverage | 1.7s | SUCCESS |
| Camel :: JBang :: Plugin :: Edit | 1.2s | SUCCESS |
| Camel :: Catalog :: Dummy Component | 1.1s | SUCCESS |
| Camel :: Endpoint DSL :: Support | 1.1s | SUCCESS |
| Camel :: JBang :: Plugin :: Generate | 1.0s | SUCCESS |
| Camel :: Catalog :: Console | 0.8s | SUCCESS |
| Camel :: JBang :: Integration tests | 0.8s | SUCCESS |
| Camel :: JBang :: Main | 0.7s | SUCCESS |
| Camel :: Launcher :: Container | 0.6s | SUCCESS |
| Camel :: JBang :: Plugin :: MCP | 0.6s | SUCCESS |
| Camel :: JBang :: Plugin :: Route Parser | 0.5s | SUCCESS |
| Camel :: Endpoint DSL | n/a | |
| Camel :: Integration Tests | n/a | |
| Camel :: JBang :: Core | n/a | |
| Camel :: Kafka | n/a | |
| Camel :: YAML DSL | n/a | |
**Top 20 slowest modules:**
- `Camel :: Launcher` (50.7s)
- `Camel :: JBang :: MCP` (41.4s)
- `Camel :: JBang :: Plugin :: TUI` (34.3s)
- `Camel :: Catalog :: Camel Catalog` (20.0s)
- `Camel :: Component DSL` (19.6s)
- `Camel :: JBang :: Plugin :: Kubernetes` (15.9s)
- `Camel :: Docs` (15.3s)
- `Camel :: Catalog :: Camel Report Maven Plugin` (9.2s)
- `Camel :: YAML DSL :: Validator` (8.6s)
- `Camel :: Catalog :: Camel Route Parser` (8.1s)
- `Camel :: Kamelet Main` (7.4s)
- `Camel :: JBang :: Plugin :: Testing` (7.3s)
- `Camel :: All Components Sync point` (5.4s)
- `Camel :: YAML DSL :: Deserializers` (5.3s)
- `Camel :: JBang :: Plugin :: Validate` (5.2s)
- `Camel :: YAML DSL :: Validator Maven Plugin` (4.3s)
- `Camel :: Catalog :: Maven` (3.5s)
- `Camel :: YAML DSL :: Maven Plugins` (2.9s)
- `Camel :: Catalog :: Suggest (deprecated)` (2.4s)
- `Camel :: Assembly` (1.7s)
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/35120679086)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24779: camel-kafka - reduce per-message allocations on the producer, consumer and transform hot paths [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26522:
URL: https://github.com/apache/camel/pull/26522#issuecomment-5700872435
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24779: camel-kafka - reduce per-message allocations on the producer, consumer and transform hot paths [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #26522:
URL: https://github.com/apache/camel/pull/26522
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org