You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "JiriOndrusek (via GitHub)" <gi...@apache.org> on 2026/10/08 12:26:34 UTC
[PR] CAMEL-25463: camel-langchain4j-ingest - read a body only up to maxDocumentSize [camel]
JiriOndrusek opened a new pull request, #27569:
URL: https://github.com/apache/camel/pull/27569
# Description
[CAMEL-25463](https://issues.apache.org/jira/browse/CAMEL-25463)
A stream or file body was read whole before `maxDocumentSize` was checked. It is now read only one byte (media) or character (text) past the cap, enough to refuse it.
- The cap is checked before the blank check, since a capped read sees only the start. A body over the cap now fails even if all blank; it answered `EMPTY` before.
- With a cap, a `File` or `Path` body decodes strictly, like a file-consumer body.
- Docs: stream caching (on by default) still reads a stream whole before the endpoint, into memory unless it spools, so limit the size at the consumer too.
# Target
- [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch)
# Tracking
- [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it).
# Apache Camel coding standards and style
- [x] I checked that each commit in the pull request has a meaningful subject line and body.
- [x] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes.
# AI-assisted contributions
- [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used.
_Claude Code on behalf of @JiriOndrusek_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25463: camel-langchain4j-ingest - read a body only up to maxDocumentSize [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27569:
URL: https://github.com/apache/camel/pull/27569#issuecomment-6059849897
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25463: camel-langchain4j-ingest - read a body only up to maxDocumentSize [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #27569:
URL: https://github.com/apache/camel/pull/27569#issuecomment-6060793293
@JiriOndrusek you need to rebase from main
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25463: camel-langchain4j-ingest - read a body only up to maxDocumentSize [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27569:
URL: https://github.com/apache/camel/pull/27569#issuecomment-6060715881
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `catalog/camel-catalog`
- `components/camel-ai/camel-langchain4j-ingest`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 of 701 tested, 26 compile-only — current: 9 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **9 affected modules** (current approach: 9).
**Skip-tests mode would test 9 modules** (2 direct + 8 downstream), **skip tests for 26** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (9)</summary>
- `camel-jbang-mcp` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-mcp` ← downstream of org.apache.camel:camel-jbang-core
- `camel-jbang-plugin-route-parser` ← downstream of org.apache.camel:camel-route-parser
- `camel-jbang-plugin-tui` ← downstream of org.apache.camel:camel-catalog
- `camel-jbang-plugin-validate` ← downstream of org.apache.camel:camel-yaml-dsl-validator
- `camel-langchain4j-ingest` ← components/camel-ai/camel-langchain4j-ingest/src/main/docs/langchain4j-ingest-component.adoc, components/camel-ai/camel-langchain4j-ingest/src/main/java/org/apache/camel/component/langchain4j/ingest/IngestService.java, components/camel-ai/camel-langchain4j-ingest/src/main/java/org/apache/camel/component/langchain4j/ingest/LangChain4jIngestProducer.java, components/camel-ai/camel-langchain4j-ingest/src/test/java/org/apache/camel/component/langchain4j/ingest/LangChain4jIngestMediaTest.java, components/camel-ai/camel-langchain4j-ingest/src/test/java/org/apache/camel/component/langchain4j/ingest/LangChain4jIngestProducerTest.java
- `camel-launcher-container` ← downstream of org.apache.camel:camel-launcher
- `camel-yaml-dsl-validator` ← downstream of org.apache.camel:camel-catalog
- `camel-yaml-dsl-validator-maven-plugin` ← downstream of org.apache.camel:camel-yaml-dsl-validator
</details>
<details><summary>Modules with tests skipped (26)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
<details><summary>All tested modules (36 modules, 3m 48s total)</summary>
**Total reactor time:** 3m 48s
| Module | Duration | Status |
| --- | --- | --- |
| Camel :: AI :: LangChain4j :: Ingest | 35.0s | SUCCESS |
| Camel :: JBang :: Plugin :: TUI | 32.4s | FAILURE |
| Camel :: Component DSL | 16.3s | SUCCESS |
| Camel :: Catalog :: Camel Catalog | 15.4s | SUCCESS |
| Camel :: YAML DSL | 14.9s | SUCCESS |
| Camel :: JBang :: MCP | 14.3s | SUCCESS |
| Camel :: YAML DSL :: Validator | 13.1s | SUCCESS |
| Camel :: JBang :: Plugin :: Validate | 12.0s | SUCCESS |
| Camel :: Docs | 10.7s | SUCCESS |
| Camel :: Kamelet Main | 9.7s | SUCCESS |
| Camel :: JBang :: Plugin :: Testing | 8.4s | SUCCESS |
| Camel :: YAML DSL :: Deserializers | 6.9s | SUCCESS |
| Camel :: JBang :: Plugin :: Kubernetes | 6.6s | SUCCESS |
| Camel :: Catalog :: Camel Route Parser | 6.1s | SUCCESS |
| Camel :: Catalog :: Camel Report Maven Plugin | 4.8s | SUCCESS |
| Camel :: YAML DSL :: Validator Maven Plugin | 3.8s | SUCCESS |
| Camel :: All Components Sync point | 3.3s | SUCCESS |
| Camel :: YAML DSL :: Maven Plugins | 2.5s | SUCCESS |
| Camel :: Catalog :: Suggest (deprecated) | 2.3s | SUCCESS |
| Camel :: Catalog :: Maven | 1.6s | SUCCESS |
| Camel :: Assembly | 1.1s | SUCCESS |
| Camel :: JBang :: Plugin :: Edit | 1.0s | SUCCESS |
| Camel :: Coverage | 0.9s | SUCCESS |
| Camel :: JBang :: Plugin :: Generate | 0.9s | SUCCESS |
| Camel :: Catalog :: Dummy Component | 0.9s | SUCCESS |
| Camel :: Catalog :: Console | 0.8s | SUCCESS |
| Camel :: Endpoint DSL :: Support | 0.7s | SUCCESS |
| Camel :: JBang :: Main | 0.5s | SUCCESS |
| Camel :: JBang :: Plugin :: Route Parser | 0.5s | SUCCESS |
| Camel :: JBang :: Plugin :: MCP | 0.5s | SUCCESS |
| Camel :: Endpoint DSL | n/a | |
| Camel :: Integration Tests | n/a | |
| Camel :: JBang :: Core | n/a | |
| Camel :: JBang :: Integration tests | n/a | |
| Camel :: Launcher | n/a | |
| Camel :: Launcher :: Container | n/a | |
**Top 20 slowest modules:**
- `Camel :: AI :: LangChain4j :: Ingest` (35.0s)
- `Camel :: JBang :: Plugin :: TUI` (32.4s)
- `Camel :: Component DSL` (16.3s)
- `Camel :: Catalog :: Camel Catalog` (15.4s)
- `Camel :: YAML DSL` (14.9s)
- `Camel :: JBang :: MCP` (14.3s)
- `Camel :: YAML DSL :: Validator` (13.1s)
- `Camel :: JBang :: Plugin :: Validate` (12.0s)
- `Camel :: Docs` (10.7s)
- `Camel :: Kamelet Main` (9.7s)
- `Camel :: JBang :: Plugin :: Testing` (8.4s)
- `Camel :: YAML DSL :: Deserializers` (6.9s)
- `Camel :: JBang :: Plugin :: Kubernetes` (6.6s)
- `Camel :: Catalog :: Camel Route Parser` (6.1s)
- `Camel :: Catalog :: Camel Report Maven Plugin` (4.8s)
- `Camel :: YAML DSL :: Validator Maven Plugin` (3.8s)
- `Camel :: All Components Sync point` (3.3s)
- `Camel :: YAML DSL :: Maven Plugins` (2.5s)
- `Camel :: Catalog :: Suggest (deprecated)` (2.3s)
- `Camel :: Catalog :: Maven` (1.6s)
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/37776883288)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25463: camel-langchain4j-ingest - read a body only up to maxDocumentSize [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on code in PR #27569:
URL: https://github.com/apache/camel/pull/27569#discussion_r4219987755
##########
components/camel-ai/camel-langchain4j-ingest/src/main/java/org/apache/camel/component/langchain4j/ingest/LangChain4jIngestProducer.java:
##########
@@ -290,15 +294,64 @@ private IngestResult ingestUnlessFiltered(Exchange exchange, String documentId,
if (announced != null) {
mediaService.checkSize(documentId, announced);
}
- // bytes, not a String - a charset conversion would corrupt them. A null body stays
- // EMPTY; anything else must convert, so a wrong body type (a POJO, say) fails as an
- // InvalidPayloadException instead of being answered EMPTY by a silent null conversion
- byte[] bytes = exchange.getMessage().getBody() == null
- ? null
- : exchange.getMessage().getMandatoryBody(byte[].class);
- return mediaService.ingest(documentId, bytes, mimeType, medium);
- }
- return textService.ingest(documentId, exchange.getMessage().getBody(String.class));
+ return mediaService.ingest(documentId, mediaBody(exchange), mimeType, medium);
+ }
+ return textService.ingest(documentId, textBody(exchange));
+ }
+
+ /**
+ * The media body as bytes, not a String - a charset conversion would corrupt them. A null body stays EMPTY;
+ * anything else must convert, so a wrong body type (a POJO, say) fails as an InvalidPayloadException instead of
+ * being answered EMPTY by a silent null conversion. With maxDocumentSize set, a body not yet in memory is read only
+ * one byte past the cap, enough for the size check to refuse it.
+ */
+ private byte[] mediaBody(Exchange exchange) throws InvalidPayloadException, IOException {
+ Object body = exchange.getMessage().getBody();
+ int limit = readLimit();
+ InputStream stream = body == null || body instanceof byte[] || limit < 0
+ ? null : exchange.getMessage().getBody(InputStream.class);
+ if (stream == null) {
+ return body == null ? null : exchange.getMessage().getMandatoryBody(byte[].class);
+ }
+ try (stream) {
+ return stream.readNBytes(limit);
+ }
+ }
+
+ /**
+ * The text body. With maxDocumentSize set, a body not yet a String is read only one character past the cap, enough
+ * for the size check to refuse it.
+ */
+ private String textBody(Exchange exchange) throws IOException {
+ Object body = exchange.getMessage().getBody();
+ int limit = readLimit();
+ Reader reader = body == null || body instanceof String || limit < 0
+ ? null : readerOf(exchange);
+ if (reader == null) {
+ return exchange.getMessage().getBody(String.class);
+ }
+ try (reader) {
+ StringBuilder text = new StringBuilder();
+ char[] chunk = new char[8192];
+ int read;
+ while (text.length() < limit
+ && (read = reader.read(chunk, 0, Math.min(chunk.length, limit - text.length()))) != -1) {
+ text.append(chunk, 0, read);
+ }
+ return text.toString();
+ }
+ }
+
+ /** The body as a Reader, or null; a File or a Path converts to a BufferedReader, not to a Reader. */
+ private static Reader readerOf(Exchange exchange) {
Review Comment:
Optional, not blocking: with a cap, a `File`/`Path` body is decoded through `Files.newBufferedReader`, which uses UTF-8 when no exchange charset is set and throws `MalformedInputException` on bad bytes. Without a cap it goes through `IOConverter.toString(Path)`, which falls back to `new String(bytes)`: the platform charset, with bad bytes replaced. So a Latin-1 file that ingests without a cap fails once `maxDocumentSize` is set.
The PR description says this is intended, to match file-consumer bodies, and that makes sense. It might be worth one sentence in the component doc too, so users don't find it only by hitting the exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25463: camel-langchain4j-ingest - read a body only up to maxDocumentSize [camel]
Posted by "gnodet-bot (via GitHub)" <gi...@apache.org>.
gnodet-bot commented on code in PR #27569:
URL: https://github.com/apache/camel/pull/27569#discussion_r4219833432
##########
components/camel-ai/camel-langchain4j-ingest/src/main/java/org/apache/camel/component/langchain4j/ingest/LangChain4jIngestProducer.java:
##########
@@ -290,15 +294,64 @@ private IngestResult ingestUnlessFiltered(Exchange exchange, String documentId,
if (announced != null) {
mediaService.checkSize(documentId, announced);
}
- // bytes, not a String - a charset conversion would corrupt them. A null body stays
- // EMPTY; anything else must convert, so a wrong body type (a POJO, say) fails as an
- // InvalidPayloadException instead of being answered EMPTY by a silent null conversion
- byte[] bytes = exchange.getMessage().getBody() == null
- ? null
- : exchange.getMessage().getMandatoryBody(byte[].class);
- return mediaService.ingest(documentId, bytes, mimeType, medium);
- }
- return textService.ingest(documentId, exchange.getMessage().getBody(String.class));
+ return mediaService.ingest(documentId, mediaBody(exchange), mimeType, medium);
+ }
+ return textService.ingest(documentId, textBody(exchange));
+ }
+
+ /**
+ * The media body as bytes, not a String - a charset conversion would corrupt them. A null body stays EMPTY;
+ * anything else must convert, so a wrong body type (a POJO, say) fails as an InvalidPayloadException instead of
+ * being answered EMPTY by a silent null conversion. With maxDocumentSize set, a body not yet in memory is read only
+ * one byte past the cap, enough for the size check to refuse it.
+ */
+ private byte[] mediaBody(Exchange exchange) throws InvalidPayloadException, IOException {
+ Object body = exchange.getMessage().getBody();
+ int limit = readLimit();
+ InputStream stream = body == null || body instanceof byte[] || limit < 0
+ ? null : exchange.getMessage().getBody(InputStream.class);
+ if (stream == null) {
+ return body == null ? null : exchange.getMessage().getMandatoryBody(byte[].class);
+ }
+ try (stream) {
+ return stream.readNBytes(limit);
+ }
+ }
+
+ /**
+ * The text body. With maxDocumentSize set, a body not yet a String is read only one character past the cap, enough
+ * for the size check to refuse it.
+ */
+ private String textBody(Exchange exchange) throws IOException {
+ Object body = exchange.getMessage().getBody();
+ int limit = readLimit();
+ Reader reader = body == null || body instanceof String || limit < 0
+ ? null : readerOf(exchange);
+ if (reader == null) {
+ return exchange.getMessage().getBody(String.class);
+ }
+ try (reader) {
+ StringBuilder text = new StringBuilder();
+ char[] chunk = new char[8192];
+ int read;
+ while (text.length() < limit
+ && (read = reader.read(chunk, 0, Math.min(chunk.length, limit - text.length()))) != -1) {
+ text.append(chunk, 0, read);
+ }
+ return text.toString();
+ }
+ }
+
+ /** The body as a Reader, or null; a File or a Path converts to a BufferedReader, not to a Reader. */
Review Comment:
💡 **Minor:** The Javadoc accurately states the *what* — `File`/`Path` bodies convert to `BufferedReader`, not `Reader` — but the *why* is non-obvious: Camel's `IOConverter.toReader` has return type `BufferedReader` and is registered under `BufferedReader.class`, so `getBody(Reader.class)` silently returns `null` for those types. Adding one line like:
> `// IOConverter.toReader returns BufferedReader, registered under BufferedReader.class`
or expanding the Javadoc to mention the registration mechanism would save future maintainers a debugging session. Not a blocker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-25463: camel-langchain4j-ingest - read a body only up to maxDocumentSize [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #27569:
URL: https://github.com/apache/camel/pull/27569
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org