You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "JiriOndrusek (via GitHub)" <gi...@apache.org> on 2026/10/06 14:30:57 UTC
[PR] [camel-main] Fixes #9152. Compose the langchain4j-ingest pipelines from the catalog Kamelets [camel-quarkus]
JiriOndrusek opened a new pull request, #9282:
URL: https://github.com/apache/camel-quarkus/pull/9282
Fixes #9152
Replaces the extension's own route builder with a composition of the catalog Kamelets: `langchain4j-ingest-file-source`, `tika-extract-text-action` or `docling-convert-action`, and `langchain4j-ingest-sink`, always loaded from the classpath. Each Kamelet runs as its own route, and the extension registers the four Kamelets for native itself. Ingestion results stay the same apart from the edge cases listed in the 4.0 migration guide, which also covers the native image size, the three new dependencies and the Tika reply headers.
**Not mergeable yet:** needs the camel-kamelets 4.23.0 release, the first with these Kamelets. Until then the root pom pins 4.23.0-SNAPSHOT, marked with a TODO.
Tested locally: deployment tests, JVM and native integration tests.
Follow-up: #9280 adds `filter.*` and `modality=media` on top.
_Claude Code on behalf of @JiriOndrusek. AI-generated; please verify before merging._
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [camel-main] Fixes #9152. Compose the langchain4j-ingest pipelines from the catalog Kamelets [camel-quarkus]
Posted by "jamesnetherton (via GitHub)" <gi...@apache.org>.
jamesnetherton commented on code in PR #9282:
URL: https://github.com/apache/camel-quarkus/pull/9282#discussion_r4204435517
##########
docs/modules/ROOT/pages/reference/extensions/langchain4j-ingest.adoc:
##########
@@ -64,6 +64,16 @@ Each file is read as UTF-8 text — unless a `parser` turns binary formats into
Every segment carries `camel_ingest_pipeline` and `camel_ingest_document_id` metadata, so retrieval can cite which document an answer came from (up to 3.39.x the keys were `camel_quarkus_pipeline` and `camel_quarkus_document_id` — see the migration guide).
Apart from `enabled` and the `source.*` settings shown here, properties are fixed at build time.
A pipeline declared through runtime properties alone — nothing but a `source.directory`, say — is invisible to build-time validation; its checks, including the clash with an equally named `@Ingest` pipeline, report at startup instead.
+Each pipeline is composed from the catalog's `langchain4j-ingest-file-source`, parser action and `langchain4j-ingest-sink` Kamelets: its route `langchain4j-ingest-<name>` plus one route per Kamelet it uses, `langchain4j-ingest-<name>-source` (directory pipelines), `-parser` (with a `parser`) and `-sink`.
+The pipelines always load these Kamelets from the classpath, whatever `camel.component.kamelet.location` says; shipping an equally named Kamelet replaces internal steps — including the identity capture — and is advanced surgery.
+Application-wide `camel.kamelet.<kamelet>.*` properties and `CAMEL_KAMELET_*` environment variables for these four Kamelets reach the pipelines too, for every option the extension leaves unset, so do not set them.
+To pause a directory pipeline, stop its `langchain4j-ingest-<name>-source` route, or both routes: with `langchain4j-ingest-<name>` alone stopped, the directory is still polled and each new file fails after the Kamelet's 30 s timeout.
+
+[TIP]
+====
+In native mode the kamelet extension embeds every Kamelet on the classpath by default (`quarkus.camel.kamelet.identifiers=*`) — with the `camel-kamelets` catalog this extension brings, some 260 of them.
+This extension embeds its own four Kamelets itself, so list only the application's own Kamelets in `quarkus.camel.kamelet.identifiers` to keep the image small, or set it to `none` (a name matching no Kamelet) when there are none.
+====
Review Comment:
IMO the optimization benefit here is overstated and the `quarkus.camel.kamelet.identifiers=none` idiom is a bit of a hack.
I'd remove it for now (in the docs and itest) and I can think about how to optimize the Kamelet extension better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [camel-main] Fixes #9152. Compose the langchain4j-ingest pipelines from the catalog Kamelets [camel-quarkus]
Posted by "jiriOndrusek-agent (via GitHub)" <gi...@apache.org>.
jiriOndrusek-agent commented on PR #9282:
URL: https://github.com/apache/camel-quarkus/pull/9282#issuecomment-6213336073
Superseded by #9339 (rebased on main after #9315).
_Generated with [Claude Code](https://claude.com/claude-code) on behalf of @JiriOndrusek_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [camel-main] Fixes #9152. Compose the langchain4j-ingest pipelines from the catalog Kamelets [camel-quarkus]
Posted by "jiriOndrusek-agent (via GitHub)" <gi...@apache.org>.
jiriOndrusek-agent commented on code in PR #9282:
URL: https://github.com/apache/camel-quarkus/pull/9282#discussion_r4207080509
##########
docs/modules/ROOT/pages/reference/extensions/langchain4j-ingest.adoc:
##########
@@ -64,6 +64,16 @@ Each file is read as UTF-8 text — unless a `parser` turns binary formats into
Every segment carries `camel_ingest_pipeline` and `camel_ingest_document_id` metadata, so retrieval can cite which document an answer came from (up to 3.39.x the keys were `camel_quarkus_pipeline` and `camel_quarkus_document_id` — see the migration guide).
Apart from `enabled` and the `source.*` settings shown here, properties are fixed at build time.
A pipeline declared through runtime properties alone — nothing but a `source.directory`, say — is invisible to build-time validation; its checks, including the clash with an equally named `@Ingest` pipeline, report at startup instead.
+Each pipeline is composed from the catalog's `langchain4j-ingest-file-source`, parser action and `langchain4j-ingest-sink` Kamelets: its route `langchain4j-ingest-<name>` plus one route per Kamelet it uses, `langchain4j-ingest-<name>-source` (directory pipelines), `-parser` (with a `parser`) and `-sink`.
+The pipelines always load these Kamelets from the classpath, whatever `camel.component.kamelet.location` says; shipping an equally named Kamelet replaces internal steps — including the identity capture — and is advanced surgery.
+Application-wide `camel.kamelet.<kamelet>.*` properties and `CAMEL_KAMELET_*` environment variables for these four Kamelets reach the pipelines too, for every option the extension leaves unset, so do not set them.
+To pause a directory pipeline, stop its `langchain4j-ingest-<name>-source` route, or both routes: with `langchain4j-ingest-<name>` alone stopped, the directory is still polled and each new file fails after the Kamelet's 30 s timeout.
+
+[TIP]
+====
+In native mode the kamelet extension embeds every Kamelet on the classpath by default (`quarkus.camel.kamelet.identifiers=*`) — with the `camel-kamelets` catalog this extension brings, some 260 of them.
+This extension embeds its own four Kamelets itself, so list only the application's own Kamelets in `quarkus.camel.kamelet.identifiers` to keep the image small, or set it to `none` (a name matching no Kamelet) when there are none.
+====
Review Comment:
Done, removed from the docs (including the migration guide) and the itest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [camel-main] Fixes #9152. Compose the langchain4j-ingest pipelines from the catalog Kamelets [camel-quarkus]
Posted by "JiriOndrusek (via GitHub)" <gi...@apache.org>.
JiriOndrusek commented on PR #9282:
URL: https://github.com/apache/camel-quarkus/pull/9282#issuecomment-6213355969
superseded by https://github.com/apache/camel-quarkus/pull/9339
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [camel-main] Fixes #9152. Compose the langchain4j-ingest pipelines from the catalog Kamelets [camel-quarkus]
Posted by "JiriOndrusek (via GitHub)" <gi...@apache.org>.
JiriOndrusek commented on code in PR #9282:
URL: https://github.com/apache/camel-quarkus/pull/9282#discussion_r4206898778
##########
docs/modules/ROOT/pages/reference/extensions/langchain4j-ingest.adoc:
##########
@@ -64,6 +64,16 @@ Each file is read as UTF-8 text — unless a `parser` turns binary formats into
Every segment carries `camel_ingest_pipeline` and `camel_ingest_document_id` metadata, so retrieval can cite which document an answer came from (up to 3.39.x the keys were `camel_quarkus_pipeline` and `camel_quarkus_document_id` — see the migration guide).
Apart from `enabled` and the `source.*` settings shown here, properties are fixed at build time.
A pipeline declared through runtime properties alone — nothing but a `source.directory`, say — is invisible to build-time validation; its checks, including the clash with an equally named `@Ingest` pipeline, report at startup instead.
+Each pipeline is composed from the catalog's `langchain4j-ingest-file-source`, parser action and `langchain4j-ingest-sink` Kamelets: its route `langchain4j-ingest-<name>` plus one route per Kamelet it uses, `langchain4j-ingest-<name>-source` (directory pipelines), `-parser` (with a `parser`) and `-sink`.
+The pipelines always load these Kamelets from the classpath, whatever `camel.component.kamelet.location` says; shipping an equally named Kamelet replaces internal steps — including the identity capture — and is advanced surgery.
+Application-wide `camel.kamelet.<kamelet>.*` properties and `CAMEL_KAMELET_*` environment variables for these four Kamelets reach the pipelines too, for every option the extension leaves unset, so do not set them.
+To pause a directory pipeline, stop its `langchain4j-ingest-<name>-source` route, or both routes: with `langchain4j-ingest-<name>` alone stopped, the directory is still polled and each new file fails after the Kamelet's 30 s timeout.
+
+[TIP]
+====
+In native mode the kamelet extension embeds every Kamelet on the classpath by default (`quarkus.camel.kamelet.identifiers=*`) — with the `camel-kamelets` catalog this extension brings, some 260 of them.
+This extension embeds its own four Kamelets itself, so list only the application's own Kamelets in `quarkus.camel.kamelet.identifiers` to keep the image small, or set it to `none` (a name matching no Kamelet) when there are none.
+====
Review Comment:
I agree that it is more like a hack. I'll remove it. thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [camel-main] Fixes #9152. Compose the langchain4j-ingest pipelines from the catalog Kamelets [camel-quarkus]
Posted by "JiriOndrusek (via GitHub)" <gi...@apache.org>.
JiriOndrusek closed pull request #9282: [camel-main] Fixes #9152. Compose the langchain4j-ingest pipelines from the catalog Kamelets
URL: https://github.com/apache/camel-quarkus/pull/9282
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org