You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "oscerd (via GitHub)" <gi...@apache.org> on 2026/08/30 10:07:43 UTC
[PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
oscerd opened a new pull request, #25899:
URL: https://github.com/apache/camel/pull/25899
## Issue
[CAMEL-24527](https://issues.apache.org/jira/browse/CAMEL-24527)
## Problem
The `authToken` option (and the token resolved from `oauthProfile`) was honoured by **only** the chat
task. `ChatPredictor.getPythonScript()` builds a `token='...'` clause and passes it to
`transformers.pipeline()`; none of the other nine task predictors do. As a result, text generation,
summarization, question answering, classification, sentence embeddings, ASR, TTS, text-to-image and
zero-shot classification all failed with HTTP 401 when loading a gated or private model, even with a
token configured.
## Fix
Apply the token centrally in `AbstractTaskPredictor.loadModel()` by exporting it as the standard
`HF_TOKEN` environment variable at the top of the generated handler script:
```python
import os
os.environ['HF_TOKEN'] = '<token>'
```
`transformers` / `huggingface_hub` read `HF_TOKEN` from the environment when no explicit token is
passed, so this authenticates **every** task in a single place instead of threading a token clause
through nine predictors and nine Python templates. The token is read from `config.getAuthToken()`,
which holds either the `authToken` option or the value resolved from an OAuth profile
(`HuggingFaceProducer.resolveOAuthToken`). When no token is configured, the script is unchanged.
## Testing
- New `AuthTokenInjectionTest` verifies the token is exported as `HF_TOKEN` when configured and that
the script is left untouched when it is not.
- `mvn -Psourcecheck validate` green.
_Claude Code on behalf of oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
Posted by "Croway (via GitHub)" <gi...@apache.org>.
Croway commented on PR #25899:
URL: https://github.com/apache/camel/pull/25899#issuecomment-5476098856
```
There are uncommitted changes
HEAD detached at pull/25899/merge
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-agent-component.adoc
modified: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-embeddings-component.adoc
modified: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/spring-ai-chat-component.adoc
no changes added to commit (use "git add" and/or "git commit -a")
diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-agent-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-agent-component.adoc
index a331982d11df..e70651121988 100644
--- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-agent-component.adoc
+++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-agent-component.adoc
@@ -1178,7 +1178,6 @@ You can also define the `ResponseFormat` at the `ChatModel` level. See the https
* The same schema file can be shared across `camel-openai` and `camel-langchain4j-agent` components
====
-[[structured_error_exchange_properties]]
=== Structured error exchange properties
When a LangChain4j agent call fails, Camel sets structured metadata on the exchange **before** the model exception propagates. This works even when GenAI observability is disabled.
diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-embeddings-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-embeddings-component.adoc
index 387d9e382f24..7ea8ec94431c 100644
--- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-embeddings-component.adoc
+++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/langchain4j-embeddings-component.adoc
@@ -395,7 +395,6 @@ YAML::
----
====
-[[structured_error_exchange_properties]]
=== Structured error exchange properties
When a LangChain4j embeddings call fails, Camel sets structured metadata on the exchange **before** the model exception propagates. This works even when GenAI observability is disabled.
diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/spring-ai-chat-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/spring-ai-chat-component.adoc
index c8d8c0a84040..2dea4dd89a09 100644
--- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/spring-ai-chat-component.adoc
+++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/spring-ai-chat-component.adoc
@@ -1407,7 +1407,6 @@ The component automatically adds Spring AI's `SimpleLoggerAdvisor` to log reques
logging.level.org.springframework.ai.chat.client.advisor=DEBUG
----
-[[structured_error_exchange_properties]]
=== Structured error exchange properties
When a Spring AI chat call fails, Camel sets structured metadata on the exchange **before** the Spring AI exception propagates. This works even when GenAI observability is disabled.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on code in PR #25899:
URL: https://github.com/apache/camel/pull/25899#discussion_r3889142674
##########
components/camel-ai/camel-huggingface/src/main/java/org/apache/camel/component/huggingface/tasks/AbstractTaskPredictor.java:
##########
@@ -107,6 +107,22 @@ public void setEndpoint(HuggingFaceEndpoint endpoint) {
protected abstract String getPythonScript();
+ /**
+ * Prepends the configured Hugging Face token to the generated handler as the {@code HF_TOKEN} environment variable
+ * so that every task can load gated or private models. {@code transformers.pipeline()} reads {@code HF_TOKEN} from
+ * the environment when no explicit token is passed; previously only the chat task passed a token, so the other
+ * tasks failed with 401 on gated models. The token comes from the {@code authToken} option or is resolved from an
+ * OAuth profile (see {@link org.apache.camel.component.huggingface.HuggingFaceProducer}), both surfaced through
+ * {@code config.getAuthToken()}.
+ */
+ protected String withAuthToken(String pythonScript) {
+ String authToken = config.getAuthToken();
+ if (authToken == null || authToken.isEmpty()) {
+ return pythonScript;
+ }
Review Comment:
Now that the token is applied centrally here via `HF_TOKEN`, the `tokenClause` in `ChatPredictor.getPythonScript()` (line 113) is redundant — chat gets the token through both the env var and the explicit `token='...'` kwarg. Consider removing the `tokenClause` logic from `ChatPredictor` and the `%s` placeholder from `chat.py` to match the PR's intent of centralizing token handling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on code in PR #25899:
URL: https://github.com/apache/camel/pull/25899#discussion_r3889386335
##########
components/camel-ai/camel-huggingface/src/main/java/org/apache/camel/component/huggingface/tasks/AbstractTaskPredictor.java:
##########
@@ -107,6 +107,22 @@ public void setEndpoint(HuggingFaceEndpoint endpoint) {
protected abstract String getPythonScript();
+ /**
+ * Prepends the configured Hugging Face token to the generated handler as the {@code HF_TOKEN} environment variable
+ * so that every task can load gated or private models. {@code transformers.pipeline()} reads {@code HF_TOKEN} from
+ * the environment when no explicit token is passed; previously only the chat task passed a token, so the other
+ * tasks failed with 401 on gated models. The token comes from the {@code authToken} option or is resolved from an
+ * OAuth profile (see {@link org.apache.camel.component.huggingface.HuggingFaceProducer}), both surfaced through
+ * {@code config.getAuthToken()}.
+ */
+ protected String withAuthToken(String pythonScript) {
+ String authToken = config.getAuthToken();
+ if (authToken == null || authToken.isEmpty()) {
+ return pythonScript;
+ }
+ return "import os\nos.environ['HF_TOKEN'] = '" + authToken + "'\n\n" + pythonScript;
Review Comment:
This builds the Python literal by naive string concatenation. If `authToken` (or a token resolved from an OAuth profile) contains a single quote, it breaks out of the string literal and corrupts — or injects into — a script that then gets executed by the Python engine. This mirrors the existing pattern in `ChatPredictor.getPythonScript()` (`", token='" + config.getAuthToken() + "'"`), so it's not new, but this is now the shared path for all 10 tasks, so it's the right place to fix it.
```suggestion
return "import os\nos.environ['HF_TOKEN'] = '" + authToken.replace("'", "\\'") + "'\n\n" + pythonScript;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on PR #25899:
URL: https://github.com/apache/camel/pull/25899#issuecomment-5478634634
Thanks — the DEBUG note was worth acting on rather than filing away, so I have taken it too.
You are right about the change in blast radius: that line only ran for chat before, and centralising the token made it apply to all ten tasks, with the token now sitting in the script it prints. Rather than redact the string, `loadModel()` now logs the script **before** `withAuthToken()` prepends the token, and writes the token-bearing version to the handler file:
```java
String pythonScript = getPythonScript();
// logged before the token is prepended: withAuthToken writes the configured token into the
// script, and this now runs for every task rather than only chat
if (LOG.isDebugEnabled()) {
LOG.debug("Generated Python script for task {}:\n{}", config.getTask(), pythonScript);
}
Files.writeString(handlerPath, withAuthToken(pythonScript));
```
The log keeps its diagnostic value — the token is one prepended line, not part of the script anyone is debugging — and there is no redaction to get wrong later.
The other two points from this round were already handled in the previous commit: the token is escaped for single quotes, and `ChatPredictor`'s `tokenClause` plus the `%s` placeholder in `chat.py` are gone, with a test asserting no `token=` survives in the generated script. @gnodet's empty-string coverage is in as well.
Module tests pass, and a full reactor build from the root is green with nothing left uncommitted.
_Claude Code on behalf of oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #25899:
URL: https://github.com/apache/camel/pull/25899
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25899:
URL: https://github.com/apache/camel/pull/25899#issuecomment-5468170580
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `components/camel-ai/camel-huggingface`
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 9 tested, 27 compile-only — current: 9 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **36 affected modules** (current approach: 9).
<details><summary>:warning: Modules only in Scalpel (27)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
**Skip-tests mode would test 9 modules** (1 direct + 8 downstream), **skip tests for 27** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (9)</summary>
- `camel-huggingface`
- `camel-jbang-mcp`
- `camel-jbang-plugin-mcp`
- `camel-jbang-plugin-route-parser`
- `camel-jbang-plugin-tui`
- `camel-jbang-plugin-validate`
- `camel-launcher-container`
- `camel-yaml-dsl-validator`
- `camel-yaml-dsl-validator-maven-plugin`
</details>
<details><summary>Modules with tests skipped (27)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
<details><summary>All tested modules (36 modules)</summary>
- Camel :: AI :: Hugging Face
- Camel :: All Components Sync point
- Camel :: Assembly
- Camel :: Catalog :: Camel Catalog
- Camel :: Catalog :: Camel Report Maven Plugin
- Camel :: Catalog :: Camel Route Parser
- Camel :: Catalog :: Console
- Camel :: Catalog :: Dummy Component
- Camel :: Catalog :: Maven
- Camel :: Catalog :: Suggest
- Camel :: Component DSL
- Camel :: Coverage
- Camel :: Docs
- Camel :: Endpoint DSL
- Camel :: Endpoint DSL :: Support
- Camel :: Integration Tests
- Camel :: JBang :: Core
- Camel :: JBang :: Integration tests
- Camel :: JBang :: MCP
- Camel :: JBang :: Main
- Camel :: JBang :: Plugin :: Edit
- Camel :: JBang :: Plugin :: Generate
- Camel :: JBang :: Plugin :: Kubernetes
- Camel :: JBang :: Plugin :: MCP
- Camel :: JBang :: Plugin :: Route Parser
- Camel :: JBang :: Plugin :: TUI
- Camel :: JBang :: Plugin :: Testing
- Camel :: JBang :: Plugin :: Validate
- Camel :: Kamelet Main
- Camel :: Launcher
- Camel :: Launcher :: Container
- Camel :: YAML DSL
- Camel :: YAML DSL :: Deserializers
- Camel :: YAML DSL :: Maven Plugins
- Camel :: YAML DSL :: Validator
- Camel :: YAML DSL :: Validator Maven Plugin
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/33305716502)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25899:
URL: https://github.com/apache/camel/pull/25899#issuecomment-5468054412
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24527: camel-huggingface - apply the configured token to every task, not only chat [camel]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on PR #25899:
URL: https://github.com/apache/camel/pull/25899#issuecomment-5475493455
Thanks @davsclaus and @gnodet — addressed in the latest commit:
1. **Token escaping** (davsclaus): the token is now interpolated as `authToken.replace("'", "\\'")`, so a single quote in the token (or an OAuth-resolved value) can no longer break out of / inject into the Python string literal. Since this is the shared path for all 10 tasks, it's fixed in `withAuthToken`.
2. **Redundant chat token clause** (gnodet): removed the `tokenClause` from `ChatPredictor.getPythonScript()` and the trailing placeholder from `chat.py`. The token now flows only through the central `HF_TOKEN` env var, so chat no longer passes it twice.
3. **Tests**: added the empty-token case and a single-quote escape case to `AuthTokenInjectionTest`, plus a new `ChatScriptFormatTest` guarding `chat.py`'s format-argument alignment after the placeholder was removed.
On the backport note: I'll open a `camel-4.22.x` backport once this merges (the `camel-huggingface` module doesn't exist on `camel-4.18.x`).
_Claude Code on behalf of oscerd_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org