You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@texera.apache.org by gi...@apache.org on 2026/07/28 21:19:34 UTC
(texera) 01/01: fix(workflow-operator): send provider-specific model id on HF text-gen chat fallback routes (#6972)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-6972-bfe2b535c031cda0508bae6a9a6d2b7f62a886c6
in repository https://gitbox.apache.org/repos/asf/texera.git
commit 3f58aceacdca6a4657c38b64d692ad2e8e6f592d
Author: Prateek Ganigi <91...@users.noreply.github.com>
AuthorDate: Tue Jul 28 14:19:11 2026 -0700
fix(workflow-operator): send provider-specific model id on HF text-gen chat fallback routes (#6972)
### What changes were proposed in this PR?
Fixes two related bugs in the HuggingFace inference operator's
provider-fallback loop (`_post_with_fallback` in the generated Python,
emitted by `PythonCodegenBase.scala`) that break the `text-generation` /
`image-text-to-text` path, which is the operator's default task.
1. **Wrong model id on provider-scoped chat routes.** The chat branch
posted the same payload, carrying `"model": self.MODEL_ID` (the HF Hub
ID), to every provider's route (`router.huggingface.co/{provider}/...`).
Provider-scoped routes require the provider's *own* model name
(`providerId`, e.g. the Hub's `Qwen/Qwen2.5-72B-Instruct` is a different
string on Groq or Fireworks), which `_resolve_providers` already fetches
and `_call_provider` already uses for the other task families. Providers
whose internal name differs from the Hub ID rejected every request with
400/404, so fallback only succeeded when the names happened to match.
The chat branch now posts a per-attempt copy with the model overridden:
chat_payload = {**pipeline_payload, "model": provider_id}
(a copy, not in-place mutation, pipeline_payload is reused for the next
provider
attempt).
2. **Malformed hf-inference chat URL.** The chat URL for hf-inference
was built as `hf-inference/v1/chat/completions`, but hf-inference
expects the model in the URL path:
`hf-inference/models/{model-id}/v1/chat/completions` (the form this file
already uses for pipeline tasks). Since hf-inference sorts first in
`PROVIDER_COST_PRIORITY`, the cheapest provider failed on every row,
wasting a doomed request (up to the 120 s timeout) per row before
falling through.
Net effect: text-gen rows now succeed on the cheapest live provider
instead of failing with "All inference providers failed" or silently
drifting to pricier providers. No behavior change for other task
families - `_call_provider` already handled them correctly.
### Any related issues, documentation, discussions?
Closes #6965
### How was this PR tested?
Added a generated-code test to HuggingFaceInferenceOpDescSpec ("send the
provider-specific model id on provider-scoped chat routes") pinning both
fixes: the corrected hf-inference chat URL, and the chat branch posting
the per-provider payload copy (asserted as an anchored two-line block so
pipeline routes that legitimately post pipeline_payload directly stay
unaffected).
Run:
sbt "WorkflowOperator/testOnly
org.apache.texera.amber.operator.huggingFace.*"
Full HF package passes (123 tests, 12 suites), including
PythonCodeRawInvalidTextSpec, which py_compiles the generated Python of
all 117 Python operators, verifying the edited template still emits
syntactically valid Python. scalafmt clean.
### Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Fable 5 in compliance with ASF.
---
.../huggingFace/codegen/PythonCodegenBase.scala | 15 ++++++++++++---
.../huggingFace/HuggingFaceInferenceOpDescSpec.scala | 17 +++++++++++++++++
2 files changed, 29 insertions(+), 3 deletions(-)
diff --git a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala
index 7cd305bfb6..f2cf479afb 100644
--- a/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala
+++ b/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala
@@ -201,9 +201,18 @@ object PythonCodegenBase {
| prov_task = prov.get("task", "")
| try:
| if self.TASK in ("text-generation", "image-text-to-text"):
- | route = self.CHAT_ROUTES.get(provider_name, "v1/chat/completions")
- | url = f"https://router.huggingface.co/{provider_name}/{route}"
- | resp = requests.post(url, headers=json_headers, json=pipeline_payload, timeout=120)
+ | if provider_name == "hf-inference":
+ | # hf-inference expects the model in the URL path, like the
+ | # pipeline route below.
+ | url = f"https://router.huggingface.co/hf-inference/models/{self.MODEL_ID}/v1/chat/completions"
+ | else:
+ | route = self.CHAT_ROUTES.get(provider_name, "v1/chat/completions")
+ | url = f"https://router.huggingface.co/{provider_name}/{route}"
+ | # Provider-scoped routes need the provider's own model name
+ | # (providerId), not the HF Hub ID. Copy instead of mutating:
+ | # pipeline_payload is reused for the next provider attempt.
+ | chat_payload = {**pipeline_payload, "model": provider_id}
+ | resp = requests.post(url, headers=json_headers, json=chat_payload, timeout=120)
| elif is_model_author and prov_task in ("image-to-text", "image-text-to-text") and provider_name not in ("zai-org",):
| url = f"https://router.huggingface.co/{provider_name}/v1/chat/completions"
| img_b64 = ""
diff --git a/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala b/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala
index 83f6239903..97bc9dc8ff 100644
--- a/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala
+++ b/common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala
@@ -113,6 +113,23 @@ class HuggingFaceInferenceOpDescSpec extends AnyFlatSpec with Matchers {
code should include("""body["choices"][0]["message"]["content"]""")
}
+ it should "send the provider-specific model id on provider-scoped chat routes" in {
+ val code = makeDesc().generatePythonCode()
+ // hf-inference's chat-completions endpoint carries the model in the URL path.
+ code should include(
+ "https://router.huggingface.co/hf-inference/models/{self.MODEL_ID}/v1/chat/completions"
+ )
+ // The chat branch posts a per-provider copy of the payload with the
+ // provider's own model name (providerId) — the Hub ID is only valid on
+ // hf-inference itself. Matched as one block so the assertion is anchored
+ // to the chat branch: pipeline routes elsewhere legitimately post the
+ // shared pipeline_payload directly.
+ code should include(
+ "chat_payload = {**pipeline_payload, \"model\": provider_id}\n" +
+ " resp = requests.post(url, headers=json_headers, json=chat_payload, timeout=120)"
+ )
+ }
+
it should
"emit a runtime check that rejects malformed MODEL_ID values before any HF URL is built" in {
val code = makeDesc().generatePythonCode()