You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "anishshiva7 (via GitHub)" <gi...@apache.org> on 2026/06/08 22:58:23 UTC
[PR] feat(huggingface): add qa and ranking tasks [texera]
anishshiva7 opened a new pull request, #5574:
URL: https://github.com/apache/texera/pull/5574
⚠️ This PR is stacked on `hf/04-audio-mediagen`. Until that lands, the diff below may also include earlier HuggingFace task-family changes depending on which base GitHub is showing. The new code in this PR is `codegen/QaRankingCodegen.scala`, the QA/ranking-related additions to `codegen/PythonCodegenBase.scala`, the new QA/ranking fields on `HuggingFaceInferenceOpDesc.scala`, and the QA/ranking task tests in `HuggingFaceInferenceOpDescSpec.scala`. Once PR 4 merges and this PR is retargeted to `main`, the diff should auto-clean to the PR 5 QA/ranking changes only.
What changes were proposed in this PR?
--------------------------------------
Adds the QA/ranking/classification task family --- 5 HF pipeline tasks --- as a new `TaskCodegen` plugged into the dispatcher established by the text-generation PR:
QA tasks: `question-answering`, `table-question-answering`
classification/ranking tasks: `zero-shot-classification`, `sentence-similarity`, `text-ranking`
`codegen/QaRankingCodegen.scala` supplies the per-task payload + parse Python branches for all 5 tasks.
`CodegenContext` is extended with `contextColumn`, `candidateLabels`, and `sentencesColumn` (`EncodableString`).
`HuggingFaceInferenceOpDesc.scala` gains 3 new `@JsonProperty` fields and registers `QaRankingCodegen` in the dispatcher.
`PythonCodegenBase.scala` grows to host the shared QA/ranking infrastructure:
- Per-row validation for the new column-named fields.
- `question-answering` payload handling with prompt + context.
- `table-question-answering` payload handling with table data.
- `zero-shot-classification` payload handling with candidate labels.
- `sentence-similarity` and `text-ranking` payload handling with sentence inputs.
- Response parsing for QA/ranking outputs.
User-input strings continue to flow through `pyb"..."` + `EncodableString` so they reach Python as `self.decode_python_template('<base64>')` rather than raw literals. `PythonCodeRawInvalidTextSpec` still passes with 117/117 descriptors py_compile cleanly.
Any related issues, documentation, or discussions?
--------------------------------------------------
Tracking issue: Add HuggingFace question answering and ranking tasks [apache#5292](https://github.com/apache/texera/issues/5292)
Closes Add HuggingFace question answering and ranking tasks [apache#5292](https://github.com/apache/texera/issues/5292)
Stacked on: PR 4 audio/media generation tasks / `hf/04-audio-mediagen`
Parent issue: Add Hugging Face inference operator [apache#5041](https://github.com/apache/texera/issues/5041)
Closed sibling issue: Add HuggingFaceModelResource REST endpoints for HF operator UI [apache#5134](https://github.com/apache/texera/issues/5134)
How was this PR tested?
-----------------------
`sbt "WorkflowOperator/compile; WorkflowOperator/Test/compile"` clean.
`sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.huggingFace.HuggingFaceInferenceOpDescSpec org.apache.texera.amber.util.PythonCodeRawInvalidTextSpec"` --- 31 focused tests pass, including HuggingFace QA/ranking task coverage and the raw Python descriptor scan.
`sbt "WorkflowOperator / scalafmtCheck"` clean.
`sbt "WorkflowOperator / Test / scalafmtCheck"` clean.
`PythonCodeRawInvalidTextSpec` --- 117/117 descriptors py_compile cleanly with the new operator code paths, no marker leaks.
Was this PR authored or co-authored using generative AI tooling?
----------------------------------------------------------------
Yes, co-authored with generative AI tooling (Codex).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #5574:
URL: https://github.com/apache/texera/pull/5574#discussion_r3444858228
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala:
##########
@@ -463,12 +491,29 @@ object PythonCodegenBase {
| # --- resolve all available inference providers for this model (tried in order) ---
| providers = self._resolve_providers(token)
|
- | # --- validate prompt column exists (required for non-image tasks) ---
- | if task not in image_tasks:
+ | # --- validate prompt column exists (skipped for image tasks and binary-only audio tasks) ---
+ | if task not in image_tasks and task not in audio_only_tasks:
| assert prompt_col in table.columns, (
| f"Prompt column '{prompt_col}' not found in input table. "
| f"Available columns: {list(table.columns)}"
| )
+ | if task == "zero-shot-classification":
+ | assert self.CANDIDATE_LABELS and self.CANDIDATE_LABELS.strip(), (
+ | "Candidate Labels are required for zero-shot-classification. "
+ | "Provide a comma-separated list of labels."
+ | )
Review Comment:
For zero-shot-classification, this validation only checks that `CANDIDATE_LABELS` is a non-empty string. Inputs like `",,,"` or whitespace will pass validation but produce an empty `labels` list at runtime, which will fail downstream in a less clear way. Validate that at least one non-empty label exists after splitting/trimming.
##########
amber/src/main/scala/org/apache/texera/web/auth/UserAuthenticator.scala:
##########
@@ -35,7 +35,7 @@ import java.util.Optional
object UserAuthenticator extends Authenticator[JwtContext, SessionUser] with LazyLogging {
override def authenticate(context: JwtContext): Optional[SessionUser] = {
try {
- Optional.of(JwtParser.claimsToSessionUser(context.getJwtClaims))
+ JwtParser.claimsToOptionalSessionUser(context.getJwtClaims)
Review Comment:
The scaladoc still references `JwtParser.claimsToSessionUser`, but the implementation now delegates to `claimsToOptionalSessionUser`. Update the comment so it reflects the new behavior (and the fact that missing/malformed claims can return `Optional.empty()`).
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala:
##########
@@ -831,12 +960,12 @@ object PythonCodegenBase {
| if not content_type or content_type == "application/octet-stream":
| from urllib.parse import urlparse as _urlparse
| ext = os.path.splitext(_urlparse(url).path.lower())[1]
- | mime_map = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".gif": "image/gif", ".webp": "image/webp", ".svg": "image/svg+xml", ".mp4": "video/mp4", ".webm": "video/webm"}
+ | mime_map = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".gif": "image/gif", ".webp": "image/webp", ".svg": "image/svg+xml", ".mp3": "audio/mpeg", ".mpeg": "audio/mpeg", ".wav": "audio/wav", ".flac": "audio/flac", ".ogg": "audio/ogg", ".oga": "audio/ogg", ".m4a": "audio/mp4", ".mp4": "video/mp4", ".webm": "video/webm"}
Review Comment:
The `.m4a` MIME guess is inconsistent: `_get_audio_content_type` maps `.m4a` to `audio/m4a`, but `_url_to_data_url` maps `.m4a` to `audio/mp4`. This leads to different MIME strings depending on which path produces the data URL. Pick one mapping (ideally consistent with the rest of the codebase) and use it in both places.
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala:
##########
@@ -821,6 +928,28 @@ object PythonCodegenBase {
| return text[start_pos:pos], pos
| return None, start_pos
|
+ | def _get_audio_content_type(self):
+ | audio_input = str(self.AUDIO_INPUT or "").strip().lower()
+ | if audio_input.startswith("data:"):
+ | header = audio_input.split(",", 1)[0]
+ | if ";" in header:
+ | return header[5:header.index(";")]
+ | return header[5:]
+ | extension_map = {
+ | ".mp3": "audio/mpeg",
+ | ".mpeg": "audio/mpeg",
+ | ".wav": "audio/wav",
+ | ".flac": "audio/flac",
+ | ".ogg": "audio/ogg",
+ | ".oga": "audio/ogg",
+ | ".webm": "audio/webm",
+ | ".opus": "audio/webm;codecs=opus",
+ | ".amr": "audio/amr",
+ | ".m4a": "audio/m4a",
+ | }
+ | _, ext = os.path.splitext(audio_input)
+ | return extension_map.get(ext, "audio/mpeg")
Review Comment:
`_get_audio_content_type` uses `os.path.splitext(audio_input)` directly. When `AUDIO_INPUT` is a URL with query params (common for signed URLs), this yields an extension like `.mp3?X-Amz-...` and the lookup falls back to `audio/mpeg` even for non-mpeg audio. Parse the URL path before splitting the extension (similar to `_url_to_data_url`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on code in PR #5574:
URL: https://github.com/apache/texera/pull/5574#discussion_r3383896106
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegen.scala:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.huggingFace.codegen
+
+/**
+ * Codegen for question-answering, zero-shot, similarity, and ranking tasks.
+ *
+ * These tasks are prompt-driven but need extra per-row or per-operator
+ * inputs: context text, candidate labels, table contents, or a list of
+ * comparison sentences/documents.
+ */
+object QaRankingCodegen extends TaskCodegen {
+
+ override val task: String = "question-answering"
+
+ override val tasks: Set[String] = Set(
+ "question-answering",
+ "table-question-answering",
+ "zero-shot-classification",
+ "sentence-similarity",
+ "text-ranking"
+ )
+
+ override def payloadPython(ctx: CodegenContext): String =
+ """ if task == "question-answering":
+ | ctx_val = row[self.CONTEXT_COLUMN]
+ | ctx_val = "" if pd.isna(ctx_val) else str(ctx_val)
+ | payload = {"inputs": {"question": prompt_value, "context": ctx_val}}
+ | elif task == "table-question-answering":
+ | payload = {"inputs": {"query": prompt_value, "table": table_dict}}
+ | elif task == "zero-shot-classification":
+ | labels = [l.strip() for l in self.CANDIDATE_LABELS.split(",") if l.strip()]
+ | payload = {
+ | "inputs": prompt_value,
+ | "parameters": {"candidate_labels": labels},
+ | }
+ | elif task in ("sentence-similarity", "text-ranking"):
+ | sent_val = row[self.SENTENCES_COLUMN]
+ | sent_val = "" if pd.isna(sent_val) else str(sent_val)
+ | sentences_list = [s.strip() for s in sent_val.split(",") if s.strip()]
+ | payload = {
+ | "inputs": {
+ | "source_sentence": prompt_value,
+ | "sentences": sentences_list,
+ | }
+ | }
+ | else:
+ | payload = {"inputs": prompt_value}""".stripMargin
+
+ override def parsePython(ctx: CodegenContext): String =
+ """ if task == "question-answering":
+ | return body.get("answer", json.dumps(body))
+ | elif task == "table-question-answering":
+ | return body.get("answer", json.dumps(body))
Review Comment:
`body.get(...)` assumes `body` is a dict, but some providers return a list for QA. On a non-dict, `.get` raises `AttributeError`, which is not caught by `_parse_response`'s `except (KeyError, IndexError, TypeError)`, so instead of the intended `json.dumps(body)` fallback it surfaces as a misleading "Request failed" for that row. Guard with `isinstance`:
```suggestion
""" if task == "question-answering":
| return body.get("answer", json.dumps(body)) if isinstance(body, dict) else json.dumps(body)
| elif task == "table-question-answering":
| return body.get("answer", json.dumps(body)) if isinstance(body, dict) else json.dumps(body)
```
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala:
##########
@@ -0,0 +1,897 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.huggingFace.codegen
+
+import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext
+
+/**
+ * Builds the Python script emitted by HuggingFaceInferenceOpDesc.
+ *
+ * The script defines a `ProcessTableOperator` class with:
+ * - Per-instance configuration set in `open(self)` from base64-encoded
+ * values that the `pyb"..."` macro decodes at runtime (so user-input
+ * strings never appear as raw Python literals in the source).
+ * - A provider-fallback system that walks the HF Hub's inference-provider
+ * list cheapest-first and tries each provider's native chat-completions
+ * route, with HF Inference Router as the default.
+ * - A `process_table` loop that validates the prompt column, builds the
+ * per-row payload via the per-task codegen, posts to the resolved
+ * provider, and parses the response.
+ * - A `_parse_response` task switch whose branches are provided by the
+ * per-task codegen.
+ *
+ * Per-task variation lives in `TaskCodegen` implementations. This class
+ * holds only what is shared across all HF tasks; per-task helpers (image
+ * loading, audio MIME inference, media-URL fetching, etc.) will be added
+ * in subsequent PRs as the corresponding task families land.
+ */
+object PythonCodegenBase {
+
+ def render(ctx: CodegenContext, codegen: TaskCodegen): String = {
+ val payload = codegen.payloadPython(ctx)
+ val parse = codegen.parsePython(ctx)
+ val hfApiToken = ctx.hfApiToken
+ val modelId = ctx.modelId
+ val promptColumn = ctx.promptColumn
+ val resultColumn = ctx.resultColumn
+ val task = ctx.task
+ val systemPrompt = ctx.systemPrompt
+ val maxNewTokens = ctx.safeMaxTokens
+ val temperature = ctx.safeTemp
+ val imageInput = ctx.imageInput
+ val inputImageColumn = ctx.inputImageColumn
+ val audioInput = ctx.audioInput
+ val inputAudioColumn = ctx.inputAudioColumn
+ val contextColumn = ctx.contextColumn
+ val candidateLabels = ctx.candidateLabels
+ val sentencesColumn = ctx.sentencesColumn
+ pyb"""import os
+ |import re
+ |import json
+ |import base64
+ |import requests
+ |import pandas as pd
+ |from urllib.parse import urlparse
+ |from pytexera import *
+ |
+ |# Defensive format check for MODEL_ID before it is interpolated into
+ |# HF URL paths. The base host is hardcoded so the worst case isn't
+ |# SSRF, but rejecting `..` segments / query strings / fragments /
+ |# control chars keeps the operator's request shape predictable.
+ |_HF_MODEL_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*(/[A-Za-z0-9._-]+)+$$")
+ |
+ |class ProcessTableOperator(UDFTableOperator):
+ |
+ | # Providers ranked cheapest-first (lower index = cheaper).
+ | # Unknown providers are appended at the end.
+ | PROVIDER_COST_PRIORITY = [
+ | "hf-inference",
+ | "cerebras",
+ | "sambanova",
+ | "groq",
+ | "novita",
+ | "nebius",
+ | "fireworks-ai",
+ | "together",
+ | "hyperbolic",
+ | "scaleway",
+ | "nscale",
+ | "ovhcloud",
+ | "deepinfra",
+ | "featherless-ai",
+ | "baseten",
+ | "publicai",
+ | "nvidia",
+ | "openai",
+ | "cohere",
+ | "clarifai",
+ | ]
+ |
+ | # Per-provider chat-completions route overrides. Providers not listed
+ | # here use the default `v1/chat/completions` path. Single source of
+ | # truth for both _post_with_fallback (text-gen) and _call_provider
+ | # (OpenAI-compatible fallback) so the two stay in sync as providers
+ | # are added.
+ | CHAT_ROUTES = {
+ | "groq": "openai/v1/chat/completions",
+ | "fireworks-ai": "inference/v1/chat/completions",
+ | "cohere": "compatibility/v1/chat/completions",
+ | "clarifai": "v2/ext/openai/v1/chat/completions",
+ | "deepinfra": "v1/openai/chat/completions",
+ | }
+ |
+ | # Third-party providers that speak the OpenAI chat-completions
+ | # protocol. Used by _call_provider's OpenAI-compatible branch.
+ | OPENAI_COMPATIBLE_PROVIDERS = (
+ | "cerebras", "sambanova", "groq", "novita", "nebius",
+ | "fireworks-ai", "together", "hyperbolic", "cohere", "clarifai",
+ | "deepinfra", "featherless-ai", "nscale", "nvidia", "openai",
+ | "ovhcloud", "publicai", "scaleway", "baseten",
+ | )
+ |
+ | def open(self):
+ | # User-provided strings reach the operator via base64-encoded
+ | # decode expressions so they cannot break Python syntax or
+ | # leak raw text into the generated source.
+ | self.HF_API_TOKEN = $hfApiToken
+ | self.MODEL_ID = $modelId
+ | self.PROMPT_COLUMN = $promptColumn
+ | self.RESULT_COLUMN = $resultColumn
+ | self.TASK = $task
+ | self.SYSTEM_PROMPT = $systemPrompt
+ | self.MAX_NEW_TOKENS = $maxNewTokens
+ | self.TEMPERATURE = $temperature
+ | self.IMAGE_INPUT = $imageInput
+ | self.INPUT_IMAGE_COLUMN = $inputImageColumn
+ | self.AUDIO_INPUT = $audioInput
+ | self.INPUT_AUDIO_COLUMN = $inputAudioColumn
+ | self.CONTEXT_COLUMN = $contextColumn
+ | self.CANDIDATE_LABELS = $candidateLabels
+ | self.SENTENCES_COLUMN = $sentencesColumn
+ |
+ | def _resolve_providers(self, token):
+ | '''Query the HF Hub API for inference providers serving this model.
+ | Returns a list of dicts with 'name' and 'providerId' sorted
+ | cheapest-first. Falls back to hf-inference if anything goes wrong.
+ | '''
+ | try:
+ | resp = requests.get(
+ | f"https://huggingface.co/api/models/{self.MODEL_ID}",
+ | headers={"Authorization": f"Bearer {token}"},
+ | params={"expand[]": "inferenceProviderMapping"},
+ | timeout=30,
+ | )
+ | if resp.status_code == 200:
+ | data = resp.json()
+ | mapping = (
+ | data.get("inferenceProviderMapping")
+ | or data.get("inference_provider_mapping")
+ | or {}
+ | )
+ | if mapping:
+ | live = [
+ | {
+ | "name": p,
+ | "providerId": v.get("providerId", self.MODEL_ID),
+ | "task": v.get("task", ""),
+ | "isModelAuthor": v.get("isModelAuthor", False),
+ | }
+ | for p, v in mapping.items()
+ | if isinstance(v, dict) and v.get("status") == "live"
+ | ]
+ | if live:
+ | priority = {name: idx for idx, name in enumerate(self.PROVIDER_COST_PRIORITY)}
+ | live.sort(key=lambda prov: priority.get(prov["name"], len(self.PROVIDER_COST_PRIORITY)))
+ | return live
+ | except Exception:
+ | pass
+ | return [{"name": "hf-inference", "providerId": self.MODEL_ID}]
+ |
+ | def _post_with_fallback(self, providers, json_headers, raw_binary_headers, pipeline_payload, use_raw_binary_body, prompt_value):
+ | '''Try providers in order, using the correct API route for each.
+ | Returns (response, provider_summary). provider_summary is None on
+ | success or a string describing what failed.
+ | '''
+ | RETRYABLE = (400, 404, 422, 429, 502, 503)
+ | last_resp = None
+ | errors = []
+ | for prov in providers:
+ | provider_name = prov["name"]
+ | provider_id = prov["providerId"]
+ | is_model_author = prov.get("isModelAuthor", False)
+ | prov_task = prov.get("task", "")
+ | try:
+ | if self.TASK in ("text-generation", "image-text-to-text"):
+ | route = self.CHAT_ROUTES.get(provider_name, "v1/chat/completions")
+ | url = f"https://router.huggingface.co/{provider_name}/{route}"
+ | resp = requests.post(url, headers=json_headers, json=pipeline_payload, timeout=120)
+ | elif is_model_author and prov_task in ("image-to-text", "image-text-to-text") and provider_name not in ("zai-org",):
+ | url = f"https://router.huggingface.co/{provider_name}/v1/chat/completions"
+ | img_b64 = ""
+ | if use_raw_binary_body and isinstance(pipeline_payload, bytes):
+ | img_b64 = base64.b64encode(pipeline_payload).decode("utf-8")
+ | chat_payload = {
+ | "model": provider_id,
+ | "messages": [{
+ | "role": "user",
+ | "content": [
+ | {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}} if img_b64 else None,
+ | {"type": "text", "text": prompt_value if prompt_value else "What is in this image?"},
+ | ],
+ | }],
+ | }
+ | chat_payload["messages"][0]["content"] = [c for c in chat_payload["messages"][0]["content"] if c is not None]
+ | resp = requests.post(url, headers=json_headers, json=chat_payload, timeout=120)
+ | elif provider_name == "hf-inference":
+ | url = f"https://router.huggingface.co/hf-inference/models/{self.MODEL_ID}"
+ | if use_raw_binary_body:
+ | resp = requests.post(url, headers=raw_binary_headers, data=pipeline_payload, timeout=120)
+ | else:
+ | resp = requests.post(url, headers=json_headers, json=pipeline_payload, timeout=120)
+ | else:
+ | resp = self._call_provider(provider_name, provider_id, json_headers, raw_binary_headers, pipeline_payload, use_raw_binary_body, prompt_value)
+ | except Exception as e:
+ | errors.append(f"{provider_name}: {type(e).__name__}")
+ | continue
+ | if resp.status_code in (200, 201):
+ | return resp, None
+ | if resp.status_code == 401:
+ | return resp, None
+ | try:
+ | detail = resp.json().get("error", resp.text[:200])
+ | except Exception:
+ | detail = resp.text[:200] if resp.text else "no details"
+ | errors.append(f"{provider_name}: HTTP {resp.status_code} - {detail}")
+ | last_resp = resp
+ | if resp.status_code not in RETRYABLE:
+ | return resp, "; ".join(errors)
+ | summary = "; ".join(errors) if errors else "no providers available"
+ | return last_resp, summary
+ |
+ | def _call_provider(self, provider_name, provider_id, json_headers, raw_binary_headers, pipeline_payload, use_raw_binary_body, prompt_value):
+ | '''Route to a third-party provider using its native API format.
+ | Handles OpenAI-compatible chat providers for text-gen, zai-org's
+ | custom API, Replicate / Fal-ai / Wavespeed for media-generation
+ | and image-to-image, and an unknown-provider fallback that tries
+ | the pipeline format then chat completions.
+ | '''
+ | base = f"https://router.huggingface.co/{provider_name}"
+ | task = self.TASK
+ | img_b64 = ""
+ | if use_raw_binary_body and isinstance(pipeline_payload, bytes):
+ | img_b64 = base64.b64encode(pipeline_payload).decode("utf-8")
+ |
+ | # zai-org: custom /api/paas/v4/ surface.
+ | if provider_name == "zai-org":
+ | zai_headers = {**json_headers, "x-source-channel": "hugging_face", "accept-language": "en-US,en"}
+ | if task in ("image-to-text", "image-text-to-text"):
+ | url = f"{base}/api/paas/v4/layout_parsing"
+ | file_data = f"data:image/png;base64,{img_b64}" if img_b64 else ""
+ | return requests.post(url, headers=zai_headers, json={"model": provider_id, "file": file_data}, timeout=120)
+ | url = f"{base}/api/paas/v4/chat/completions"
+ | messages = [{"role": "user", "content": prompt_value}]
+ | if img_b64:
+ | messages = [{"role": "user", "content": [
+ | {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
+ | {"type": "text", "text": prompt_value if prompt_value else "What is in this image?"},
+ | ]}]
+ | return requests.post(url, headers=zai_headers, json={"model": provider_id, "messages": messages}, timeout=120)
+ |
+ | # Replicate: synchronous predictions endpoint with polling fallback.
+ | if provider_name == "replicate":
+ | url = f"{base}/v1/models/{provider_id}/predictions"
+ | hdrs = {**json_headers, "Prefer": "wait"}
+ | if task == "text-to-speech":
+ | inp = {"text": prompt_value}
+ | elif task in ("text-to-image", "text-to-video"):
+ | inp = {"prompt": prompt_value}
+ | elif task == "automatic-speech-recognition" and img_b64:
+ | inp = {"audio": f"data:audio/wav;base64,{img_b64}"}
+ | elif task == "image-to-image" and img_b64:
+ | data_url = f"data:image/png;base64,{img_b64}"
+ | inp = {"image": data_url, "images": [data_url], "input_image": data_url, "prompt": prompt_value}
+ | elif img_b64:
+ | inp = {"image": f"data:image/png;base64,{img_b64}", "prompt": prompt_value}
+ | else:
+ | inp = {"prompt": prompt_value}
+ | resp = requests.post(url, headers=hdrs, json={"input": inp}, timeout=120)
+ | if resp.status_code == 202:
+ | import time as _time
+ | pred = resp.json()
+ | poll_url = pred.get("urls", {}).get("get", "")
+ | if not poll_url:
+ | return resp
+ | from urllib.parse import urlparse as _urlparse
+ | poll_path = _urlparse(poll_url).path
+ | poll_url = f"{base}{poll_path}"
+ | # Worst case: 300 polls × 2s = ~10 minutes per row before we give
+ | # up. Sized for text-to-video which legitimately takes minutes on
+ | # Replicate. process_table is synchronous, so emit a progress
+ | # line every 30 polls (~1 min) to distinguish slow work from a
+ | # hang in the worker log.
+ | for poll_idx in range(300):
+ | _time.sleep(2)
+ | poll_resp = requests.get(poll_url, headers=json_headers, timeout=30)
+ | if poll_resp.status_code != 200:
+ | continue
+ | status = poll_resp.json().get("status", "")
+ | if status in ("succeeded", "failed", "canceled"):
+ | return poll_resp
+ | if (poll_idx + 1) % 30 == 0:
+ | print(f"[hf] Replicate still running for model '{self.MODEL_ID}' after {(poll_idx + 1) * 2}s; will wait up to 600s.")
+ | return poll_resp
+ | return resp
+ |
+ | # Fal-ai: per-model endpoint.
+ | if provider_name == "fal-ai":
+ | url = f"{base}/{provider_id}"
+ | if task == "text-to-speech":
+ | return requests.post(url, headers=json_headers, json={"text": prompt_value}, timeout=120)
+ | if task in ("text-to-image", "text-to-video"):
+ | return requests.post(url, headers=json_headers, json={"prompt": prompt_value}, timeout=120)
+ | if task == "image-to-image" and img_b64:
+ | data_url = f"data:image/png;base64,{img_b64}"
+ | return requests.post(url, headers=json_headers, json={"image_url": data_url, "image_urls": [data_url], "prompt": prompt_value}, timeout=120)
+ | if img_b64:
+ | return requests.post(url, headers=json_headers, json={"image_url": f"data:image/png;base64,{img_b64}", "prompt": prompt_value}, timeout=120)
+ | return requests.post(url, headers=json_headers, json={"prompt": prompt_value}, timeout=120)
+ |
+ | # Wavespeed: async submit + poll.
+ | if provider_name == "wavespeed":
+ | url = f"{base}/api/v3/{provider_id}"
+ | payload = {"prompt": prompt_value}
+ | if img_b64:
+ | payload["image"] = img_b64
+ | payload["images"] = [img_b64]
+ | submit_resp = requests.post(url, headers=json_headers, json=payload, timeout=120)
+ | if submit_resp.status_code not in (200, 201):
+ | return submit_resp
+ | get_path = submit_resp.json().get("data", {}).get("urls", {}).get("get", "")
+ | if not get_path:
+ | return submit_resp
+ | from urllib.parse import urlparse as _urlparse
+ | result_url = f"{base}{_urlparse(get_path).path}"
+ | import time as _time
+ | poll_resp = submit_resp
+ | # Worst case: 120 polls × 1s = ~2 minutes per row. Emit a progress
+ | # line every 30 polls (~30 s) so the worker log distinguishes slow
+ | # work from a hang.
+ | for poll_idx in range(120):
+ | _time.sleep(1)
+ | poll_resp = requests.get(result_url, headers=json_headers, timeout=30)
+ | if poll_resp.status_code != 200:
+ | continue
+ | status = poll_resp.json().get("data", {}).get("status", "")
+ | if status in ("completed", "failed"):
+ | return poll_resp
+ | if (poll_idx + 1) % 30 == 0:
+ | print(f"[hf] Wavespeed still running for model '{self.MODEL_ID}' after {poll_idx + 1}s; will wait up to 120s.")
+ | return poll_resp
+ |
+ | if provider_name in self.OPENAI_COMPATIBLE_PROVIDERS:
+ | if task == "text-to-image":
+ | url = f"{base}/v1/images/generations"
+ | return requests.post(url, headers=json_headers, json={"model": provider_id, "prompt": prompt_value}, timeout=120)
+ | if task == "text-to-speech":
+ | url = f"{base}/v1/audio/speech"
+ | return requests.post(url, headers=json_headers, json={"model": provider_id, "input": prompt_value}, timeout=120)
+ | url = f"{base}/{self.CHAT_ROUTES.get(provider_name, 'v1/chat/completions')}"
+ | messages = [{"role": "user", "content": prompt_value}]
+ | if img_b64:
+ | messages = [{"role": "user", "content": [
+ | {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
+ | {"type": "text", "text": prompt_value if prompt_value else "What is in this image?"},
+ | ]}]
+ | return requests.post(
+ | url,
+ | headers=json_headers,
+ | json={"model": provider_id, "messages": messages},
+ | timeout=120,
+ | )
+ |
+ | # Unknown provider: try pipeline format, then chat completions.
+ | url = f"{base}/{provider_id}"
+ | if use_raw_binary_body:
+ | resp = requests.post(url, headers=raw_binary_headers, data=pipeline_payload, timeout=120)
+ | else:
+ | resp = requests.post(url, headers=json_headers, json=pipeline_payload, timeout=120)
+ | if resp.status_code in (400, 404, 422):
+ | url = f"{base}/v1/chat/completions"
+ | messages = [{"role": "user", "content": prompt_value}]
+ | if img_b64:
+ | messages = [{"role": "user", "content": [
+ | {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
+ | {"type": "text", "text": prompt_value if prompt_value else "Describe this image."},
+ | ]}]
+ | resp2 = requests.post(
+ | url,
+ | headers=json_headers,
+ | json={"model": provider_id, "messages": messages},
+ | timeout=120,
+ | )
+ | if resp2.status_code == 200:
+ | return resp2
+ | return resp
+ |
+ | @overrides
+ | def process_table(self, table: Table, port: int) -> Iterator[Optional[TableLike]]:
+ | prompt_col = self.PROMPT_COLUMN
+ | result_col = self.RESULT_COLUMN
+ | task = self.TASK
+ | image_only_tasks = ("image-classification", "object-detection", "image-segmentation", "image-to-text")
+ | image_prompt_tasks = ("visual-question-answering", "document-question-answering", "zero-shot-image-classification", "image-text-to-text", "image-to-image")
+ | image_tasks = image_only_tasks + image_prompt_tasks
+ | audio_only_tasks = ("automatic-speech-recognition", "audio-classification")
+ |
+ | # --- validate MODEL_ID format before any HF URL is built ---
+ | if not _HF_MODEL_ID_PATTERN.match(self.MODEL_ID or ""):
+ | raise ValueError(
+ | f"Invalid Hugging Face model ID '{self.MODEL_ID}'. "
+ | f"Expected format like 'org/model-name' or 'org/model-name/revision'."
+ | )
+ |
+ | # --- resolve API token ---
+ | token = self.HF_API_TOKEN if self.HF_API_TOKEN else os.environ.get("HF_TOKEN", "")
+ | if not token:
+ | raise ValueError(
+ | "Hugging Face API token is not set. "
+ | "Provide it in the operator config or via HF_TOKEN env var."
+ | )
+ |
+ | # --- resolve all available inference providers for this model (tried in order) ---
+ | providers = self._resolve_providers(token)
+ |
+ | # --- validate prompt column exists (skipped for binary-only tasks) ---
+ | if task not in image_only_tasks and task not in audio_only_tasks:
+ | assert prompt_col in table.columns, (
+ | f"Prompt column '{prompt_col}' not found in input table. "
+ | f"Available columns: {list(table.columns)}"
+ | )
+ | if task == "question-answering":
Review Comment:
`zero-shot-classification` has no check that `CANDIDATE_LABELS` is non-empty, so an empty value sends `candidate_labels: []` and produces a confusing provider-side error instead of an actionable one (unlike the context and sentences checks here). Add an upfront assert:
```suggestion
| if task == "zero-shot-classification":
| assert self.CANDIDATE_LABELS and self.CANDIDATE_LABELS.strip(), (
| "Candidate Labels are required for zero-shot-classification. "
| "Provide a comma-separated list of labels."
| )
| if task == "question-answering":
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on code in PR #5574:
URL: https://github.com/apache/texera/pull/5574#discussion_r3447821350
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegen.scala:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.huggingFace.codegen
+
+/**
+ * Codegen for question-answering, zero-shot, similarity, and ranking tasks.
+ *
+ * These tasks are prompt-driven but need extra per-row or per-operator
+ * inputs: context text, candidate labels, table contents, or a list of
+ * comparison sentences/documents.
+ */
+object QaRankingCodegen extends TaskCodegen {
+
+ override val task: String = "question-answering"
+
+ override val tasks: Set[String] = Set(
+ "question-answering",
+ "table-question-answering",
+ "zero-shot-classification",
+ "sentence-similarity",
+ "text-ranking"
+ )
+
+ override def payloadPython(ctx: CodegenContext): String =
+ """ if task == "question-answering":
+ | ctx_val = row[self.CONTEXT_COLUMN]
+ | ctx_val = "" if pd.isna(ctx_val) else str(ctx_val)
+ | payload = {"inputs": {"question": prompt_value, "context": ctx_val}}
+ | elif task == "table-question-answering":
+ | payload = {"inputs": {"query": prompt_value, "table": table_dict}}
+ | elif task == "zero-shot-classification":
+ | labels = [l.strip() for l in self.CANDIDATE_LABELS.split(",") if l.strip()]
+ | payload = {
+ | "inputs": prompt_value,
+ | "parameters": {"candidate_labels": labels},
+ | }
+ | elif task in ("sentence-similarity", "text-ranking"):
Review Comment:
**`text-ranking` may be sending the wrong body.** It reuses `sentence-similarity`'s `{source_sentence, sentences}` shape, but `text-ranking` is not a standard HF task and rerankers usually expect `{query, texts}`. As written it will likely always fail upstream.
Fix (if confirmed): check what the endpoint expects and give `text-ranking` its own payload. No suggestion since the right shape depends on the endpoint.
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegen.scala:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.huggingFace.codegen
+
+/**
+ * Codegen for question-answering, zero-shot, similarity, and ranking tasks.
+ *
+ * These tasks are prompt-driven but need extra per-row or per-operator
+ * inputs: context text, candidate labels, table contents, or a list of
+ * comparison sentences/documents.
+ */
+object QaRankingCodegen extends TaskCodegen {
+
+ override val task: String = "question-answering"
+
+ override val tasks: Set[String] = Set(
+ "question-answering",
+ "table-question-answering",
+ "zero-shot-classification",
+ "sentence-similarity",
+ "text-ranking"
+ )
+
+ override def payloadPython(ctx: CodegenContext): String =
+ """ if task == "question-answering":
+ | ctx_val = row[self.CONTEXT_COLUMN]
+ | ctx_val = "" if pd.isna(ctx_val) else str(ctx_val)
+ | payload = {"inputs": {"question": prompt_value, "context": ctx_val}}
+ | elif task == "table-question-answering":
+ | payload = {"inputs": {"query": prompt_value, "table": table_dict}}
+ | elif task == "zero-shot-classification":
+ | labels = [l.strip() for l in self.CANDIDATE_LABELS.split(",") if l.strip()]
Review Comment:
**This label parsing is duplicated** in the validation block in `PythonCodegenBase`, where it wraps the value with `str(...)`. Here it does not, so the two can drift apart.
Fix: make this line match:
```suggestion
| labels = [l.strip() for l in str(self.CANDIDATE_LABELS).split(",") if l.strip()]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "anishshiva7 (via GitHub)" <gi...@apache.org>.
anishshiva7 commented on code in PR #5574:
URL: https://github.com/apache/texera/pull/5574#discussion_r3411197282
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/PythonCodegenBase.scala:
##########
@@ -0,0 +1,897 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.huggingFace.codegen
+
+import org.apache.texera.amber.pybuilder.PythonTemplateBuilder.PythonTemplateBuilderStringContext
+
+/**
+ * Builds the Python script emitted by HuggingFaceInferenceOpDesc.
+ *
+ * The script defines a `ProcessTableOperator` class with:
+ * - Per-instance configuration set in `open(self)` from base64-encoded
+ * values that the `pyb"..."` macro decodes at runtime (so user-input
+ * strings never appear as raw Python literals in the source).
+ * - A provider-fallback system that walks the HF Hub's inference-provider
+ * list cheapest-first and tries each provider's native chat-completions
+ * route, with HF Inference Router as the default.
+ * - A `process_table` loop that validates the prompt column, builds the
+ * per-row payload via the per-task codegen, posts to the resolved
+ * provider, and parses the response.
+ * - A `_parse_response` task switch whose branches are provided by the
+ * per-task codegen.
+ *
+ * Per-task variation lives in `TaskCodegen` implementations. This class
+ * holds only what is shared across all HF tasks; per-task helpers (image
+ * loading, audio MIME inference, media-URL fetching, etc.) will be added
+ * in subsequent PRs as the corresponding task families land.
+ */
+object PythonCodegenBase {
+
+ def render(ctx: CodegenContext, codegen: TaskCodegen): String = {
+ val payload = codegen.payloadPython(ctx)
+ val parse = codegen.parsePython(ctx)
+ val hfApiToken = ctx.hfApiToken
+ val modelId = ctx.modelId
+ val promptColumn = ctx.promptColumn
+ val resultColumn = ctx.resultColumn
+ val task = ctx.task
+ val systemPrompt = ctx.systemPrompt
+ val maxNewTokens = ctx.safeMaxTokens
+ val temperature = ctx.safeTemp
+ val imageInput = ctx.imageInput
+ val inputImageColumn = ctx.inputImageColumn
+ val audioInput = ctx.audioInput
+ val inputAudioColumn = ctx.inputAudioColumn
+ val contextColumn = ctx.contextColumn
+ val candidateLabels = ctx.candidateLabels
+ val sentencesColumn = ctx.sentencesColumn
+ pyb"""import os
+ |import re
+ |import json
+ |import base64
+ |import requests
+ |import pandas as pd
+ |from urllib.parse import urlparse
+ |from pytexera import *
+ |
+ |# Defensive format check for MODEL_ID before it is interpolated into
+ |# HF URL paths. The base host is hardcoded so the worst case isn't
+ |# SSRF, but rejecting `..` segments / query strings / fragments /
+ |# control chars keeps the operator's request shape predictable.
+ |_HF_MODEL_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*(/[A-Za-z0-9._-]+)+$$")
+ |
+ |class ProcessTableOperator(UDFTableOperator):
+ |
+ | # Providers ranked cheapest-first (lower index = cheaper).
+ | # Unknown providers are appended at the end.
+ | PROVIDER_COST_PRIORITY = [
+ | "hf-inference",
+ | "cerebras",
+ | "sambanova",
+ | "groq",
+ | "novita",
+ | "nebius",
+ | "fireworks-ai",
+ | "together",
+ | "hyperbolic",
+ | "scaleway",
+ | "nscale",
+ | "ovhcloud",
+ | "deepinfra",
+ | "featherless-ai",
+ | "baseten",
+ | "publicai",
+ | "nvidia",
+ | "openai",
+ | "cohere",
+ | "clarifai",
+ | ]
+ |
+ | # Per-provider chat-completions route overrides. Providers not listed
+ | # here use the default `v1/chat/completions` path. Single source of
+ | # truth for both _post_with_fallback (text-gen) and _call_provider
+ | # (OpenAI-compatible fallback) so the two stay in sync as providers
+ | # are added.
+ | CHAT_ROUTES = {
+ | "groq": "openai/v1/chat/completions",
+ | "fireworks-ai": "inference/v1/chat/completions",
+ | "cohere": "compatibility/v1/chat/completions",
+ | "clarifai": "v2/ext/openai/v1/chat/completions",
+ | "deepinfra": "v1/openai/chat/completions",
+ | }
+ |
+ | # Third-party providers that speak the OpenAI chat-completions
+ | # protocol. Used by _call_provider's OpenAI-compatible branch.
+ | OPENAI_COMPATIBLE_PROVIDERS = (
+ | "cerebras", "sambanova", "groq", "novita", "nebius",
+ | "fireworks-ai", "together", "hyperbolic", "cohere", "clarifai",
+ | "deepinfra", "featherless-ai", "nscale", "nvidia", "openai",
+ | "ovhcloud", "publicai", "scaleway", "baseten",
+ | )
+ |
+ | def open(self):
+ | # User-provided strings reach the operator via base64-encoded
+ | # decode expressions so they cannot break Python syntax or
+ | # leak raw text into the generated source.
+ | self.HF_API_TOKEN = $hfApiToken
+ | self.MODEL_ID = $modelId
+ | self.PROMPT_COLUMN = $promptColumn
+ | self.RESULT_COLUMN = $resultColumn
+ | self.TASK = $task
+ | self.SYSTEM_PROMPT = $systemPrompt
+ | self.MAX_NEW_TOKENS = $maxNewTokens
+ | self.TEMPERATURE = $temperature
+ | self.IMAGE_INPUT = $imageInput
+ | self.INPUT_IMAGE_COLUMN = $inputImageColumn
+ | self.AUDIO_INPUT = $audioInput
+ | self.INPUT_AUDIO_COLUMN = $inputAudioColumn
+ | self.CONTEXT_COLUMN = $contextColumn
+ | self.CANDIDATE_LABELS = $candidateLabels
+ | self.SENTENCES_COLUMN = $sentencesColumn
+ |
+ | def _resolve_providers(self, token):
+ | '''Query the HF Hub API for inference providers serving this model.
+ | Returns a list of dicts with 'name' and 'providerId' sorted
+ | cheapest-first. Falls back to hf-inference if anything goes wrong.
+ | '''
+ | try:
+ | resp = requests.get(
+ | f"https://huggingface.co/api/models/{self.MODEL_ID}",
+ | headers={"Authorization": f"Bearer {token}"},
+ | params={"expand[]": "inferenceProviderMapping"},
+ | timeout=30,
+ | )
+ | if resp.status_code == 200:
+ | data = resp.json()
+ | mapping = (
+ | data.get("inferenceProviderMapping")
+ | or data.get("inference_provider_mapping")
+ | or {}
+ | )
+ | if mapping:
+ | live = [
+ | {
+ | "name": p,
+ | "providerId": v.get("providerId", self.MODEL_ID),
+ | "task": v.get("task", ""),
+ | "isModelAuthor": v.get("isModelAuthor", False),
+ | }
+ | for p, v in mapping.items()
+ | if isinstance(v, dict) and v.get("status") == "live"
+ | ]
+ | if live:
+ | priority = {name: idx for idx, name in enumerate(self.PROVIDER_COST_PRIORITY)}
+ | live.sort(key=lambda prov: priority.get(prov["name"], len(self.PROVIDER_COST_PRIORITY)))
+ | return live
+ | except Exception:
+ | pass
+ | return [{"name": "hf-inference", "providerId": self.MODEL_ID}]
+ |
+ | def _post_with_fallback(self, providers, json_headers, raw_binary_headers, pipeline_payload, use_raw_binary_body, prompt_value):
+ | '''Try providers in order, using the correct API route for each.
+ | Returns (response, provider_summary). provider_summary is None on
+ | success or a string describing what failed.
+ | '''
+ | RETRYABLE = (400, 404, 422, 429, 502, 503)
+ | last_resp = None
+ | errors = []
+ | for prov in providers:
+ | provider_name = prov["name"]
+ | provider_id = prov["providerId"]
+ | is_model_author = prov.get("isModelAuthor", False)
+ | prov_task = prov.get("task", "")
+ | try:
+ | if self.TASK in ("text-generation", "image-text-to-text"):
+ | route = self.CHAT_ROUTES.get(provider_name, "v1/chat/completions")
+ | url = f"https://router.huggingface.co/{provider_name}/{route}"
+ | resp = requests.post(url, headers=json_headers, json=pipeline_payload, timeout=120)
+ | elif is_model_author and prov_task in ("image-to-text", "image-text-to-text") and provider_name not in ("zai-org",):
+ | url = f"https://router.huggingface.co/{provider_name}/v1/chat/completions"
+ | img_b64 = ""
+ | if use_raw_binary_body and isinstance(pipeline_payload, bytes):
+ | img_b64 = base64.b64encode(pipeline_payload).decode("utf-8")
+ | chat_payload = {
+ | "model": provider_id,
+ | "messages": [{
+ | "role": "user",
+ | "content": [
+ | {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}} if img_b64 else None,
+ | {"type": "text", "text": prompt_value if prompt_value else "What is in this image?"},
+ | ],
+ | }],
+ | }
+ | chat_payload["messages"][0]["content"] = [c for c in chat_payload["messages"][0]["content"] if c is not None]
+ | resp = requests.post(url, headers=json_headers, json=chat_payload, timeout=120)
+ | elif provider_name == "hf-inference":
+ | url = f"https://router.huggingface.co/hf-inference/models/{self.MODEL_ID}"
+ | if use_raw_binary_body:
+ | resp = requests.post(url, headers=raw_binary_headers, data=pipeline_payload, timeout=120)
+ | else:
+ | resp = requests.post(url, headers=json_headers, json=pipeline_payload, timeout=120)
+ | else:
+ | resp = self._call_provider(provider_name, provider_id, json_headers, raw_binary_headers, pipeline_payload, use_raw_binary_body, prompt_value)
+ | except Exception as e:
+ | errors.append(f"{provider_name}: {type(e).__name__}")
+ | continue
+ | if resp.status_code in (200, 201):
+ | return resp, None
+ | if resp.status_code == 401:
+ | return resp, None
+ | try:
+ | detail = resp.json().get("error", resp.text[:200])
+ | except Exception:
+ | detail = resp.text[:200] if resp.text else "no details"
+ | errors.append(f"{provider_name}: HTTP {resp.status_code} - {detail}")
+ | last_resp = resp
+ | if resp.status_code not in RETRYABLE:
+ | return resp, "; ".join(errors)
+ | summary = "; ".join(errors) if errors else "no providers available"
+ | return last_resp, summary
+ |
+ | def _call_provider(self, provider_name, provider_id, json_headers, raw_binary_headers, pipeline_payload, use_raw_binary_body, prompt_value):
+ | '''Route to a third-party provider using its native API format.
+ | Handles OpenAI-compatible chat providers for text-gen, zai-org's
+ | custom API, Replicate / Fal-ai / Wavespeed for media-generation
+ | and image-to-image, and an unknown-provider fallback that tries
+ | the pipeline format then chat completions.
+ | '''
+ | base = f"https://router.huggingface.co/{provider_name}"
+ | task = self.TASK
+ | img_b64 = ""
+ | if use_raw_binary_body and isinstance(pipeline_payload, bytes):
+ | img_b64 = base64.b64encode(pipeline_payload).decode("utf-8")
+ |
+ | # zai-org: custom /api/paas/v4/ surface.
+ | if provider_name == "zai-org":
+ | zai_headers = {**json_headers, "x-source-channel": "hugging_face", "accept-language": "en-US,en"}
+ | if task in ("image-to-text", "image-text-to-text"):
+ | url = f"{base}/api/paas/v4/layout_parsing"
+ | file_data = f"data:image/png;base64,{img_b64}" if img_b64 else ""
+ | return requests.post(url, headers=zai_headers, json={"model": provider_id, "file": file_data}, timeout=120)
+ | url = f"{base}/api/paas/v4/chat/completions"
+ | messages = [{"role": "user", "content": prompt_value}]
+ | if img_b64:
+ | messages = [{"role": "user", "content": [
+ | {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
+ | {"type": "text", "text": prompt_value if prompt_value else "What is in this image?"},
+ | ]}]
+ | return requests.post(url, headers=zai_headers, json={"model": provider_id, "messages": messages}, timeout=120)
+ |
+ | # Replicate: synchronous predictions endpoint with polling fallback.
+ | if provider_name == "replicate":
+ | url = f"{base}/v1/models/{provider_id}/predictions"
+ | hdrs = {**json_headers, "Prefer": "wait"}
+ | if task == "text-to-speech":
+ | inp = {"text": prompt_value}
+ | elif task in ("text-to-image", "text-to-video"):
+ | inp = {"prompt": prompt_value}
+ | elif task == "automatic-speech-recognition" and img_b64:
+ | inp = {"audio": f"data:audio/wav;base64,{img_b64}"}
+ | elif task == "image-to-image" and img_b64:
+ | data_url = f"data:image/png;base64,{img_b64}"
+ | inp = {"image": data_url, "images": [data_url], "input_image": data_url, "prompt": prompt_value}
+ | elif img_b64:
+ | inp = {"image": f"data:image/png;base64,{img_b64}", "prompt": prompt_value}
+ | else:
+ | inp = {"prompt": prompt_value}
+ | resp = requests.post(url, headers=hdrs, json={"input": inp}, timeout=120)
+ | if resp.status_code == 202:
+ | import time as _time
+ | pred = resp.json()
+ | poll_url = pred.get("urls", {}).get("get", "")
+ | if not poll_url:
+ | return resp
+ | from urllib.parse import urlparse as _urlparse
+ | poll_path = _urlparse(poll_url).path
+ | poll_url = f"{base}{poll_path}"
+ | # Worst case: 300 polls × 2s = ~10 minutes per row before we give
+ | # up. Sized for text-to-video which legitimately takes minutes on
+ | # Replicate. process_table is synchronous, so emit a progress
+ | # line every 30 polls (~1 min) to distinguish slow work from a
+ | # hang in the worker log.
+ | for poll_idx in range(300):
+ | _time.sleep(2)
+ | poll_resp = requests.get(poll_url, headers=json_headers, timeout=30)
+ | if poll_resp.status_code != 200:
+ | continue
+ | status = poll_resp.json().get("status", "")
+ | if status in ("succeeded", "failed", "canceled"):
+ | return poll_resp
+ | if (poll_idx + 1) % 30 == 0:
+ | print(f"[hf] Replicate still running for model '{self.MODEL_ID}' after {(poll_idx + 1) * 2}s; will wait up to 600s.")
+ | return poll_resp
+ | return resp
+ |
+ | # Fal-ai: per-model endpoint.
+ | if provider_name == "fal-ai":
+ | url = f"{base}/{provider_id}"
+ | if task == "text-to-speech":
+ | return requests.post(url, headers=json_headers, json={"text": prompt_value}, timeout=120)
+ | if task in ("text-to-image", "text-to-video"):
+ | return requests.post(url, headers=json_headers, json={"prompt": prompt_value}, timeout=120)
+ | if task == "image-to-image" and img_b64:
+ | data_url = f"data:image/png;base64,{img_b64}"
+ | return requests.post(url, headers=json_headers, json={"image_url": data_url, "image_urls": [data_url], "prompt": prompt_value}, timeout=120)
+ | if img_b64:
+ | return requests.post(url, headers=json_headers, json={"image_url": f"data:image/png;base64,{img_b64}", "prompt": prompt_value}, timeout=120)
+ | return requests.post(url, headers=json_headers, json={"prompt": prompt_value}, timeout=120)
+ |
+ | # Wavespeed: async submit + poll.
+ | if provider_name == "wavespeed":
+ | url = f"{base}/api/v3/{provider_id}"
+ | payload = {"prompt": prompt_value}
+ | if img_b64:
+ | payload["image"] = img_b64
+ | payload["images"] = [img_b64]
+ | submit_resp = requests.post(url, headers=json_headers, json=payload, timeout=120)
+ | if submit_resp.status_code not in (200, 201):
+ | return submit_resp
+ | get_path = submit_resp.json().get("data", {}).get("urls", {}).get("get", "")
+ | if not get_path:
+ | return submit_resp
+ | from urllib.parse import urlparse as _urlparse
+ | result_url = f"{base}{_urlparse(get_path).path}"
+ | import time as _time
+ | poll_resp = submit_resp
+ | # Worst case: 120 polls × 1s = ~2 minutes per row. Emit a progress
+ | # line every 30 polls (~30 s) so the worker log distinguishes slow
+ | # work from a hang.
+ | for poll_idx in range(120):
+ | _time.sleep(1)
+ | poll_resp = requests.get(result_url, headers=json_headers, timeout=30)
+ | if poll_resp.status_code != 200:
+ | continue
+ | status = poll_resp.json().get("data", {}).get("status", "")
+ | if status in ("completed", "failed"):
+ | return poll_resp
+ | if (poll_idx + 1) % 30 == 0:
+ | print(f"[hf] Wavespeed still running for model '{self.MODEL_ID}' after {poll_idx + 1}s; will wait up to 120s.")
+ | return poll_resp
+ |
+ | if provider_name in self.OPENAI_COMPATIBLE_PROVIDERS:
+ | if task == "text-to-image":
+ | url = f"{base}/v1/images/generations"
+ | return requests.post(url, headers=json_headers, json={"model": provider_id, "prompt": prompt_value}, timeout=120)
+ | if task == "text-to-speech":
+ | url = f"{base}/v1/audio/speech"
+ | return requests.post(url, headers=json_headers, json={"model": provider_id, "input": prompt_value}, timeout=120)
+ | url = f"{base}/{self.CHAT_ROUTES.get(provider_name, 'v1/chat/completions')}"
+ | messages = [{"role": "user", "content": prompt_value}]
+ | if img_b64:
+ | messages = [{"role": "user", "content": [
+ | {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
+ | {"type": "text", "text": prompt_value if prompt_value else "What is in this image?"},
+ | ]}]
+ | return requests.post(
+ | url,
+ | headers=json_headers,
+ | json={"model": provider_id, "messages": messages},
+ | timeout=120,
+ | )
+ |
+ | # Unknown provider: try pipeline format, then chat completions.
+ | url = f"{base}/{provider_id}"
+ | if use_raw_binary_body:
+ | resp = requests.post(url, headers=raw_binary_headers, data=pipeline_payload, timeout=120)
+ | else:
+ | resp = requests.post(url, headers=json_headers, json=pipeline_payload, timeout=120)
+ | if resp.status_code in (400, 404, 422):
+ | url = f"{base}/v1/chat/completions"
+ | messages = [{"role": "user", "content": prompt_value}]
+ | if img_b64:
+ | messages = [{"role": "user", "content": [
+ | {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_b64}"}},
+ | {"type": "text", "text": prompt_value if prompt_value else "Describe this image."},
+ | ]}]
+ | resp2 = requests.post(
+ | url,
+ | headers=json_headers,
+ | json={"model": provider_id, "messages": messages},
+ | timeout=120,
+ | )
+ | if resp2.status_code == 200:
+ | return resp2
+ | return resp
+ |
+ | @overrides
+ | def process_table(self, table: Table, port: int) -> Iterator[Optional[TableLike]]:
+ | prompt_col = self.PROMPT_COLUMN
+ | result_col = self.RESULT_COLUMN
+ | task = self.TASK
+ | image_only_tasks = ("image-classification", "object-detection", "image-segmentation", "image-to-text")
+ | image_prompt_tasks = ("visual-question-answering", "document-question-answering", "zero-shot-image-classification", "image-text-to-text", "image-to-image")
+ | image_tasks = image_only_tasks + image_prompt_tasks
+ | audio_only_tasks = ("automatic-speech-recognition", "audio-classification")
+ |
+ | # --- validate MODEL_ID format before any HF URL is built ---
+ | if not _HF_MODEL_ID_PATTERN.match(self.MODEL_ID or ""):
+ | raise ValueError(
+ | f"Invalid Hugging Face model ID '{self.MODEL_ID}'. "
+ | f"Expected format like 'org/model-name' or 'org/model-name/revision'."
+ | )
+ |
+ | # --- resolve API token ---
+ | token = self.HF_API_TOKEN if self.HF_API_TOKEN else os.environ.get("HF_TOKEN", "")
+ | if not token:
+ | raise ValueError(
+ | "Hugging Face API token is not set. "
+ | "Provide it in the operator config or via HF_TOKEN env var."
+ | )
+ |
+ | # --- resolve all available inference providers for this model (tried in order) ---
+ | providers = self._resolve_providers(token)
+ |
+ | # --- validate prompt column exists (skipped for binary-only tasks) ---
+ | if task not in image_only_tasks and task not in audio_only_tasks:
+ | assert prompt_col in table.columns, (
+ | f"Prompt column '{prompt_col}' not found in input table. "
+ | f"Available columns: {list(table.columns)}"
+ | )
+ | if task == "question-answering":
Review Comment:
fixed in dc8dbb267 by validating that candidate labels are non-empty before processing zero-shot classification
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5574:
URL: https://github.com/apache/texera/pull/5574#issuecomment-4654321363
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/5574?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `75.52743%` with `58 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 52.31%. Comparing base ([`e987f13`](https://app.codecov.io/gh/apache/texera/commit/e987f133f4a1d4be131add116cb3d8303a8acce6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`8507ca5`](https://app.codecov.io/gh/apache/texera/commit/8507ca531d3851c79b9c57c9dc1e37160c886d5d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 9 commits behind head on main.
| [Files with missing lines](https://app.codecov.io/gh/apache/texera/pull/5574?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...mage-upload/hugging-face-image-upload.component.ts](https://app.codecov.io/gh/apache/texera/pull/5574?src=pr&el=tree&filepath=frontend%2Fsrc%2Fapp%2Fworkspace%2Fcomponent%2Fhugging-face-image-upload%2Fhugging-face-image-upload.component.ts&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZnJvbnRlbmQvc3JjL2FwcC93b3Jrc3BhY2UvY29tcG9uZW50L2h1Z2dpbmctZmFjZS1pbWFnZS11cGxvYWQvaHVnZ2luZy1mYWNlLWltYWdlLXVwbG9hZC5jb21wb25lbnQudHM=) | 50.00% | [41 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/texera/pull/5574?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ge-upload/hugging-face-image-upload.component.html](https://app.codecov.io/gh/apache/texera/pull/5574?src=pr&el=tree&filepath=frontend%2Fsrc%2Fapp%2Fworkspace%2Fcomponent%2Fhugging-face-image-upload%2Fhugging-face-image-upload.component.html&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZnJvbnRlbmQvc3JjL2FwcC93b3Jrc3BhY2UvY29tcG9uZW50L2h1Z2dpbmctZmFjZS1pbWFnZS11cGxvYWQvaHVnZ2luZy1mYWNlLWltYWdlLXVwbG9hZC5jb21wb25lbnQuaHRtbA==) | 38.88% | [11 Missing :warning: ](https://app.codecov.io/gh/apache/texera/pull/5574?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rator/huggingFace/HuggingFaceInferenceOpDesc.scala](https://app.codecov.io/gh/apache/texera/pull/5574?src=pr&el=tree&filepath=common%2Fworkflow-operator%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Ftexera%2Famber%2Foperator%2FhuggingFace%2FHuggingFaceInferenceOpDesc.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3dvcmtmbG93LW9wZXJhdG9yL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvdGV4ZXJhL2FtYmVyL29wZXJhdG9yL2h1Z2dpbmdGYWNlL0h1Z2dpbmdGYWNlSW5mZXJlbmNlT3BEZXNjLnNjYWxh) | 94.73% | [0 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/texera/pull/5574?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ber/operator/huggingFace/codegen/TaskCodegen.scala](https://app.codecov.io/gh/apache/texera/pull/5574?src=pr&el=tree&filepath=common%2Fworkflow-operator%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Ftexera%2Famber%2Foperator%2FhuggingFace%2Fcodegen%2FTaskCodegen.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3dvcmtmbG93LW9wZXJhdG9yL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvdGV4ZXJhL2FtYmVyL29wZXJhdG9yL2h1Z2dpbmdGYWNlL2NvZGVnZW4vVGFza0NvZGVnZW4uc2NhbGE=) | 85.71% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/texera/pull/5574?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #5574 +/- ##
============================================
+ Coverage 51.88% 52.31% +0.42%
- Complexity 2472 2516 +44
============================================
Files 1067 1077 +10
Lines 41258 41535 +277
Branches 4437 4464 +27
============================================
+ Hits 21408 21729 +321
+ Misses 18591 18533 -58
- Partials 1259 1273 +14
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `64.61% <ø> (+22.39%)` | :arrow_up: | |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.76% <ø> (ø)` | | Carriedforward from [8a83dc2](https://app.codecov.io/gh/apache/texera/commit/8a83dc246b3fa80618de412e8d2171addb40128d?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.66% <96.26%> (+0.78%)` | :arrow_up: | Carriedforward from [8a83dc2](https://app.codecov.io/gh/apache/texera/commit/8a83dc246b3fa80618de412e8d2171addb40128d?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `1.65% <ø> (ø)` | | |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.06% <ø> (ø)` | | |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.32% <ø> (ø)` | | Carriedforward from [8a83dc2](https://app.codecov.io/gh/apache/texera/commit/8a83dc246b3fa80618de412e8d2171addb40128d?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `46.45% <48.54%> (+0.08%)` | :arrow_up: | Carriedforward from [8a83dc2](https://app.codecov.io/gh/apache/texera/commit/8a83dc246b3fa80618de412e8d2171addb40128d?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.69% <ø> (ø)` | | Carriedforward from [8a83dc2](https://app.codecov.io/gh/apache/texera/commit/8a83dc246b3fa80618de412e8d2171addb40128d?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [python](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.83% <ø> (ø)` | | Carriedforward from [8a83dc2](https://app.codecov.io/gh/apache/texera/commit/8a83dc246b3fa80618de412e8d2171addb40128d?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/5574/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `58.69% <ø> (ø)` | | Carriedforward from [8a83dc2](https://app.codecov.io/gh/apache/texera/commit/8a83dc246b3fa80618de412e8d2171addb40128d?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/5574?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "xuang7 (via GitHub)" <gi...@apache.org>.
xuang7 merged PR #5574:
URL: https://github.com/apache/texera/pull/5574
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "anishshiva7 (via GitHub)" <gi...@apache.org>.
anishshiva7 commented on PR #5574:
URL: https://github.com/apache/texera/pull/5574#issuecomment-4654339942
/request-review @Ma77Ball
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5574:
URL: https://github.com/apache/texera/pull/5574#issuecomment-4775675842
<!-- texera-reviewer-suggestion -->
### Automated Reviewer Suggestions
Based on the `git blame` history of the changed files, we recommend the following reviewers:
- No candidates found from `git blame` history.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5574:
URL: https://github.com/apache/texera/pull/5574#issuecomment-4704750580
<!-- texera-benchmarks-comment -->
## ✅ No material benchmark regressions detected
🟢 0 better · 🔴 0 worse · ⚪ 15 noise (<±5%) · 0 without baseline
> Compared against main `83df2b5` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
[Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/27525145228)
| | config | throughput | MB/s | latency | max Δ latest / 7d |
|---|---|---:|---:|---:|---:|
| ⚪ | bs=10 sw=10 sl=64 | 404 | 0.247 | 24,440/32,859/32,859 us | ⚪ within ±5% / 🟢 -6.1% |
| ⚪ | bs=100 sw=10 sl=64 | 824 | 0.503 | 120,409/147,587/147,587 us | ⚪ within ±5% / 🔴 -7.6% |
| ⚪ | bs=1000 sw=10 sl=64 | 943 | 0.576 | 1,059,729/1,128,736/1,128,736 us | ⚪ within ±5% / 🔴 +10.3% |
<details><summary>Baseline details</summary>
Latest main `83df2b5` from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---:|---:|---:|---:|---:|
| bs=10 sw=10 sl=64 | throughput | 404 tuples/sec | 423 tuples/sec | 410.82 tuples/sec | -4.5% | -1.7% |
| bs=10 sw=10 sl=64 | MB/s | 0.247 MB/s | 0.258 MB/s | 0.251 MB/s | -4.3% | -1.5% |
| bs=10 sw=10 sl=64 | p50 | 24,440 us | 23,478 us | 23,785 us | +4.1% | +2.8% |
| bs=10 sw=10 sl=64 | p95 | 32,859 us | 33,933 us | 34,980 us | -3.2% | -6.1% |
| bs=10 sw=10 sl=64 | p99 | 32,859 us | 33,933 us | 34,980 us | -3.2% | -6.1% |
| bs=100 sw=10 sl=64 | throughput | 824 tuples/sec | 838 tuples/sec | 891.94 tuples/sec | -1.7% | -7.6% |
| bs=100 sw=10 sl=64 | MB/s | 0.503 MB/s | 0.512 MB/s | 0.544 MB/s | -1.8% | -7.6% |
| bs=100 sw=10 sl=64 | p50 | 120,409 us | 118,816 us | 112,277 us | +1.3% | +7.2% |
| bs=100 sw=10 sl=64 | p95 | 147,587 us | 140,641 us | 139,802 us | +4.9% | +5.6% |
| bs=100 sw=10 sl=64 | p99 | 147,587 us | 140,641 us | 139,802 us | +4.9% | +5.6% |
| bs=1000 sw=10 sl=64 | throughput | 943 tuples/sec | 940 tuples/sec | 1,041 tuples/sec | +0.3% | -9.4% |
| bs=1000 sw=10 sl=64 | MB/s | 0.576 MB/s | 0.574 MB/s | 0.635 MB/s | +0.3% | -9.3% |
| bs=1000 sw=10 sl=64 | p50 | 1,059,729 us | 1,056,727 us | 972,714 us | +0.3% | +8.9% |
| bs=1000 sw=10 sl=64 | p95 | 1,128,736 us | 1,123,235 us | 1,023,057 us | +0.5% | +10.3% |
| bs=1000 sw=10 sl=64 | p99 | 1,128,736 us | 1,123,235 us | 1,023,057 us | +0.5% | +10.3% |
</details>
<details><summary>Raw CSV</summary>
```csv
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,494.48,200,128000,404,0.247,24439.53,32859.37,32859.37
1,100,10,64,20,2427.57,2000,1280000,824,0.503,120409.30,147586.79,147586.79
2,1000,10,64,20,21200.17,20000,12800000,943,0.576,1059728.96,1128736.30,1128736.30
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "anishshiva7 (via GitHub)" <gi...@apache.org>.
anishshiva7 commented on PR #5574:
URL: https://github.com/apache/texera/pull/5574#issuecomment-4704831313
/request-review @Ma77Ball
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(huggingface): add qa and ranking tasks [texera]
Posted by "anishshiva7 (via GitHub)" <gi...@apache.org>.
anishshiva7 commented on code in PR #5574:
URL: https://github.com/apache/texera/pull/5574#discussion_r3411196421
##########
common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/QaRankingCodegen.scala:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.amber.operator.huggingFace.codegen
+
+/**
+ * Codegen for question-answering, zero-shot, similarity, and ranking tasks.
+ *
+ * These tasks are prompt-driven but need extra per-row or per-operator
+ * inputs: context text, candidate labels, table contents, or a list of
+ * comparison sentences/documents.
+ */
+object QaRankingCodegen extends TaskCodegen {
+
+ override val task: String = "question-answering"
+
+ override val tasks: Set[String] = Set(
+ "question-answering",
+ "table-question-answering",
+ "zero-shot-classification",
+ "sentence-similarity",
+ "text-ranking"
+ )
+
+ override def payloadPython(ctx: CodegenContext): String =
+ """ if task == "question-answering":
+ | ctx_val = row[self.CONTEXT_COLUMN]
+ | ctx_val = "" if pd.isna(ctx_val) else str(ctx_val)
+ | payload = {"inputs": {"question": prompt_value, "context": ctx_val}}
+ | elif task == "table-question-answering":
+ | payload = {"inputs": {"query": prompt_value, "table": table_dict}}
+ | elif task == "zero-shot-classification":
+ | labels = [l.strip() for l in self.CANDIDATE_LABELS.split(",") if l.strip()]
+ | payload = {
+ | "inputs": prompt_value,
+ | "parameters": {"candidate_labels": labels},
+ | }
+ | elif task in ("sentence-similarity", "text-ranking"):
+ | sent_val = row[self.SENTENCES_COLUMN]
+ | sent_val = "" if pd.isna(sent_val) else str(sent_val)
+ | sentences_list = [s.strip() for s in sent_val.split(",") if s.strip()]
+ | payload = {
+ | "inputs": {
+ | "source_sentence": prompt_value,
+ | "sentences": sentences_list,
+ | }
+ | }
+ | else:
+ | payload = {"inputs": prompt_value}""".stripMargin
+
+ override def parsePython(ctx: CodegenContext): String =
+ """ if task == "question-answering":
+ | return body.get("answer", json.dumps(body))
+ | elif task == "table-question-answering":
+ | return body.get("answer", json.dumps(body))
Review Comment:
fixed in dc8dbb267 by guarding QA response parsing with isinstance(body, dict) before calling .get
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org