You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "atiaomar1978-hub (via GitHub)" <gi...@apache.org> on 2026/09/19 01:19:51 UTC

[PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

atiaomar1978-hub opened a new pull request, #26608:
URL: https://github.com/apache/camel/pull/26608

   # Description
   
   _AI-generated on behalf of atiaomar1978-hub._
   
   Closes the remaining `camel-openai` gaps tracked in [CAMEL-24671](https://issues.apache.org/jira/browse/CAMEL-24671) after the Responses API work in CAMEL-24670.
   
   ### Changes
   
   1. **Chat-completion multimodal input** — PDF (`file` content part) and short audio clips (`input_audio`, WAV/MP3) via `OpenAIChatCompletionMultimodalSupport`.
   2. **Diarized transcription** — `diarized_json` handling with optional chunking/speaker/keyword/language/include params; re-parses JSON payloads returned through the OpenAI Java SDK string handler; exposes `CamelOpenAIAudioDiarizedSegments`.
   3. **Audio upload filename inference** — multipart uploads without a usable file name infer `audio.wav` / `audio.mp3` from MIME type (basename/path aware).
   4. **GenAI observability + error metadata** — audio (transcription/translation/speech), image (generation/edit) and tool-execution producers; adds `TRANSCRIPTION` to `GenAiOperationName`.
   
   ### Review follow-ups
   
   - `audioChunkingStrategy` is omitted unless explicitly set (`auto` / `vad` only when configured).
   - VAD chunking sends `type=server_vad`.
   - `storeFullResponse=true` stores the reparsed diarized response when applicable.
   - Chat `input_audio` accepts WAV/MP3 MIME types only; catalog docs regenerated.
   
   ### Tests
   
   ```
   mvn -pl components/camel-ai/camel-openai,test-infra/camel-test-infra-openai-mock test \
     -Dtest=OpenAIChatCompletionMultimodalMockTest,OpenAIAudioTranscriptionDiarizedMockTest,\
   OpenAIAudioTranscriptionObservabilityTest,OpenAIImageGenerationObservabilityTest,\
   OpenAIMimeTypeHelperTest,OpenAIVisionBodyTypesMockTest,OpenAIAudioTranscriptionMockTest
   ```
   
   # Target
   
   - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch)
   
   # Tracking
   - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it).
   
   # Apache Camel coding standards and style
   
   - [x] I checked that each commit in the pull request has a meaningful subject line and body.
   - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes.
   
   # AI-assisted contributions
   
   - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on code in PR #26608:
URL: https://github.com/apache/camel/pull/26608#discussion_r4056474425


##########
components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIChatCompletionMultimodalSupport.java:
##########
@@ -0,0 +1,229 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.openai;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Base64;
+import java.util.List;
+
+import com.openai.models.chat.completions.ChatCompletionContentPart;
+import com.openai.models.chat.completions.ChatCompletionContentPartImage;
+import com.openai.models.chat.completions.ChatCompletionContentPartInputAudio;
+import com.openai.models.chat.completions.ChatCompletionContentPartText;
+import com.openai.models.chat.completions.ChatCompletionMessageParam;
+import com.openai.models.chat.completions.ChatCompletionUserMessageParam;
+import org.apache.camel.Exchange;
+import org.apache.camel.Message;
+import org.apache.camel.WrappedFile;
+import org.apache.camel.util.ObjectHelper;
+
+/**
+ * Builds chat-completion user messages from text, image, PDF and audio bodies.
+ */
+final class OpenAIChatCompletionMultimodalSupport {
+
+    private OpenAIChatCompletionMultimodalSupport() {
+    }
+
+    static ChatCompletionMessageParam buildUserMessage(Message in, String userPrompt) throws Exception {
+        Object body = in.getBody();
+
+        if (body instanceof WrappedFile || body instanceof File || body instanceof Path) {
+            return buildFileMessage(in, userPrompt);
+        }
+        if (body instanceof byte[] || body instanceof InputStream) {
+            return buildBinaryMessage(in, userPrompt);
+        }
+        return buildTextMessage(in, userPrompt);
+    }
+
+    private static ChatCompletionMessageParam buildTextMessage(Message in, String userPrompt) {
+        String prompt = userPrompt != null ? userPrompt : in.getBody(String.class);
+        if (prompt == null || prompt.trim().isEmpty()) {
+            return null;
+        }
+        return createTextMessage(prompt);
+    }
+
+    private static ChatCompletionMessageParam buildFileMessage(Message in, String userPrompt) throws Exception {
+        Object body = in.getBody();
+        File inputFile = null;
+        if (body instanceof WrappedFile<?> wrappedFile && wrappedFile.getFile() instanceof File file) {
+            inputFile = file;
+        } else if (body instanceof File file) {
+            inputFile = file;
+        } else if (body instanceof Path path) {
+            inputFile = path.toFile();
+        }
+
+        String mime = inputFile != null
+                ? MimeTypeHelper.resolveForFile(in, inputFile) : MimeTypeHelper.resolveForBinary(in);
+
+        if (MimeTypeHelper.isText(mime)) {
+            String prompt = userPrompt;
+            if (prompt == null || prompt.isEmpty()) {
+                prompt = in.getBody(String.class);
+            }
+            if (prompt == null || prompt.isEmpty()) {
+                throw new IllegalArgumentException(
+                        "File content or user message configuration must contain the prompt text");
+            }
+            return createTextMessage(prompt);
+        }
+        if (MimeTypeHelper.isImage(mime)) {
+            byte[] image = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createImageMessage(image, mime, userPrompt);
+        }
+        if (MimeTypeHelper.isPdf(mime)) {
+            byte[] document = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createFileMessage(document, mime, fileName(in, inputFile), userPrompt);
+        }
+        if (MimeTypeHelper.isAudio(mime)) {
+            byte[] audio = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createAudioMessage(audio, mime, userPrompt);
+        }
+        throw unsupportedMimeType(mime,
+                inputFile != null ? inputFile.getName() : in.getHeader(Exchange.FILE_NAME, String.class));
+    }
+
+    private static ChatCompletionMessageParam buildBinaryMessage(Message in, String userPrompt) throws Exception {
+        String mime = MimeTypeHelper.resolveForBinary(in);
+        if (MimeTypeHelper.isText(mime)) {
+            return buildTextMessage(in, userPrompt);
+        }
+        if (MimeTypeHelper.isImage(mime)) {
+            return createImageMessage(readBodyBytes(in), mime, userPrompt);
+        }
+        if (MimeTypeHelper.isPdf(mime)) {
+            return createFileMessage(readBodyBytes(in), mime, fileName(in, null), userPrompt);
+        }
+        if (MimeTypeHelper.isAudio(mime)) {
+            return createAudioMessage(readBodyBytes(in), mime, userPrompt);
+        }
+        throw unsupportedMimeType(mime, in.getHeader(Exchange.FILE_NAME, String.class));
+    }

Review Comment:
   Confirming from the CI run on the `Regen` head ([35495100065](https://github.com/apache/camel/actions/runs/35495100065)): `OpenAIVisionBodyTypesMockTest.byteArrayBodyWithoutMimeInfoIsTreatedAsText` fails on all three reruns (`expected: <text response> but was: <hello bytes>` — the exchange failed, so the original body came back). That test predates this PR (CAMEL-23739), so the `throw unsupportedMimeType(...)` in `buildBinaryMessage` changes an existing contract.
   
   The earlier review asked to stop the silent text fallback for an *unrecognised* MIME type; a `byte[]` with *no* MIME information at all (no `CamelOpenAIMediaType` / `Content-Type` / `CamelFileContentType` / `CamelFileName`) is a different case and should keep falling back to text. Suggest: `if (mime == null || MimeTypeHelper.isText(mime)) return buildTextMessage(in, userPrompt);` before the image/pdf/audio guards, and keep the throw for a MIME type that is present but unsupported. That keeps the existing test green and the new `OpenAIChatCompletionMultimodalMockTest` case meaningful.
   
   The PR is otherwise approved; it can be merged once this run is green.
   
   _Claude Code on behalf of davsclaus_



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "gnodet-bot (via GitHub)" <gi...@apache.org>.
gnodet-bot commented on code in PR #26608:
URL: https://github.com/apache/camel/pull/26608#discussion_r4056342626


##########
components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioSpeechProducer.java:
##########
@@ -84,12 +87,21 @@ public void process(Exchange exchange) throws Exception {
 
         SpeechCreateParams params = paramsBuilder.build();
 

Review Comment:
   ⚠️ **Wrong operation name for TTS spans.** `GENERATE_CONTENT` is the catch-all label for image generation and Gemini-style operations — it makes speech spans indistinguishable from image-generation spans in any telemetry dashboard. The [OTel GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) define `gen_ai.operation.name = "text_to_speech"` for TTS. `TRANSCRIPTION` and `TRANSLATION` were correctly added to `GenAiOperationName` in this PR — `SPEECH` (mapped to `"text_to_speech"`) was overlooked.
   
   ```suggestion
           GenAiObservation observation = OpenAIGenAiProducerSupport.start(exchange, GenAiOperationName.SPEECH, model);
   ```
   
   And add the missing enum value in `GenAiOperationName.java`:
   ```java
   /** Text-to-speech audio generation operation. */
   SPEECH("text_to_speech"),
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26608:
URL: https://github.com/apache/camel/pull/26608#issuecomment-5748208132

   <!-- regen-patch -->
   ### :warning: Generated files are out of date
   
   The [CI build](https://github.com/apache/camel/actions/runs/35466009514) regenerated files that are not committed on this PR.
   
   If *Allow edits from maintainers* is enabled, a maintainer bot will automatically
   push a "Regen" commit to this branch shortly.
   
   Otherwise, you can apply the patch manually — the missing changes are available
   as the `regen-patch` artifact of that run:
   
   ```bash
   gh pr checkout 26608 --repo apache/camel
   gh run download 35466009514 --repo apache/camel -n regen-patch
   git apply --index regen.patch && rm regen.patch pr-number
   git commit -m "Regen" && git push
   ```
   
   Alternatively, run `./etc/scripts/regen.sh` locally and commit the resulting changes.
   If the patch does not apply cleanly, update the branch from the base branch first —
   the patch was generated against the merge with the base branch.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #26608:
URL: https://github.com/apache/camel/pull/26608#issuecomment-5750808907

   there are merge conflicts to resolve, rebase on main


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "gnodet-bot (via GitHub)" <gi...@apache.org>.
gnodet-bot commented on code in PR #26608:
URL: https://github.com/apache/camel/pull/26608#discussion_r4056379166


##########
components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIChatCompletionMultimodalSupport.java:
##########
@@ -0,0 +1,229 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.openai;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Base64;
+import java.util.List;
+
+import com.openai.models.chat.completions.ChatCompletionContentPart;
+import com.openai.models.chat.completions.ChatCompletionContentPartImage;
+import com.openai.models.chat.completions.ChatCompletionContentPartInputAudio;
+import com.openai.models.chat.completions.ChatCompletionContentPartText;
+import com.openai.models.chat.completions.ChatCompletionMessageParam;
+import com.openai.models.chat.completions.ChatCompletionUserMessageParam;
+import org.apache.camel.Exchange;
+import org.apache.camel.Message;
+import org.apache.camel.WrappedFile;
+import org.apache.camel.util.ObjectHelper;
+
+/**
+ * Builds chat-completion user messages from text, image, PDF and audio bodies.
+ */
+final class OpenAIChatCompletionMultimodalSupport {
+
+    private OpenAIChatCompletionMultimodalSupport() {
+    }
+
+    static ChatCompletionMessageParam buildUserMessage(Message in, String userPrompt) throws Exception {
+        Object body = in.getBody();
+
+        if (body instanceof WrappedFile || body instanceof File || body instanceof Path) {
+            return buildFileMessage(in, userPrompt);
+        }
+        if (body instanceof byte[] || body instanceof InputStream) {
+            return buildBinaryMessage(in, userPrompt);
+        }
+        return buildTextMessage(in, userPrompt);
+    }
+
+    private static ChatCompletionMessageParam buildTextMessage(Message in, String userPrompt) {
+        String prompt = userPrompt != null ? userPrompt : in.getBody(String.class);
+        if (prompt == null || prompt.trim().isEmpty()) {
+            return null;
+        }
+        return createTextMessage(prompt);
+    }
+
+    private static ChatCompletionMessageParam buildFileMessage(Message in, String userPrompt) throws Exception {
+        Object body = in.getBody();
+        File inputFile = null;
+        if (body instanceof WrappedFile<?> wrappedFile && wrappedFile.getFile() instanceof File file) {
+            inputFile = file;
+        } else if (body instanceof File file) {
+            inputFile = file;
+        } else if (body instanceof Path path) {
+            inputFile = path.toFile();
+        }
+
+        String mime = inputFile != null
+                ? MimeTypeHelper.resolveForFile(in, inputFile) : MimeTypeHelper.resolveForBinary(in);
+
+        if (MimeTypeHelper.isText(mime)) {
+            String prompt = userPrompt;
+            if (prompt == null || prompt.isEmpty()) {
+                prompt = in.getBody(String.class);
+            }
+            if (prompt == null || prompt.isEmpty()) {
+                throw new IllegalArgumentException(
+                        "File content or user message configuration must contain the prompt text");
+            }
+            return createTextMessage(prompt);
+        }
+        if (MimeTypeHelper.isImage(mime)) {
+            byte[] image = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createImageMessage(image, mime, userPrompt);
+        }
+        if (MimeTypeHelper.isPdf(mime)) {
+            byte[] document = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createFileMessage(document, mime, fileName(in, inputFile), userPrompt);
+        }
+        if (MimeTypeHelper.isAudio(mime)) {
+            byte[] audio = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createAudioMessage(audio, mime, userPrompt);
+        }
+        throw unsupportedMimeType(mime,
+                inputFile != null ? inputFile.getName() : in.getHeader(Exchange.FILE_NAME, String.class));
+    }
+
+    private static ChatCompletionMessageParam buildBinaryMessage(Message in, String userPrompt) throws Exception {
+        String mime = MimeTypeHelper.resolveForBinary(in);
+        if (MimeTypeHelper.isText(mime)) {
+            return buildTextMessage(in, userPrompt);
+        }
+        if (MimeTypeHelper.isImage(mime)) {
+            return createImageMessage(readBodyBytes(in), mime, userPrompt);
+        }
+        if (MimeTypeHelper.isPdf(mime)) {
+            return createFileMessage(readBodyBytes(in), mime, fileName(in, null), userPrompt);
+        }
+        if (MimeTypeHelper.isAudio(mime)) {
+            return createAudioMessage(readBodyBytes(in), mime, userPrompt);
+        }
+        throw unsupportedMimeType(mime, in.getHeader(Exchange.FILE_NAME, String.class));
+    }

Review Comment:
   ⚠️ **CI failure: untyped `byte[]` should fall back to text, not throw.**
   
   `resolveForBinary()` returns `null` when no MIME header (`CamelOpenAIMediaType`, `Content-Type`, `CamelFileContentType`) and no `CamelFileName` extension are present. With `mime == null`, none of the guards above match, so this throws — breaking the pre-existing contract that a raw `byte[]` without type info is treated as text (e.g. a prompt stored as bytes). Fix:
   
   ```suggestion
           if (mime == null || mime.isBlank()) {
               return buildTextMessage(in, userPrompt);
           }
           throw unsupportedMimeType(mime, in.getHeader(Exchange.FILE_NAME, String.class));
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26608:
URL: https://github.com/apache/camel/pull/26608#issuecomment-5748305151

   <!-- ci-tested-modules -->
   :test_tube: **CI tested the following changed modules:**
   
   - `catalog/camel-catalog`
   - `components/camel-ai/camel-ai-observability-api`
   - `components/camel-ai/camel-openai`
   - `dsl/camel-endpointdsl`
   - `test-infra/camel-test-infra-openai-mock`
   
   > :information_source: Dependent modules were not tested because the total number of affected modules exceeded the threshold (50). Use the `test-dependents` label to force testing all dependents.
   
   ---
   
   <details><summary>:microscope: Scalpel shadow comparison — Scalpel: 72 of 692 tested, 26 compile-only — current: 72 all tested</summary>
   
   [Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **100 affected modules** (current approach: 72).
   
   **Skip-tests mode would test 72 modules** (5 direct + 69 downstream), **skip tests for 26** (generated code, meta-modules)
   
   <details><summary>:warning: Modules only in Scalpel (28)</summary>
   
   - `apache-camel`
   - `camel-allcomponents`
   - `camel-catalog`
   - `camel-catalog-console`
   - `camel-catalog-maven`
   - `camel-catalog-suggest`
   - `camel-componentdsl`
   - `camel-endpointdsl`
   - `camel-endpointdsl-support`
   - `camel-itest`
   - `camel-jbang-core`
   - `camel-jbang-it`
   - `camel-jbang-main`
   - `camel-jbang-plugin-edit`
   - `camel-jbang-plugin-generate`
   - `camel-jbang-plugin-kubernetes`
   - `camel-jbang-plugin-test`
   - `camel-kamelet-main`
   - `camel-launcher`
   - `camel-report-maven-plugin`
   - `camel-route-parser`
   - `camel-salesforce-maven-plugin`
   - `camel-yaml-dsl`
   - `camel-yaml-dsl-deserializers`
   - `camel-yaml-dsl-maven-plugin`
   - `coverage`
   - `docs`
   - `dummy-component`
   
   </details>
   
   
   <details><summary>Modules Scalpel would test (72)</summary>
   
   - `camel-a2a` ← downstream of org.apache.camel:camel-platform-http-vertx
   - `camel-ai-observability` ← downstream of org.apache.camel:camel-ai-observability-api
   - `camel-ai-observability-api` ← components/camel-ai/camel-ai-observability-api/src/main/java/org/apache/camel/component/ai/observability/GenAiOperationName.java
   - `camel-as2` ← downstream of org.apache.camel:camel-jetty
   - `camel-catalog` ← catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/openai.json, catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/openai-component.adoc
   - `camel-clickup` ← downstream of org.apache.camel:camel-netty-http
   - `camel-consul` ← downstream of org.apache.camel:camel-spring-main
   - `camel-core-all` ← downstream of org.apache.camel:camel-main
   - `camel-csv` ← downstream of org.apache.camel:camel-main
   - `camel-endpointdsl` ← dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/OpenAIEndpointBuilderFactory.java
   - `camel-fhir` ← downstream of org.apache.camel:camel-hl7
   - `camel-groovy` ← downstream of org.apache.camel:camel-main
   - `camel-hazelcast` ← downstream of org.apache.camel:camel-spring-main
   - `camel-hl7` ← downstream of org.apache.camel:camel-netty
   - `camel-ibm-watsonx-ai` ← downstream of org.apache.camel:camel-vertx-websocket
   - `camel-iso8583` ← downstream of org.apache.camel:camel-groovy
   - `camel-java-joor-dsl` ← downstream of org.apache.camel:camel-main
   - `camel-jbang-mcp` ← downstream of org.apache.camel:camel-catalog
   - `camel-jbang-plugin-mcp` ← downstream of org.apache.camel:camel-jbang-core
   - `camel-jbang-plugin-route-parser` ← downstream of org.apache.camel:camel-route-parser
   - `camel-jbang-plugin-tui` ← downstream of org.apache.camel:camel-catalog
   - `camel-jbang-plugin-validate` ← downstream of org.apache.camel:camel-yaml-dsl-validator
   - `camel-jetty` ← downstream of org.apache.camel:camel-main
   - `camel-jsonpath` ← downstream of org.apache.camel:camel-platform-http-vertx
   - `camel-kamelet` ← downstream of org.apache.camel:camel-main
   - `camel-kamelet-main-support` ← downstream of org.apache.camel:camel-main
   - `camel-knative-http` ← downstream of org.apache.camel:camel-platform-http-vertx
   - `camel-langchain4j-agent` ← downstream of org.apache.camel:camel-ai-observability-api
   - `camel-langchain4j-chat` ← downstream of org.apache.camel:camel-ai-observability-api
   - `camel-langchain4j-embeddings` ← downstream of org.apache.camel:camel-ai-observability-api
   - `camel-langchain4j-embeddingstore` ← downstream of org.apache.camel:camel-langchain4j-embeddings
   - `camel-launcher-container` ← downstream of org.apache.camel:camel-launcher
   - `camel-mail` ← downstream of org.apache.camel:camel-spring-main
   - `camel-mail-microsoft-oauth` ← downstream of org.apache.camel:camel-mail
   - `camel-main` ← downstream of org.apache.camel:camel-ai-observability-api
   - `camel-maven-plugin` ← downstream of org.apache.camel:camel-spring-main
   - `camel-mcp-server` ← downstream of org.apache.camel:camel-openai
   - `camel-micrometer-prometheus` ← downstream of org.apache.camel:camel-platform-http-main
   - `camel-microprofile-health` ← downstream of org.apache.camel:camel-main
   - `camel-netty` ← downstream of org.apache.camel:camel-main
   - `camel-netty-http` ← downstream of org.apache.camel:camel-jetty
   - `camel-oauth` ← downstream of org.apache.camel:camel-platform-http-main
   - `camel-observability-services` ← downstream of org.apache.camel:camel-micrometer-prometheus
   - `camel-once` ← downstream of org.apache.camel:camel-groovy
   - `camel-openai` ← components/camel-ai/camel-openai/src/generated/java/org/apache/camel/component/openai/OpenAIEndpointConfigurer.java, components/camel-ai/camel-openai/src/generated/java/org/apache/camel/component/openai/OpenAIEndpointUriFactory.java, components/camel-ai/camel-openai/src/generated/resources/META-INF/org/apache/camel/component/openai/openai.json, components/camel-ai/camel-openai/src/generated/resources/META-INF/services/org/apache/camel/urifactory/llm-endpoint, components/camel-ai/camel-openai/src/main/docs/openai-component.adoc, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/MimeTypeHelper.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioSpeechProducer.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioSupport.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioTranscriptionProducer.java,
  components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioTranslationProducer.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIChatCompletionMultimodalSupport.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIConfiguration.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIConstants.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIConversationHistoryTrimmer.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIGenAiProducerSupport.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIImageEditProducer.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIImageGenerationProducer.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/
 OpenAIProducer.java, components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIToolExecutionProducer.java, components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIAudioSpeechObservabilityTest.java, components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIAudioTranscriptionDiarizedMockTest.java, components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIAudioTranscriptionObservabilityTest.java, components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIAudioTranslationObservabilityTest.java, components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIChatCompletionMultimodalMockTest.java, components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIImageGenerationObservabilityTest.java, components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIMimeTypeHelperTe
 st.java, components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIVisionBodyTypesMockTest.java
   - `camel-openapi-validator` ← downstream of org.apache.camel:camel-rest-openapi
   - `camel-platform-http-main` ← downstream of org.apache.camel:camel-main
   - `camel-platform-http-vertx` ← downstream of org.apache.camel:camel-rest-openapi
   - `camel-plc4x` ← downstream of org.apache.camel:camel-main
   - `camel-pubnub` ← downstream of org.apache.camel:camel-main
   - `camel-quickfix` ← downstream of org.apache.camel:camel-jetty
   - `camel-rest-openapi` ← downstream of org.apache.camel:camel-netty-http
   - `camel-restdsl-openapi-plugin` ← downstream of org.apache.camel:camel-jetty
   - `camel-salesforce` ← downstream of org.apache.camel:camel-csv
   - `camel-salesforce-codegen` ← downstream of org.apache.camel:camel-salesforce
   - `camel-spring-ai-chat` ← downstream of org.apache.camel:camel-ai-observability-api
   - `camel-spring-main` ← downstream of org.apache.camel:camel-main
   - `camel-state-store` ← downstream of org.apache.camel:camel-test-main-junit5
   - `camel-syslog` ← downstream of org.apache.camel:camel-netty
   - `camel-telegram` ← downstream of org.apache.camel:camel-netty-http
   - `camel-test-infra-all` ← downstream of org.apache.camel:camel-test-infra-openai-mock
   - `camel-test-infra-openai-mock` ← test-infra/camel-test-infra-openai-mock/src/main/java/org/apache/camel/test/infra/openai/mock/AudioTranscriptionExpectation.java, test-infra/camel-test-infra-openai-mock/src/main/java/org/apache/camel/test/infra/openai/mock/AudioTranscriptionResponseBuilder.java, test-infra/camel-test-infra-openai-mock/src/main/java/org/apache/camel/test/infra/openai/mock/OpenAIMockBuilder.java, test-infra/camel-test-infra-openai-mock/src/main/java/org/apache/camel/test/infra/openai/mock/OpenAIMockServerHandler.java
   - `camel-test-main-junit5` ← downstream of org.apache.camel:camel-main
   - `camel-test-main-junit6` ← downstream of org.apache.camel:camel-main
   - `camel-vertx-websocket` ← downstream of org.apache.camel:camel-main
   - `camel-webhook` ← downstream of org.apache.camel:camel-netty-http
   - `camel-whatsapp` ← downstream of org.apache.camel:camel-netty-http
   - `camel-xml-io-dsl` ← downstream of org.apache.camel:camel-main
   - `camel-xml-jaxb-dsl` ← downstream of org.apache.camel:camel-main
   - `camel-xml-jaxb-dsl-test-definition` ← downstream of org.apache.camel:camel-main
   - `camel-xml-jaxb-dsl-test-spring` ← downstream of org.apache.camel:camel-main
   - `camel-yaml-dsl-validator` ← downstream of org.apache.camel:camel-catalog
   - `camel-yaml-dsl-validator-maven-plugin` ← downstream of org.apache.camel:camel-yaml-dsl-validator
   - `camel-zookeeper` ← downstream of org.apache.camel:camel-spring-main
   
   </details>
   
   <details><summary>Modules with tests skipped (26)</summary>
   
   - `apache-camel`
   - `camel-allcomponents`
   - `camel-catalog-console`
   - `camel-catalog-maven`
   - `camel-catalog-suggest`
   - `camel-componentdsl`
   - `camel-endpointdsl-support`
   - `camel-itest`
   - `camel-jbang-core`
   - `camel-jbang-it`
   - `camel-jbang-main`
   - `camel-jbang-plugin-edit`
   - `camel-jbang-plugin-generate`
   - `camel-jbang-plugin-kubernetes`
   - `camel-jbang-plugin-test`
   - `camel-kamelet-main`
   - `camel-launcher`
   - `camel-report-maven-plugin`
   - `camel-route-parser`
   - `camel-salesforce-maven-plugin`
   - `camel-yaml-dsl`
   - `camel-yaml-dsl-deserializers`
   - `camel-yaml-dsl-maven-plugin`
   - `coverage`
   - `docs`
   - `dummy-component`
   
   </details>
   
   > :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
   
   </details>
   
   :warning: **Some tests are disabled on GitHub Actions** (`@DisabledIfSystemProperty(named = "ci.env.name")`) and require manual verification:
   
   - `components/camel-ai/camel-openai`: 7 test(s) disabled on GitHub Actions
   
   <details><summary>Build reactor — dependencies compiled but only changed modules were tested (5 modules, 44.8s total)</summary>
   
   **Total reactor time:** 44.8s
   
   | Module | Duration | Status |
   | --- | --- | --- |
   | Camel :: AI :: OpenAI | 29.4s | FAILURE |
   | Camel :: AI :: Observability API | 9.0s | SUCCESS |
   | Camel :: Test Infra :: OpenAI Mock | 6.4s | SUCCESS |
   | Camel :: Catalog :: Camel Catalog | n/a |  |
   | Camel :: Endpoint DSL | n/a |  |
   
   **Top 20 slowest modules:**
   - `Camel :: AI :: OpenAI` (29.4s)
   - `Camel :: AI :: Observability API` (9.0s)
   - `Camel :: Test Infra :: OpenAI Mock` (6.4s)
   
   </details>
   
   ---
   :gear: [View full build and test results](https://github.com/apache/camel/actions/runs/35495100065)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "gnodet-bot (via GitHub)" <gi...@apache.org>.
gnodet-bot commented on code in PR #26608:
URL: https://github.com/apache/camel/pull/26608#discussion_r4054135881


##########
components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIChatCompletionMultimodalSupport.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.openai;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Base64;
+import java.util.List;
+
+import com.openai.models.chat.completions.ChatCompletionContentPart;
+import com.openai.models.chat.completions.ChatCompletionContentPartImage;
+import com.openai.models.chat.completions.ChatCompletionContentPartInputAudio;
+import com.openai.models.chat.completions.ChatCompletionContentPartText;
+import com.openai.models.chat.completions.ChatCompletionMessageParam;
+import com.openai.models.chat.completions.ChatCompletionUserMessageParam;
+import org.apache.camel.Exchange;
+import org.apache.camel.Message;
+import org.apache.camel.WrappedFile;
+import org.apache.camel.util.ObjectHelper;
+
+/**
+ * Builds chat-completion user messages from text, image, PDF and audio bodies.
+ */
+final class OpenAIChatCompletionMultimodalSupport {
+
+    private OpenAIChatCompletionMultimodalSupport() {
+    }
+
+    static ChatCompletionMessageParam buildUserMessage(Message in, String userPrompt) throws Exception {
+        Object body = in.getBody();
+
+        if (body instanceof WrappedFile || body instanceof File || body instanceof Path) {
+            return buildFileMessage(in, userPrompt);
+        }
+        if (body instanceof byte[] || body instanceof InputStream) {
+            return buildBinaryMessage(in, userPrompt);
+        }
+        return buildTextMessage(in, userPrompt);
+    }
+
+    private static ChatCompletionMessageParam buildTextMessage(Message in, String userPrompt) {
+        String prompt = userPrompt != null ? userPrompt : in.getBody(String.class);
+        if (prompt == null || prompt.trim().isEmpty()) {
+            return null;
+        }
+        return createTextMessage(prompt);
+    }
+
+    private static ChatCompletionMessageParam buildFileMessage(Message in, String userPrompt) throws Exception {
+        Object body = in.getBody();
+        File inputFile = null;
+        if (body instanceof WrappedFile<?> wrappedFile && wrappedFile.getFile() instanceof File file) {
+            inputFile = file;
+        } else if (body instanceof File file) {
+            inputFile = file;
+        } else if (body instanceof Path path) {
+            inputFile = path.toFile();
+        }
+
+        String mime = inputFile != null
+                ? MimeTypeHelper.resolveForFile(in, inputFile) : MimeTypeHelper.resolveForBinary(in);
+
+        if (MimeTypeHelper.isText(mime)) {
+            String prompt = userPrompt;
+            if (prompt == null || prompt.isEmpty()) {
+                prompt = in.getBody(String.class);
+            }
+            if (prompt == null || prompt.isEmpty()) {
+                throw new IllegalArgumentException(
+                        "File content or user message configuration must contain the prompt text");
+            }
+            return createTextMessage(prompt);
+        }
+        if (MimeTypeHelper.isImage(mime)) {
+            byte[] image = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createImageMessage(image, mime, userPrompt);
+        }
+        if (MimeTypeHelper.isPdf(mime)) {
+            byte[] document = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createFileMessage(document, mime, fileName(in, inputFile), userPrompt);
+        }
+        if (MimeTypeHelper.isAudio(mime)) {
+            byte[] audio = inputFile != null ? Files.readAllBytes(inputFile.toPath()) : readBodyBytes(in);
+            return createAudioMessage(audio, mime, userPrompt);
+        }
+        throw unsupportedMimeType(mime,
+                inputFile != null ? inputFile.getName() : in.getHeader(Exchange.FILE_NAME, String.class));
+    }
+
+    private static ChatCompletionMessageParam buildBinaryMessage(Message in, String userPrompt) throws Exception {
+        String mime = MimeTypeHelper.resolveForBinary(in);
+        if (MimeTypeHelper.isImage(mime)) {
+            return createImageMessage(readBodyBytes(in), mime, userPrompt);
+        }
+        if (MimeTypeHelper.isPdf(mime)) {
+            return createFileMessage(readBodyBytes(in), mime, fileName(in, null), userPrompt);
+        }
+        if (MimeTypeHelper.isAudio(mime)) {
+            return createAudioMessage(readBodyBytes(in), mime, userPrompt);
+        }
+        return buildTextMessage(in, userPrompt);
+    }

Review Comment:
   ⚠️ **Silent text fallback for unrecognized binary body will corrupt model input**
   
   `buildBinaryMessage` is reached when the exchange body is `byte[]` or `InputStream`. If the caller didn't set a MIME header (or set an unrecognised one), `resolveForBinary` returns null, all three guards fail, and the code falls through to `buildTextMessage(in, userPrompt)` — which asks Camel to convert the raw binary body to `String`. For anything that isn't valid UTF-8 text (WAV header bytes, MP3 frames, etc.) this produces garbage that gets sent to the model silently instead of failing fast.
   
   Contrast with `buildFileMessage`, which correctly ends with `throw unsupportedMimeType(...)` for the same situation. `buildBinaryMessage` should do the same:
   
   ```suggestion
           throw unsupportedMimeType(mime, in.getHeader(Exchange.FILE_NAME, String.class));
       }
   ```
   
   Callers that genuinely want to send text as `byte[]` (or a text `InputStream`) must set a `text/plain` MIME header — just like callers of `buildFileMessage` must set one to avoid the same exception. The current silent promotion to text masks a misconfiguration that would otherwise be immediately obvious.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26608:
URL: https://github.com/apache/camel/pull/26608#issuecomment-5748137666

   :star2: Thank you for your contribution to the Apache Camel project! :star2:
     :robot: CI automation will test this PR automatically.
   
     :camel: Apache Camel Committers, please review the following items:
   
     * First-time contributors **require MANUAL approval** for the GitHub Actions to run
     * You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
     * You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
     * Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
     
     :warning: Be careful when sharing logs. Review their contents before sharing them publicly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #26608:
URL: https://github.com/apache/camel/pull/26608


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "gnodet-bot (via GitHub)" <gi...@apache.org>.
gnodet-bot commented on code in PR #26608:
URL: https://github.com/apache/camel/pull/26608#discussion_r4051821120


##########
components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioTranscriptionProducer.java:
##########
@@ -94,26 +130,93 @@ public void process(Exchange exchange) throws Exception {
                 paramsBuilder.timestampGranularities(granularities);
             }
         }
+        applyChunkingStrategy(paramsBuilder, chunkingStrategy);
+        applyCommaSeparatedList(knownSpeakerNames, paramsBuilder::addKnownSpeakerName);
+        applyCommaSeparatedList(knownSpeakerReferences, paramsBuilder::addKnownSpeakerReference);
+        applyCommaSeparatedList(keywords, paramsBuilder::addKeyword);
+        applyCommaSeparatedList(languages, paramsBuilder::addLanguage);
+        applyIncludeList(include, paramsBuilder);
+
+        return paramsBuilder.build();
+    }
 
-        TranscriptionCreateParams params = paramsBuilder.build();
-        TranscriptionCreateResponse response = getEndpoint().getClient()
-                .audio().transcriptions().create(params);
+    private static void applyChunkingStrategy(TranscriptionCreateParams.Builder paramsBuilder, String chunkingStrategy) {
+        if (ObjectHelper.isEmpty(chunkingStrategy)) {
+            return;
+        }
+        if ("auto".equalsIgnoreCase(chunkingStrategy)) {
+            paramsBuilder.chunkingStrategyAuto();
+            return;
+        }
+        if ("vad".equalsIgnoreCase(chunkingStrategy)) {
+            paramsBuilder.chunkingStrategy(TranscriptionCreateParams.ChunkingStrategy.VadConfig.builder()
+                    .type(TranscriptionCreateParams.ChunkingStrategy.VadConfig.Type.SERVER_VAD)
+                    .build());
+            return;
+        }
+        throw new IllegalArgumentException(
+                "Unsupported audio chunking strategy: " + chunkingStrategy + ". Supported values are auto and vad.");
+    }
 
+    private static void applyCommaSeparatedList(String value, Consumer<String> consumer) {
+        for (String item : OpenAIAudioSupport.parseCommaSeparatedValues(value)) {
+            consumer.accept(item);
+        }
+    }
+
+    private static void applyIncludeList(String include, TranscriptionCreateParams.Builder paramsBuilder) {
+        for (String item : OpenAIAudioSupport.parseCommaSeparatedValues(include)) {
+            paramsBuilder.addInclude(TranscriptionInclude.of(item));
+        }
+    }
+
+    private static void populateOutput(Exchange exchange, OpenAIConfiguration config, TranscriptionCreateResponse response) {
         Message out = exchange.getMessage();
+        TranscriptionCreateResponse storedResponse = response;
 
         if (response.isVerbose()) {
             TranscriptionVerbose verbose = response.asVerbose();
             out.setBody(verbose.text());
             out.setHeader(OpenAIConstants.AUDIO_DURATION, verbose.duration());
             out.setHeader(OpenAIConstants.AUDIO_DETECTED_LANGUAGE, verbose.language());
+        } else if (response.isDiarized()) {
+            applyDiarizedOutput(out, response.asDiarized());
         } else if (response.isTranscription()) {
-            out.setBody(response.asTranscription().text());
+            String text = response.asTranscription().text();
+            TranscriptionCreateResponse reparsed = tryParseStructuredTranscription(text);
+            if (reparsed != null && reparsed.isDiarized()) {
+                storedResponse = reparsed;
+                applyDiarizedOutput(out, reparsed.asDiarized());
+            } else {
+                out.setBody(text);
+            }
         } else {
             out.setBody(response.toString());
         }
 
         if (config.isStoreFullResponse()) {
-            exchange.setProperty(OpenAIConstants.AUDIO_TRANSCRIPTION_RESPONSE, response);
+            exchange.setProperty(OpenAIConstants.AUDIO_TRANSCRIPTION_RESPONSE, storedResponse);
+        }
+    }
+
+    private static void applyDiarizedOutput(Message out, TranscriptionDiarized diarized) {
+        out.setBody(diarized.text());
+        out.setHeader(OpenAIConstants.AUDIO_DURATION, diarized.duration());
+        out.setHeader(OpenAIConstants.AUDIO_DIARIZED_SEGMENTS, diarized.segments());
+    }
+
+    /**
+     * The OpenAI Java SDK delivers {@code diarized_json} responses through the plain-text handler, so the JSON payload
+     * arrives as {@link com.openai.models.audio.transcriptions.Transcription#text()}. Re-parse it when possible.
+     */
+    private static TranscriptionCreateResponse tryParseStructuredTranscription(String text) {
+        if (ObjectHelper.isEmpty(text) || !text.startsWith("{")) {
+            return null;
+        }
+        try {
+            return ObjectMappers.jsonMapper().readValue(text, TranscriptionCreateResponse.class);
+        } catch (Exception ignored) {
+            return null;
         }

Review Comment:
   ⚠️ **Silent exception swallow causes silent data corruption for `diarized_json` responses**
   
   This re-parse is the workaround for the SDK routing `diarized_json` through the plain-text handler. When the SDK delivers the JSON fine but `ObjectMappers.jsonMapper().readValue(…, TranscriptionCreateResponse.class)` fails — say, because the SDK model class doesn't map all fields of the diarized response — `tryParseStructuredTranscription` returns `null`, and the `else` branch sets the raw JSON string as the message body.
   
   The caller then gives the user a raw JSON blob with no indication that something went wrong. This is a silent correctness failure: the user asked for diarized output, gets a JSON string, and must figure out why.
   
   At minimum, log a warning so the problem is visible:
   
   ```suggestion
           try {
               return ObjectMappers.jsonMapper().readValue(text, TranscriptionCreateResponse.class);
           } catch (Exception e) {
               LOG.warn("Failed to re-parse diarized_json transcription response; returning raw text. Error: {}", e.getMessage());
               return null;
           }
   ```
   
   Also consider whether the fallback branch should set the body to the parsed JSON node (`ObjectMappers.jsonMapper().readTree(text)`) instead of the raw string, so callers can at least navigate the structure.



##########
components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioSupport.java:
##########
@@ -68,23 +71,53 @@ static void applyFileInput(
                                                    + (body != null ? body.getClass().getName() : "null")
                                                    + ". Supported: File, Path, InputStream, byte[]");
             }
-            multipartConsumer.accept(multipartWithFilename(converted, resolveFilename(in)));
+            multipartConsumer.accept(multipartWithFilename(in, converted));
         }
     }
 
     private static String resolveFilename(Message in) {
         String filename = in.getHeader(Exchange.FILE_NAME_ONLY, String.class);
+        if (ObjectHelper.isEmpty(filename)) {
+            filename = in.getHeader(Exchange.FILE_NAME, String.class);
+        }
         if (ObjectHelper.isNotEmpty(filename)) {
-            return filename;
+            filename = FileUtil.stripPath(filename);
+            if (ObjectHelper.isNotEmpty(FileUtil.onlyExt(filename))) {
+                return filename;
+            }
+        }
+        String mime = MimeTypeHelper.resolveForBinary(in);
+        String extension = MimeTypeHelper.audioExtension(mime);
+        if (ObjectHelper.isNotEmpty(extension)) {
+            return "audio." + extension;
         }
-        return "audio";
+        return "audio.mp3";

Review Comment:
   ⚠️ **Hardcoded `audio.mp3` fallback will corrupt WAV payloads**
   
   The OpenAI multipart API uses the filename extension to identify the audio format on its end. When a caller sends raw WAV bytes (or any non-MP3 audio) without setting a MIME header or a `CamelFileName`, this fallback names the upload `audio.mp3`, causing the API to treat a WAV stream as MP3 — which will produce garbage transcription or a 400 error.
   
   The previous fallback was `"audio"` (no extension), which at least let the server use content-type sniffing. This new fallback actively lies about the format.
   
   Fix: fall back to `"audio"` (no extension) unless the MIME type can actually be resolved, or throw `IllegalArgumentException` telling the caller to set `CamelOpenAIMediaType`:
   
   ```suggestion
           return "audio";
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24671: Close remaining OpenAI gaps outside the Responses API [camel]

Posted by "gnodet-bot (via GitHub)" <gi...@apache.org>.
gnodet-bot commented on code in PR #26608:
URL: https://github.com/apache/camel/pull/26608#discussion_r4052491468


##########
components/camel-ai/camel-ai-observability-api/src/main/java/org/apache/camel/component/ai/observability/GenAiOperationName.java:
##########
@@ -24,6 +24,8 @@ public enum GenAiOperationName {
     CHAT("chat"),
     EMBEDDINGS("embeddings"),
     GENERATE_CONTENT("generate_content"),
+    /** Audio transcription and translation operations. */
+    TRANSCRIPTION("transcription"),

Review Comment:
   ⚠️ **`TRANSCRIPTION` is being used for both transcription and translation** — see the companion comment on `OpenAIAudioTranslationProducer`.
   
   Add a `TRANSLATION` variant here so each operation maps to the correct semantic convention value:
   
   ```suggestion
       /** Audio transcription operation. */
       TRANSCRIPTION("transcription"),
       /** Audio translation operation. */
       TRANSLATION("translation"),
   ```



##########
components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioTranslationProducer.java:
##########
@@ -73,8 +76,17 @@ public void process(Exchange exchange) throws Exception {
         }
 
         TranslationCreateParams params = paramsBuilder.build();
-        TranslationCreateResponse response = getEndpoint().getClient()
-                .audio().translations().create(params);
+        GenAiObservation observation = OpenAIGenAiProducerSupport.start(exchange, GenAiOperationName.TRANSCRIPTION, model);

Review Comment:
   ⚠️ **Wrong operation name for translation spans**
   
   `GenAiOperationName.TRANSCRIPTION` is used here, but this is the **translation** producer. Every span emitted by `OpenAIAudioTranslationProducer` will be tagged `gen_ai.operation.name=transcription`, making translation spans indistinguishable from transcription spans in dashboards and alert rules.
   
   The `TRANSCRIPTION` enum Javadoc says "Audio transcription and translation operations" — but that was written to justify reuse, not as a hard OpenTelemetry requirement. The GenAI semantic convention specifies `translation` as a distinct operation name.
   
   The fix is to add a `TRANSLATION("translation")` variant (or at minimum `TRANSLATION("translation")` alongside `TRANSCRIPTION`) and use it here:
   
   ```suggestion
           GenAiObservation observation = OpenAIGenAiProducerSupport.start(exchange, GenAiOperationName.TRANSLATION, model);
   ```



##########
components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIAudioTranslationProducer.java:
##########
@@ -73,8 +76,17 @@ public void process(Exchange exchange) throws Exception {
         }
 
         TranslationCreateParams params = paramsBuilder.build();
-        TranslationCreateResponse response = getEndpoint().getClient()
-                .audio().translations().create(params);
+        GenAiObservation observation = OpenAIGenAiProducerSupport.start(exchange, GenAiOperationName.TRANSCRIPTION, model);

Review Comment:
   🧪 **Missing observability test for translation**
   
   This PR adds observability wrapping to both `OpenAIAudioSpeechProducer` (using `GENERATE_CONTENT`) and `OpenAIAudioTranslationProducer` (using `TRANSCRIPTION`/should be `TRANSLATION`), but neither producer has a corresponding observability test. `OpenAIAudioTranscriptionObservabilityTest` and `OpenAIImageGenerationObservabilityTest` were added for transcription and image-generation respectively — the same pattern must be applied for speech and translation.
   
   Without tests, the observability wiring for speech and translation is unverified and could silently break (e.g., missing span, wrong `gen_ai.operation.name` tag).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org