You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "weiqingy (via GitHub)" <gi...@apache.org> on 2026/06/12 07:13:00 UTC
[PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
weiqingy opened a new pull request, #843:
URL: https://github.com/apache/flink-agents/pull/843
Linked issue: #280
### Purpose of change
Today `output_schema` is honored only by prompt-engineering the request and parsing the response text; no chat-model integration uses a provider's native structured-output API, which is the most reliable strategy.
This PR adds the foundation for native structured output at the chat-model connection layer, plus the OpenAI implementation, in both Java and Python. It is the first in a small stack under #280 (Azure/Ollama, Anthropic, and DashScope follow in separate PRs; wiring native output into the ReActAgent final-output flow is a separate follow-up).
How it works:
- The request's output schema is carried to the connection through a reserved key (`__structured_output_schema__`) in the existing `modelParams`/`kwargs` map, so the abstract `chat()` signature is unchanged.
- Each connection declares a boolean native-structured-output capability (`supportsNativeStructuredOutput()` / `supports_native_structured_output`), default `false`.
- A connection applies the native API only when: a schema is present, no tools are bound on the call, the schema is a POJO (Java) / `BaseModel` (Python) rather than a `RowTypeInfo`, and the setup is same-language.
- The reserved key is always removed before the SDK call so it cannot leak into a provider request.
- The prompt-engineered path is retained as the fallback and is unaffected. In the ReAct loop tools are always bound, so the native path stays dormant there and existing behavior is unchanged.
OpenAI applies `response_format` json_schema with strict validation. The other connections only strip the reserved key for now; their native paths arrive in later PRs.
The same-language guard matters because native structured output cannot work across the language boundary (a Java `Class` is not a Python `BaseModel`), so the schema object is never marshaled across the Pemja bridge.
### Tests
Unit tests with the provider SDK mocked (no network):
- OpenAI native applied when a schema is present and no tools are bound (Java + Python); the SDK request carries `response_format` json_schema strict.
- Native NOT applied when tools are bound (the no-regression gate), and NOT applied for a `RowTypeInfo` schema.
- The reserved key never leaks to a provider SDK (Python connections forward `**kwargs`, so each strips it; a direct unit test of the base pop helper covers removal).
- Same-language threading guard: a cross-language setup with an `output_schema` does not receive the reserved key; a same-language setup does.
- Existing ReActAgent prompt-path tests remain green unchanged.
### API
Yes — additive only. `BaseChatModelConnection` gains a public reserved-key constant and a `protected` capability method (default `false`); no existing signatures change.
### Documentation
- [ ] `doc-needed`
- [x] `doc-not-needed`
- [ ] `doc-included`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-5012702538
@/private/tmp/claude-501/-Users-weiqing-Workspace-flink-agents/539c5937-6252-4ac0-a4ab-131967163cd3/scratchpad/pr843-update.md
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "GreatEugenius (via GitHub)" <gi...@apache.org>.
GreatEugenius commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-4903495465
Overall the approach looks good — reserved key in the params map, per-connection capability flag, native path gated behind "schema present + no tools", prompt path as fallback.
One thought on the strip: right now every non-native connection has to pop the key itself (the 4 non-OpenAI Python ones do it purely to avoid leaking; the Java ones don't strip at all and are only safe incidentally). Since both languages already funnel through `BaseChatModelSetup.chat()` before `connection.chat(...)`, could we strip there based on the flag?
```python
# Python
if not connection.supports_native_structured_output:
merged_kwargs.pop(STRUCTURED_OUTPUT_SCHEMA_KEY, None)
```
```java
// Java
if (!connection.supportsNativeStructuredOutput()) {
params.remove(BaseChatModelConnection.STRUCTURED_OUTPUT_SCHEMA_KEY);
}
```
That removes the 4 leak-only pops on the Python side, closes the Java asymmetry, and gives the flag a real caller on both. Native connections would still pop themselves (to discard when tools are bound). Not blocking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-4975215128
Thanks @wenjin272, I agree with both objections and the order you're suggesting.
**`modelParams` as the schema channel.** It's the wrong container: `modelParams` is provider-facing generation params, the schema is framework execution metadata. Routing it through there costs a reserved key, a strip in `BaseChatModelSetup.chat()`, a pop in the native connection (each in both languages), and three test files that only prove the key never reaches an SDK. All it buys is an unchanged `chat()` signature, which is thin at 0.3-SNAPSHOT.
I'll also retract a claim from my own design doc: that an explicit parameter would break every connection. It won't. In Java a defaulted overload `chat(messages, tools, modelParams, outputSchema)` needs no edits to non-native connections. In Python it's one named `output_schema=None` per connection, and that parameter is exactly what kills the leak, since today the schema rides inside the `**kwargs` forwarded to the SDK.
**`hasTools`.** The bigger issue is that it makes the feature inert: `BaseChatModelSetup` binds tools once in `open()` and passes them on every call, so the native path never fires for a `ReActAgent` that has both tools and an `output_schema`, which is the case it's for. `hasTools` is a proxy for "is this the final turn," and only the caller knows that.
**The root cause is the missing finalization step.** The gate only exists because there's no post-loop call. Make final-output structuring a separate call (full history, schema bound, no tools) and both problems disappear: the caller declares when structure is wanted, and the schema is a real argument. That ordering removes machinery instead of adding it.
Proposed shape:
1. Explicit schema on the chat path (deletes the reserved key, strip, pop, and no-leak tests).
2. Finalization as a dedicated post-loop call at `ChatModelAction`'s no-tool-calls branch. One caveat: the LangChain doc you linked has moved past the post-loop-call shape and now uses `ProviderStrategy` (native) plus `ToolStrategy` (a synthetic tool called inside the loop, no extra call). So the extra round-trip is a cost that prior art avoids, which is my first question below.
3. Split policy from capability: the setup declares policy (`auto`/`native`/`prompt`), the connection checks capability against the effective model at request-build time. Under `auto`, an incapable model falls back to prompt instead of failing.
On `with_structured_output` itself: it returns a per-call object, but our `ChatModelSetup` is a named `Resource`, `open()`-ed once and re-resolved by name per record. A derived Setup has no registry name and can't cross the Pemja bridge; mutating the shared one is a data race in a streaming operator. The part that ports is the declarative strategy selection, which is where (3) comes from. Open to a different shape if you had one in mind.
A few calls where I have a lean but want your read:
1. The post-loop call costs one extra LLM round-trip per record. I'd start with that, since it's simpler and the parse side already assumes a no-tool-calls final turn, and treat the forced-tool alternative (`ToolStrategy`) as a later optimization. The mechanism exists on-repo (`AnthropicChatModelConnection` strict tool use, `:186-200`) but doesn't force the call, so it'd be new behavior.
2. For how `auto` decides capability, I'd avoid a known-not-supported list (it rots on every provider release) and lean toward native only when the user explicitly configures a capable model, prompt otherwise.
3. I'd keep finalization as its own follow-up PR (a #280 sub-issue), as we'd agreed, leaving this PR the foundation. Let me know if folding it in would make the review easier.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][python] Add explicit output schema parameter to the chat path (structured-output foundation) [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 merged PR #843:
URL: https://github.com/apache/flink-agents/pull/843
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-4911867818
Thanks for the review, @GreatEugenius — this is a nice simplification. Done in f35e432.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-4979819068
Thanks @weiqingy , this direction makes sense to me, and I agree with the proposed approach and sequencing.
One point to keep in mind for the finalization follow-up: the current ReAct implementation automatically adds structured-output instructions to the prompt whenever the user declares an output_schema. If the finalization call uses a model with native structured-output support, that prompt augmentation is unnecessary and could result in two overlapping schema channels.
The finalization path should therefore select the strategy before constructing the prompt: add schema instructions only for the prompt-based fallback, and omit them when native structured output is used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-4937550181
Thanks, @GreatEugenius!
@wenjin272, could you take a look when you get a chance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][python] Add explicit output schema parameter to the chat path (structured-output foundation) [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on code in PR #843:
URL: https://github.com/apache/flink-agents/pull/843#discussion_r3637119047
##########
api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java:
##########
@@ -55,4 +77,33 @@ public ResourceType getResourceType() {
*/
public abstract ChatMessage chat(
List<ChatMessage> messages, List<Tool> tools, Map<String, Object> modelParams);
+
+ /**
+ * Process a chat request that carries an output schema, and return a chat response.
+ *
+ * <p>{@code outputSchema} is framework-level execution metadata, kept off {@code modelParams}
+ * so that it can never reach a provider SDK request as a generation parameter. It is either a
+ * POJO {@link Class} or an {@link org.apache.flink.agents.api.agents.OutputSchema} (a {@code
+ * RowTypeInfo} wrapper); the two cases are distinguished by the connection that consumes it.
+ *
+ * <p>This default implementation <b>ignores</b> {@code outputSchema} and delegates to {@link
+ * #chat(List, List, Map)}. Connections without a native structured-output translation inherit
+ * it unchanged and need no edits. A connection that does translate a schema into a native
+ * provider parameter overrides this overload, and reports its capability via {@link
+ * #supportsNativeStructuredOutput(String)}.
+ *
+ * @param messages the input chat messages
+ * @param tools the tools can be called by the model
+ * @param modelParams the additional arguments passed to the model
+ * @param outputSchema the schema the response should conform to, or null for an unconstrained
+ * response
+ * @return the chat response containing model outputs
+ */
+ public ChatMessage chat(
Review Comment:
It appears that the `BaseChatModelSetup` has never invoked this `chat` interface. Since `ChatModelAction` actually calls `BaseChatModelSetup#chat`, should we also add the corresponding interface to `BaseChatModelSetup`?
##########
python/flink_agents/api/chat_models/chat_model.py:
##########
@@ -153,6 +254,14 @@ class BaseChatModelSetup(Resource):
skill_discovery_prompt: str | None = None
allowed_commands: List[str] = Field(default_factory=list)
allowed_script_dirs: List[str] = Field(default_factory=list)
+ structured_output_strategy: StructuredOutputStrategy = Field(
Review Comment:
There seems to be a Java/Python parity issue for an explicitly configured `structured_output_strategy: null`: Java treats it as `AUTO`, while Python rejects `None` with a validation error. I think we should either normalize `null` to `AUTO` in both languages or reject it consistently.
##########
api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java:
##########
@@ -55,4 +77,33 @@ public ResourceType getResourceType() {
*/
public abstract ChatMessage chat(
List<ChatMessage> messages, List<Tool> tools, Map<String, Object> modelParams);
+
+ /**
+ * Process a chat request that carries an output schema, and return a chat response.
+ *
+ * <p>{@code outputSchema} is framework-level execution metadata, kept off {@code modelParams}
+ * so that it can never reach a provider SDK request as a generation parameter. It is either a
+ * POJO {@link Class} or an {@link org.apache.flink.agents.api.agents.OutputSchema} (a {@code
+ * RowTypeInfo} wrapper); the two cases are distinguished by the connection that consumes it.
+ *
+ * <p>This default implementation <b>ignores</b> {@code outputSchema} and delegates to {@link
+ * #chat(List, List, Map)}. Connections without a native structured-output translation inherit
+ * it unchanged and need no edits. A connection that does translate a schema into a native
+ * provider parameter overrides this overload, and reports its capability via {@link
+ * #supportsNativeStructuredOutput(String)}.
+ *
+ * @param messages the input chat messages
+ * @param tools the tools can be called by the model
+ * @param modelParams the additional arguments passed to the model
+ * @param outputSchema the schema the response should conform to, or null for an unconstrained
+ * response
+ * @return the chat response containing model outputs
+ */
+ public ChatMessage chat(
+ List<ChatMessage> messages,
+ List<Tool> tools,
+ Map<String, Object> modelParams,
+ @Nullable Object outputSchema) {
+ return chat(messages, tools, modelParams);
Review Comment:
When `NATIVE` is configured, `resolvesToNative(false)` still returns `true`, so the caller may invoke the four-argument `chat()` even when the connection does not support native structured output. With the current default implementation, `outputSchema` is silently ignored and an unconstrained response may be returned. I think the default implementation should throw `UnsupportedOperationException` for a non-null `outputSchema`; supporting connections can override it, while `null` can continue delegating to the three-argument overload.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #843:
URL: https://github.com/apache/flink-agents/pull/843#discussion_r3583441863
##########
integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java:
##########
@@ -119,6 +121,11 @@ public OpenAICompletionsConnection(
this.client = builder.build();
}
+ @Override
+ protected boolean supportsNativeStructuredOutput() {
+ return true;
Review Comment:
Good catch, conceded.
One thing to add: the user can't recover from this failure. `ChatModelAction` retries (re-failing deterministically, since a 400 for an unsupported model isn't transient), then applies `ErrorHandlingStrategy`, so the user gets a thrown error or, under `IGNORE`, a silently dropped record. Nothing falls back to prompt. It slipped past the tests because `OpenAICompletionsConnectionTest` pins `"model", "gpt-4o"`, so the native test only passes on a capable model.
On the fix, I'd split the flag rather than move it. A `ChatModelSetup` boolean isn't enough alone: `BaseChatModelSetup.chat()` merges per-call `modelParams` over the setup's, so `model` is overridable per request (nothing does that today, but the interface allows it), and the connection has its own `defaultModel` under that. So policy (`auto`/`native`/`prompt`) on the setup, and capability as a connection-side check against the effective model at request-build time. Under `auto`, an incapable model falls back to prompt instead of failing.
The stale `gpt-3.5-turbo` default looks like a separate bug anyway (the javadoc example already uses `gpt-4o-mini`, and `OpenAIResponsesModelSetup` defaults to `gpt-4o`), so I'll spin that out on its own.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][python] Add explicit output schema parameter to the chat path (structured-output foundation) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-5014586810
@wenjin272 The rework implementing the explicit-output-schema approach is up. I split it into two stacked PRs so each is one logical change and easier to review:
- #843 (this one) is now the foundation only: the explicit `output_schema` parameter plus the policy/capability split. It's mechanism-only — nothing threads a schema on the chat path yet, so behavior is unchanged.
- #919 stacks on top with the OpenAI native `response_format` translation.
The remaining providers (Azure, Ollama, Anthropic, DashScope) will follow as their own PRs, and applying structured output to an agent's final output remains the separate follow-up (#912).
The policy resolver and capability predicate have no caller in #843 by design — #919 is their first consumer.
Could you take a look when you have a chance? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-4988749151
Thanks @wenjin272.
Good catch. One nuance: today that schema instruction is registered unconditionally at agent construction, in both languages (`react_agent.py:136-139`, `ReActAgent.java:81-86`), so folding your point in means making that registration strategy-driven — emit it only on the prompt-based fallback, and omit it when the native path carries the schema through `response_format`. That reinforces the policy/capability split: one strategy decision gates both, so the two channels can't disagree.
I'll capture it as a requirement on the finalization sub-issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "GreatEugenius (via GitHub)" <gi...@apache.org>.
GreatEugenius commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-4921020055
LGTM. The centralized strip in `BaseChatModelSetup.chat` reads cleanly, and I believe this lays a solid foundation for the follow-up wiring (ReActAgent final-output). Thanks @weiqingy for the quick turnaround — looking forward to the rest of the stack! @wenjin272, could you please take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on PR #843:
URL: https://github.com/apache/flink-agents/pull/843#issuecomment-4897956185
The failing check was unrelated to this PR — it was a transient CI infra failure:
- **it-java [ubuntu-latest] [java-17] [flink-2.3]**: The `Install java` setup step failed before any test ran — `actions/setup-java@v4` got an HTTP 403 downloading the Temurin JDK 17 tarball from Adoptium's release server, so the actual IT steps were skipped. The other `it-java` matrix legs (flink-2.0/2.1/2.2) passed in the same run. Evidence: [failed job](https://github.com/apache/flink-agents/actions/runs/28816166313/job/85455984819).
Re-triggered CI (empty commit) to clear it — all checks are green now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][integrations] Support models' native structured output (foundation + OpenAI) [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on code in PR #843:
URL: https://github.com/apache/flink-agents/pull/843#discussion_r3569499416
##########
integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java:
##########
@@ -119,6 +121,11 @@ public OpenAICompletionsConnection(
this.client = builder.build();
}
+ @Override
+ protected boolean supportsNativeStructuredOutput() {
+ return true;
Review Comment:
According to the [OpenAI API documentation](https://developers.openai.com/api/docs/guides/structured-outputs#structured-outputs-vs-json-mode), only `gpt-4o-mini`, `gpt-4o-2024-08-06`, and later models support structured outputs. However, since the `DEFAULT_MODEL` in `OpenAICompletionsSetup` is set to `gpt-3.5-turbo`, requests using the default configuration will fail when sending `response_format: json_schema`, rather than gracefully falling back to the prompt.
I think we can move `supportsNativeStructuredOutput` to `ChatModelSetup`, or alternatively, change the `DEFAULT_MODEL`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][python] Add explicit output schema parameter to the chat path (structured-output foundation) [flink-agents]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #843:
URL: https://github.com/apache/flink-agents/pull/843#discussion_r3647346898
##########
api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java:
##########
@@ -55,4 +77,33 @@ public ResourceType getResourceType() {
*/
public abstract ChatMessage chat(
List<ChatMessage> messages, List<Tool> tools, Map<String, Object> modelParams);
+
+ /**
+ * Process a chat request that carries an output schema, and return a chat response.
+ *
+ * <p>{@code outputSchema} is framework-level execution metadata, kept off {@code modelParams}
+ * so that it can never reach a provider SDK request as a generation parameter. It is either a
+ * POJO {@link Class} or an {@link org.apache.flink.agents.api.agents.OutputSchema} (a {@code
+ * RowTypeInfo} wrapper); the two cases are distinguished by the connection that consumes it.
+ *
+ * <p>This default implementation <b>ignores</b> {@code outputSchema} and delegates to {@link
+ * #chat(List, List, Map)}. Connections without a native structured-output translation inherit
+ * it unchanged and need no edits. A connection that does translate a schema into a native
+ * provider parameter overrides this overload, and reports its capability via {@link
+ * #supportsNativeStructuredOutput(String)}.
+ *
+ * @param messages the input chat messages
+ * @param tools the tools can be called by the model
+ * @param modelParams the additional arguments passed to the model
+ * @param outputSchema the schema the response should conform to, or null for an unconstrained
+ * response
+ * @return the chat response containing model outputs
+ */
+ public ChatMessage chat(
Review Comment:
Good question, and yes, nothing calls the four-argument `chat` in this PR.
That is the scope here: the mechanism, with no framework path passing a schema yet. The setup-level overload is the point where the policy on the setup meets the connection-side capability, and where that resolution should live is still open. #912 lists it as an open question: "Where should `auto` resolve policy into a concrete strategy, given capability is a connection-side predicate over the effective model at request-build time?"
#912 is also the consumer. It issues a dedicated structured call at `ChatModelAction`'s no-tool-calls branch, so it is the change that both needs the setup-level entry point and can actually exercise it. Adding the overload here would settle that open question with nothing able to validate the choice, and it has to move together with normalizing effective-model resolution across Java and Python.
If you would rather see the complete path in one PR, I am happy to pull it forward here instead.
##########
api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java:
##########
@@ -55,4 +77,33 @@ public ResourceType getResourceType() {
*/
public abstract ChatMessage chat(
List<ChatMessage> messages, List<Tool> tools, Map<String, Object> modelParams);
+
+ /**
+ * Process a chat request that carries an output schema, and return a chat response.
+ *
+ * <p>{@code outputSchema} is framework-level execution metadata, kept off {@code modelParams}
+ * so that it can never reach a provider SDK request as a generation parameter. It is either a
+ * POJO {@link Class} or an {@link org.apache.flink.agents.api.agents.OutputSchema} (a {@code
+ * RowTypeInfo} wrapper); the two cases are distinguished by the connection that consumes it.
+ *
+ * <p>This default implementation <b>ignores</b> {@code outputSchema} and delegates to {@link
+ * #chat(List, List, Map)}. Connections without a native structured-output translation inherit
+ * it unchanged and need no edits. A connection that does translate a schema into a native
+ * provider parameter overrides this overload, and reports its capability via {@link
+ * #supportsNativeStructuredOutput(String)}.
+ *
+ * @param messages the input chat messages
+ * @param tools the tools can be called by the model
+ * @param modelParams the additional arguments passed to the model
+ * @param outputSchema the schema the response should conform to, or null for an unconstrained
+ * response
+ * @return the chat response containing model outputs
+ */
+ public ChatMessage chat(
+ List<ChatMessage> messages,
+ List<Tool> tools,
+ Map<String, Object> modelParams,
+ @Nullable Object outputSchema) {
+ return chat(messages, tools, modelParams);
Review Comment:
Agreed, fixed in `aead976`.
The default overload now throws `UnsupportedOperationException` for a non-null `outputSchema` and still delegates to the three-argument overload for null, so a connection that translates a schema overrides it and nothing changes for schemaless calls.
Python needed a different shape for the same contract. `chat` is abstract there, so there is no inherited body that could refuse a schema. A guard on the connection base raises `NotImplementedError`, and every connection without a native translation calls it as the first statement of `chat`, before any client access. The connection subclass walk in `test_output_schema_param_declared.py` now asserts the rejection as well, so a connection added later is held to the same contract.
One thing worth flagging since it is not visible from the diff: the throw has no production caller in either language yet, so it stays dormant until the native path lands. Java's only connection call is the three-argument one in `BaseChatModelSetup`, and on the Python side `chat_model_action.py` calls `chat` without a schema and applies the prompt fallback separately.
##########
python/flink_agents/api/chat_models/chat_model.py:
##########
@@ -153,6 +254,14 @@ class BaseChatModelSetup(Resource):
skill_discovery_prompt: str | None = None
allowed_commands: List[str] = Field(default_factory=list)
allowed_script_dirs: List[str] = Field(default_factory=list)
+ structured_output_strategy: StructuredOutputStrategy = Field(
Review Comment:
Good catch, fixed in `e537e4d`.
Python now normalizes an explicit null to `AUTO`, matching Java. I went that direction rather than rejecting in both because Java cannot reject it: `getArgument` is a plain map lookup (`ResourceDescriptor.java:93-96`), so an absent key and an explicit null reach `fromArgument` identically, and rejecting the one would reject the other.
Only null normalizes. An empty string, `0`, `False` and an unrecognized name all still raise, and the case-insensitive resolution of both the lowercase values and the Java enum names is unchanged.
I also added the Java test for an explicitly null argument. The neighbouring strategy tests build their descriptors with `Map.of`, which rejects a null value, so that case could not be expressed there and the behavior this parity fix relies on was not pinned on the Java side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [api][python] Add explicit output schema parameter to the chat path (structured-output foundation) [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on code in PR #843:
URL: https://github.com/apache/flink-agents/pull/843#discussion_r3649726742
##########
api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java:
##########
@@ -55,4 +77,33 @@ public ResourceType getResourceType() {
*/
public abstract ChatMessage chat(
List<ChatMessage> messages, List<Tool> tools, Map<String, Object> modelParams);
+
+ /**
+ * Process a chat request that carries an output schema, and return a chat response.
+ *
+ * <p>{@code outputSchema} is framework-level execution metadata, kept off {@code modelParams}
+ * so that it can never reach a provider SDK request as a generation parameter. It is either a
+ * POJO {@link Class} or an {@link org.apache.flink.agents.api.agents.OutputSchema} (a {@code
+ * RowTypeInfo} wrapper); the two cases are distinguished by the connection that consumes it.
+ *
+ * <p>This default implementation <b>ignores</b> {@code outputSchema} and delegates to {@link
+ * #chat(List, List, Map)}. Connections without a native structured-output translation inherit
+ * it unchanged and need no edits. A connection that does translate a schema into a native
+ * provider parameter overrides this overload, and reports its capability via {@link
+ * #supportsNativeStructuredOutput(String)}.
+ *
+ * @param messages the input chat messages
+ * @param tools the tools can be called by the model
+ * @param modelParams the additional arguments passed to the model
+ * @param outputSchema the schema the response should conform to, or null for an unconstrained
+ * response
+ * @return the chat response containing model outputs
+ */
+ public ChatMessage chat(
Review Comment:
make sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org