[PR] [integrations][java][python] Skip native structured output for a Map, JsonNode or dict field [flink-agents]

["weiqingy (via GitHub)" <gi...@apache.org>]

weiqingy opened a new pull request, #1265:
URL: https://github.com/apache/flink-agents/pull/1265

   Linked issue: Fixes #1253
   
   ### Purpose of change
   
   A schema with an open-keyed field now keeps the prompt path on providers whose native schema cannot express one, instead of going native and losing the field. In Java that is a `Map` or `JsonNode` field, rendered as an empty object allowing no keys, so the model could only return `{}`. In Python it is a `dict` field or an `extra="allow"` model, which OpenAI and Azure OpenAI strict mode reject and Anthropic closes to the same empty object.
   
   #### Runtime flow
   
   Each connection's native predicate, which feeds both `supportsNativeStructuredOutput` and the request, gains one last check:
   
   - **Java:** `PojoJsonSchemaGenerator.reachesAny(type, targets, options)` runs the victools generator with the connection's native options and reports whether it visits an instance of a target. Strict providers pass `Map` and `JsonNode`; Gemini passes `JsonNode`.
   - **Python:** `contains_open_object(model)` walks `model_json_schema()` for an object whose `additionalProperties` is not `false`, or that has `patternProperties`.
   
   #### Key decisions
   
   - **Java asks the generator, not the rendered schema**, because a rendered `Map<String,Object>` has no `additionalProperties` key to read. It uses the generation options, which change what is reached: on Gemini, `Map<String,JsonNode>` reaches `JsonNode`.
   - **Python reads the rendered schema**, where pydantic states openness, including `extra="allow"`.
   - **Fall back, not raise**: the prompt path can honour a map.
   - **Gemini keeps `Map` native**: its schema keeps the value type.
   - **vLLM splits by language.** Java vLLM reuses the OpenAI renderer, which closes a `Map` to `{}`. Python vLLM sends pydantic's typed `additionalProperties`, which vLLM's guided decoding enforces, so its predicate drops the check.
   
   ### Behavioral Semantics
   
   #### Interaction decisions
   
   | Schema contains (any depth) | OpenAI, Azure, Anthropic | vLLM | Gemini (Java) | Others |
   |---|---|---|---|---|
   | Only declared fields | unchanged | unchanged | unchanged | unchanged |
   | Java `Map` (values not `JsonNode`) | `INFEASIBLE` | `INFEASIBLE` | unchanged | unchanged |
   | Java `JsonNode`, incl. as a `Map` value | `INFEASIBLE` | `INFEASIBLE` | `INFEASIBLE` | unchanged |
   | Python `dict` field or `extra="allow"` | `INFEASIBLE` | unchanged | n/a | unchanged |
   
   The check adds to the existing blockers; any one gives `INFEASIBLE`.
   
   #### Behavioral contracts
   
   1. Java OpenAI, Azure OpenAI, vLLM and Anthropic answer `INFEASIBLE` for a schema class reaching a `Map` or `JsonNode`, and send no `response_format` / `output_config`.
   2. Java Gemini answers `INFEASIBLE` for one reaching a `JsonNode`, and sends no response schema. A `Map` of other values is unchanged.
   3. Python OpenAI, Azure OpenAI and Anthropic answer `INFEASIBLE` for a model with a `dict` field or `extra="allow"` at any depth, and send no native parameter.
   4. Python vLLM sends such a model natively, keeping its `additionalProperties`.
   5. A blocked schema uses the prompt path under `AUTO` and `PROMPT`, and raises before the call under `NATIVE`.
   6. Anthropic keeps a requested JSON prefill for a blocked schema (Java and Python).
   7. A `@JsonIgnore`d `Map` field does not block (Java).
   8. Every other schema is unchanged on every provider.
   
   #### Failure behavior
   
   - `NATIVE` with a blocked schema raises the existing early error naming the connection and schema.
   - Python: a model whose `model_json_schema()` raises is not reported open, so the render step still raises its `TypeError`.
   - Java: the query now runs the generator, uncaught, under every strategy, `PROMPT` included.
   
   ### Tests
   
   | Contract | Tests |
   |---|---|
   | 1 | `testNativeNotAppliedForOpenObjectSchema` (OpenAI, Azure), `testOpenObjectSchemaKeepsFallback` (Anthropic), `testQueryRecommendsNativeForServedModels` (vLLM), `testQueryAgreesWithTheNativeBranch` |
   | 2 | `nativeSchemaSkippedForJsonNode`, `queryAgreesWithTheNativeBranch` (Gemini); `reachesMapValuesOnlyUnderTheirOption` |
   | 3 | `test_output_schema_render.py`; `test_schema_admitting_undeclared_keys_keeps_the_prompt_path`, `test_query_agrees_with_the_native_branch` |
   | 4 | `test_map_schema_is_sent_natively_with_its_value_type` |
   | 5 | Existing strategy tests |
   | 6 | `testOpenObjectSchemaKeepsFallback`; `test_json_prefill_applied_when_open_schema_falls_back` |
   | 7 | `doesNotReachTypesTheSchemaLeavesOut` |
   | 8 | Unchanged rows of each agreement matrix; the suites below |
   
   Tests pinning a map schema as native on a blocked provider now pin the prompt path. Each new explicit test fails with its gate or override reverted; the agreement tests pass either way, as query and request move together.
   
   Run, all passing: `mvn test` on `integrations/chat-models/{common,openai,anthropic,gemini,ollama,watsonx,bedrock}` with `-am`; `pytest -m "not integration"` on `flink_agents/{api,plan,runtime,integrations/chat_models}`; `ruff check`; Spotless; `./tools/check-license.sh`.
   
   Not verified:
   
   - No live provider call; responses follow documented strict-mode rules.
   - vLLM's acceptance of a typed `additionalProperties` comes from its guided-decoding source, not a running server.
   - Whether Gemini enforces a `Map`'s typed values.
   - A Python `Any` field is not flagged (it renders with no `type`), though Java's `JsonNode` is.
   
   <details>
   <summary>Implementation invariants and supporting detail</summary>
   
   - The generator skips `@JsonIgnore`d fields and includes subtypes, as the native schema does. The new check composes with the Azure api-version, Anthropic caller `output_config` and Gemini tools blockers as one more conjunct.
   - Python vLLM does not apply OpenAI's strict rules; its guided-decoding backend (xgrammar) lists no `additionalProperties` among its unsupported JSON features.
   - `reachesAny` adds a custom definition provider that records an `isInstanceOf` match and returns `null`, so the generated schema is identical to `generate`'s. `generate` and `reachesAny` share one private `configure`.
   - Each Java connection keeps one `NATIVE_SCHEMA_OPTIONS` constant used by both generation and the query, so the two cannot drift. The cheap checks run first in every `canApply…`, and `reachesAny` runs only for a `Class` schema.
   - Recursive classes terminate: victools emits a `$ref` for a type it has already visited.
   - `contains_open_object` follows only keywords that hold subschemas (`properties` and `$defs` values, `items`, `prefixItems`, `anyOf`, `oneOf`, `allOf`, `not`), so a field named `additionalProperties` does not count.
   - Python vLLM's override returns `_native_output_model(output_schema) is not None`, the OpenAI predicate without the check; its request path reads the same predicate.
   - `JsonNode` on Ollama, watsonx and Bedrock is unchanged and not examined.
   - Suite sizes: common 11, OpenAI 152, Anthropic 104, Gemini 71, api 680, Ollama 35, watsonx 52, Bedrock 90; Python 2157 passed, 15 skipped.
   
   </details>
   
   ### API
   
   No public API is removed or changed. Added helpers: `PojoJsonSchemaGenerator.reachesAny` and `flink_agents.api.agents.types.contains_open_object`.
   
   Compatibility: under `AUTO`, a schema with an open-keyed field moves from native to the prompt path on OpenAI, Azure OpenAI, Anthropic and Java vLLM, as does a `JsonNode` on Gemini. Under `NATIVE` it fails before the call, instead of returning `{}` (Java, Python Anthropic) or a provider 400 (Python OpenAI, Azure OpenAI).
   
   ### Documentation
   
   - [ ] `doc-needed` <!-- Your PR changes impact docs -->
   - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
   - [x] `doc-included` <!-- Your PR already contains the necessary documentation updates -->
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   - [x] Yes
   - [ ] No
   
   Generated-by: GitHub Copilot CLI 1.0.98 (Claude Opus 5.5)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [integrations][java][python] Skip native structured output for a Map, JsonNode or dict field [flink-agents]

["wenjin272 (via GitHub)" <gi...@apache.org>]

wenjin272 merged PR #1265:
URL: https://github.com/apache/flink-agents/pull/1265


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [integrations][java][python] Skip native structured output for a Map, JsonNode or dict field [flink-agents]

["weiqingy (via GitHub)" <gi...@apache.org>]

weiqingy commented on PR #1265:
URL: https://github.com/apache/flink-agents/pull/1265#issuecomment-6206351479

   @wenjin272 Could you review this when you have time? It keeps a `Map`/`JsonNode` (Java) or `dict` (Python) output schema off the native path on providers whose strict schema would drop the field, so `AUTO` falls back to the prompt path instead of returning `{}`. Python vLLM is the one exception and stays native; the reason is under Key decisions. Thanks!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [integrations][java][python] Skip native structured output for a Map, JsonNode or dict field [flink-agents]

["weiqingy (via GitHub)" <gi...@apache.org>]

weiqingy commented on PR #1265:
URL: https://github.com/apache/flink-agents/pull/1265#issuecomment-6260619444

   Thanks for the reviews. I merged the latest `main` into the branch instead of rebasing, so the commits already reviewed keep their hashes and nothing was force-pushed. The squash merge flattens it either way.
   
   - **Conflicts with #1260:** resolved as sangkyoonnam described. `PojoJsonSchemaGenerator` keeps both sets of imports, and #1260's `Order`, `PartlyMapped` and `Sign` fixtures sit after `Catalog` in `PojoJsonSchemaGeneratorTest`. No other file conflicted; #1204's Anthropic changes merged cleanly.
   - **CI:** every Java failure at the old head was the `ExecutionTraceContext` compile error in `BuiltInInputRunMetricsTest`, which the merge picks up from #1267. On the merged tree, the common, OpenAI, Anthropic and Gemini chat-model tests pass (17, 152, 110 and 71) with Spotless, and the Python suite passes (2342 passed, 15 skipped).
   - **Enum fix and `Map` detection together:** they work together. #1260's enum definition is registered before `reachesAny`'s check and claims only enum types, so the check still reaches every field beside an enum. `reachesMapBesideEnumDefinition` (5e2ef147) pins this: `Order`, with only #1260's enums, is not reported, and `TaggedOrder`, the same enums plus a `Map`, is.
   - **Enum whose `@JsonValue` returns a `Map`:** confirmed, it renders as `{"type":"string"}` and `reachesAny` doesn't see the `Map`, both before and after #1260. I'd keep it out of this PR, since it isn't an open-keyed field. Jackson writes such an enum as an object but can't read that object back without a `@JsonCreator` (`MismatchedInputException`); it reads back only the map's `toString()`, such as `"{k=a}"`. So it doesn't round-trip through Jackson on either path. The enum section #1260 added already notes that a `@JsonValue` returning another type may need a `@JsonCreator`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org