You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "SEZ9 (via GitHub)" <gi...@apache.org> on 2026/08/05 06:52:35 UTC

[PR] [Improve][CLI] Add conditional-routing skill, PostgreSQL-CDC prerequisites, and wide-DAG wiring rules [seatunnel]

SEZ9 opened a new pull request, #11659:
URL: https://github.com/apache/seatunnel/pull/11659

   ### Purpose of this pull request
   
   Fixes #11651 — adds pattern-level generation knowledge for three scenario families where the accuracy benchmark (#11553) showed near-universal model failure, following the #11616 working principle that accuracy changes must be justified by benchmark deltas.
   
   **What's added (content-only in the skill/prompt layers):**
   
   1. **New `conditional_routing` skill** — covers "split one source into different sinks by row-level condition", the weakest measured category (0% for several models). The SOP mandates the correct wiring that models consistently get wrong: ONE source block (not reading the table twice), parallel `Sql` transforms consuming the SAME source label (not chained), mutually exclusive and complete predicates (`>= 1000` pairs with `< 1000` — not `> 1000`, which silently drops equality), one sink per branch. Includes a complete runnable HOCON pattern. Trigger-routed (split / route / filter into / 拆分 / 分流 / ...), so it activates with no code changes.
   
   2. **PostgreSQL-CDC prerequisites** in the `cdc_realtime` skill — `slot.name` (one per job), `decoding.plugin.name = pgoutput` for PG 10+, schema-qualified `table-name` (`schema.table`, not MySQL-style `db.table`), server-side `wal_level = logical` note, and a MySQL-vs-PostgreSQL option comparison table. Generated PG-CDC configs previously omitted all of these.
   
   3. **Wide-DAG wiring rules** in the `multi_pipeline` skill and the config agent's Multi-Pipeline Routing prompt section — explicit per-hop labeling for chains, re-pointing the downstream consumer after inserting a transform (a sink still reading the source label silently bypasses the transform), and the three common wiring mistakes observed in generated configs.
   
   It also carries the transform-aware routing validation fix (#11657/#11658) that these scenarios depend on: without it, correctly-generated split configs are falsely rejected by local validation.
   
   **Measured effect** — benchmark subset of 7 conditional-routing/chain tasks, `qwen3-coder-next` (the weakest evaluated model in these categories), 3 independent trials per task, baseline and candidate differing only in this PR's content:
   
   | Task | Baseline | With this PR |
   |---|---|---|
   | t3_cr_amount_split | 1/3 | **3/3** |
   | t3_cr_probe_no_overlap | 0/3 | **3/3** |
   | t3_cr_status_routing | 0/3 | **2/3** |
   | t3_cx_transform_chain_routing | 0/3 | 1/3 |
   | t3_cr_log_level_split_zh | 1/3 | 1/3 |
   | t3_cr_stream_split | 2/3 | 2/3 |
   | t3_mp_mixed_transform_pipelines | 3/3 | 2/3 |
   | **Trial passes** | **7/21** | **14/21** |
   | **Tasks passing all 3 trials** | 1/7 | 2/7 |
   
   One skill-authoring lesson is documented in the commit for future contributors: an earlier draft of the wide-DAG section used pseudo-code fragments (`Sql(plugin_input="a", plugin_output="b")` arrow notation), and A/B testing showed weaker models imitate the fragment style literally — emitting non-parseable partial configs (that draft measured 3/3 → 0/3 on the chain task). **Skill pattern blocks must be complete runnable configs.**
   
   ### Does this PR introduce _any_ user-facing change?
   
   Generation quality improvement for conditional-routing, PostgreSQL-CDC, and multi-block DAG requests; no interface or config changes. The routing validation change is described in #11658.
   
   ### How was this patch tested?
   
   - Full seatunnel-cli suite: 74 passed (includes 3 new routing regression tests).
   - Skill loading and trigger routing verified: the new skill loads alongside the existing eight and activates on split/route phrasing without displacing `transform_chain` on chain-style requests.
   - Benchmark A/B as above: 42 task-trials under identical conditions (same model, same tasks, same CLI base, 3 trials per side), only this PR's content differing between the two sides.
   - The `field_mapper` option key in the new wide-DAG example was verified against the transform's docs and metadata.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][CLI] Add conditional-routing skill, PostgreSQL-CDC prerequisites, and wide-DAG wiring rules [seatunnel]

Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11659:
URL: https://github.com/apache/seatunnel/pull/11659#issuecomment-5205087504

   Reviewed at head `2794225` (branch `improve-cli-routing-cdc-knowledge`).
   
   ## Relationship to #11658
   
   This PR and #11658 branch from the same base commit and independently carry the identical `_validate_routing_pairs`/`_extract_connector_blocks_raw` fix (byte-identical diff in `agents.py` and the same 5 regression tests) — not accidental duplication, the description explains it's because this PR's new skill content depends on that fix and #11658 wasn't merged yet. Both show `mergeable: MERGEABLE` against `dev` today; since the shared hunks are textually identical, whichever merges second will just show an empty diff for that portion once rebased — no conflict either order. Would suggest merging #11658 first for a cleaner history, but not required.
   
   ## Routing-validation fix (shared with #11658) — looks correct
   
   Traced through all 5 new tests by hand against the updated code:
   - Adding `"transform"` to the section tuple in [`_extract_connector_blocks_raw`](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/agents.py#L291) correctly closes the gap; the existing brace-counting walk already handles nested blocks like `schema { fields {...} } }` fine, so no new parsing edge case here.
   - Generalizing `source.`/`sink.` to `f"{section}.{connector_name}"` in [`_validate_routing_pairs`](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/agents.py#L343-L394) is a nice side-effect fix — also corrects diagnostic misattribution for transform-originated errors, and there's a dedicated test for it.
   - Making the `_validate_routing_pairs` call unconditional ([agents.py#L502-L505](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/agents.py#L502-L505)) correctly closes the "silently skipped without pyhocon" hole from #11657.
   
   Not a regression, just flagging for awareness: `_extract_connector_blocks_raw`'s section-finding regex only finds the *first* occurrence of each section header per config, and the label regex only matches quoted `plugin_output`/`plugin_input` values (unquoted HOCON labels silently skip routing validation). Both pre-existing, unchanged by this PR.
   
   ## New domain-knowledge content — two factual issues found
   
   Since the whole point of this PR is teaching the model *correct* facts to fix generation errors, I checked the new claims against the actual connector source rather than trusting the prose. Most of it holds up, but two things don't:
   
   **1. The PostgreSQL-CDC `table-names` format guidance is wrong.** [cdc_realtime.md#L67-L69](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/skills/cdc_realtime.md#L67-L69) and the comparison table row at [L82](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/skills/cdc_realtime.md#L82) assert entries must be 3-part `database.schema.table` and explicitly say "not `schema.table`". The actual parser, [`PostgresSourceConfigFactory.java#L86-L103`](https://github.com/apache/seatunnel/blob/bf9be04805115fc856344741414b097a21497a88/seatunnel-connectors-v2/connector-cdc/connector-cdc-postgres/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/postgres/config/PostgresSourceConfigFactory.java#L86-L103), accepts *both* 2-part (`schema.table`, used as-is) and 3-part (`database.schema.table` — the leading database segment is silently **dropped**, only `schema.tabl
 e` is forwarded to Debezium); the method's own exception message calls the canonical form `schemaName.tableName` (2-part). So the skill steers the model away from the connector's own canonical format, and implies the database segment does real filtering when it's actually discarded — real database scoping happens through the separate `database-names` option (and even that only uses [the first element](https://github.com/apache/seatunnel/blob/bf9be04805115fc856344741414b097a21497a88/seatunnel-connectors-v2/connector-cdc/connector-cdc-postgres/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/postgres/config/PostgresSourceConfigFactory.java#L69) for the JDBC connection, despite being list-typed). Suggest correcting both spots to say both forms are accepted, 2-part is canonical, and the database segment (if given) is ignored.
   
   **2. The generic CDC template fix is incomplete.** The second commit's own message says it exists to fix "the generic CDC Pattern template... still presented singular `database-name`/`table-name`... contradicting the corrected PostgreSQL-specific guidance." But it only adds a caveat comment above the template ([cdc_realtime.md#L119-L124](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/skills/cdc_realtime.md#L119-L124)) — the template body itself still reads `database-name = "<database>"` / `table-name = "<table>"`. Checked: **neither singular key exists for MySQL-CDC either.** [`MySqlIncrementalSourceFactory.optionRule()`](https://github.com/apache/seatunnel/blob/bf9be04805115fc856344741414b097a21497a88/seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/source/MySqlIncrementalSourceFactory.java#L62-L64) registers only `TABLE_NAMES`/`DATABASE_NAME
 S` (plural — inherited from the same shared [`CatalogOptions.TABLE_NAMES`](https://github.com/apache/seatunnel/blob/bf9be04805115fc856344741414b097a21497a88/seatunnel-api/src/main/java/org/apache/seatunnel/api/options/table/CatalogOptions.java#L40-L45) interface Postgres-CDC uses), no singular fallback anywhere in `connector-cdc-base` or `connector-cdc-mysql`. So the "generic" template a model copies for *any* CDC connector uses keys that don't exist for either connector this skill documents. Since this PR already touches these exact lines, seems like the natural place to finish the fix — swap the template to the plural form instead of just disclaiming it above.
   
   Smaller, same area: `database-names` is list-typed but only the first element is actually used for the connection (see link above) — the skill's phrasing "lists the databases to monitor" ([L71](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/skills/cdc_realtime.md#L71)) slightly overstates that.
   
   ## FieldMapper example looks internally inconsistent
   
   [`multi_pipeline.md#L111-L113`](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/skills/multi_pipeline.md#L111-L113):
   ```
   field_mapper = {
     order_id = id
     amount = total
   }
   ```
   [`FieldMapperTransform.java#L104-L113`](https://github.com/apache/seatunnel/blob/bf9be04805115fc856344741414b097a21497a88/seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/fieldmapper/FieldMapperTransform.java#L104-L113): the map key must be an *existing input field name* (looked up via `inputFieldNames.indexOf(key)`, and the constructor throws if a key isn't found in the input schema), and the value becomes the *new output column name*. The `amount = total` entry follows that direction (existing terse column → friendly output name), but `order_id = id` reads the opposite way — if the source table's real PK column is `id` (the natural read given the sibling entry), this entry would throw at runtime for referencing a nonexistent input field `order_id`. Given the PR's own finding that weaker models imitate example patterns literally, worth fixing the direction (`id = order_id`) so the example stays internally consistent.
   
   ## Possible skill-router interaction (not confirmed, worth checking)
   
   [`skills.py#L483`](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/skills.py#L483) caps injected skills at `_MAX_SKILLS = 2`, and `multi_pipeline` gets a forced score floor of 3 whenever the plan has 2+ pipeline slots ([skills.py#L500-L503](https://github.com/apache/seatunnel/blob/279422525829722a6310bcafa506073bda14ad08/seatunnel-cli/seatunnel_cli/skills.py#L500-L503)). `conditional_routing` shares the `"split"` trigger with the existing `transform_chain` skill, and its other triggers are fairly generic ("based on", "separate", "by status"). If a 1-source/2-sink split request ends up represented as 2 pipeline slots by the planner, `multi_pipeline`'s forced floor plus a low organic score for `conditional_routing` could push the new skill out of the top-2 cut — the exact scenario it's meant to fix. This might explain the one regression in the PR's own benchmark table (`t3_mp_mixed_transform_pipelines` 3/3→2/3), curre
 ntly attributed to single-task variance, which is plausible with only 3 trials but worth checking directly — logging `SkillRouter.match()`'s actual selection for that trial would confirm or rule it out.
   
   ## Test coverage note
   
   The 5 new regression tests for the routing fix are well-targeted (chain, parallel-split, dangling-input-still-rejected, transform-misattribution, duplicate-across-transform-location). But none of `benchmark/tasks/tier{1,2,3}*.json` exercise `Postgres-CDC` as an actual source — the PostgreSQL-CDC prerequisite content, including the `table-names` issue above, was never exercised end-to-end by the project's own benchmark harness. An `l3: run` Postgres-CDC benchmark task would likely have caught this.
   
   ## What checked out fine
   
   Worth noting since I went looking for problems: the core mechanism `conditional_routing.md` relies on — one `plugin_output` fanning out to multiple `plugin_input` consumers — is a real, tested Zeta engine topology (`LogicalDagGenerator` builds one edge per consumer from a set of target vertices, and it's already exercised by existing E2E configs). The "Zeta SQL supports WHERE only, no GROUP BY/JOIN/ORDER BY" claim is accurate (`ZetaSQLEngine` explicitly throws for all three). `slot.name`, `decoding.plugin.name`, the plural-only `database-names`, and the `debezium { }` passthrough map all check out against source too.
   
   Also: the PR description's PostgreSQL-CDC summary ("schema-qualified `table-name` (`schema.table`, not MySQL-style `db.table`)") doesn't match what's actually shipped (plural `table-names`, and — per the finding above — `schema.table` is in fact accepted). Looks like the description predates the self-correcting second commit; worth syncing before merge since it'll likely become the squash-merge commit message.
   
   ---
   
   Net: the routing-validation fix and the wide-DAG/conditional-routing wiring content look solid and are backed by real, tested engine behavior. The PostgreSQL-CDC `table-names` guidance should be corrected before merge though, since it actively points away from the connector's own canonical format — which cuts against this PR's own stated goal of fixing exactly this kind of generation error. The FieldMapper example and the incomplete template fix are worth a pass too.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][CLI] Add conditional-routing skill, PostgreSQL-CDC prerequisites, and wide-DAG wiring rules [seatunnel]

Posted by "SEZ9 (via GitHub)" <gi...@apache.org>.
SEZ9 commented on PR #11659:
URL: https://github.com/apache/seatunnel/pull/11659#issuecomment-5201963791

   @DanielLeens Fixed in head `2794225` — and thanks for framing Issue 1 the way you did: the "weaker models imitate concrete examples over prose caveats" argument from this PR's own commit message applies verbatim, so leaving the Pattern template contradicting the corrected prose would indeed have undermined exactly the fix that just landed.
   
   Applied Option A, plus the two prose lines you located:
   
   - **Pattern template** (`cdc_realtime.md` Pattern block): the singular keys now carry an inline HOCON comment directly above them showing PostgreSQL-CDC's real convention (`database-names = ["<database>"]` / `table-names = ["<database>.<schema>.<table>"]`, plural, list-typed, fully qualified) and pointing to `get_connector_info` for exact keys.
   - **SOP step 4**: no longer enumerates `database-name, table-name` as if universal — now says the database/table selection keys vary by connector, with the PostgreSQL plural forms called out.
   - **Constraints**: the hyphens-not-underscores rule now uses `database-name`/`database-names` as its example instead of asserting the singular form as the one true convention.
   
   The remaining singular keys in the generic MySQL-oriented prose (lines 50–54) are untouched: they're accurate for the MySQL-CDC family the file's generic sections describe, and rewriting them is beyond what this PR should carry.
   
   One process note for transparency: while pushing this round I force-pushed over a `Merge branch 'dev'` sync commit that had been added to the branch head. No review changes were lost — it contained only the dev-side sync (the HugeGraph connector merge) — and the PR is mergeable against current dev. Apologies for the noise; subsequent updates will fast-forward.
   
   Full suite: 76 passed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][CLI] Add conditional-routing skill, PostgreSQL-CDC prerequisites, and wide-DAG wiring rules [seatunnel]

Posted by "SEZ9 (via GitHub)" <gi...@apache.org>.
SEZ9 commented on PR #11659:
URL: https://github.com/apache/seatunnel/pull/11659#issuecomment-5242513101

   @DanielLeens All four items fixed in `14e8d1a` (fast-forward on the current head this time). First — apologies for leaving your 2026-08-06T13:13 comment unanswered for four days; the findings were correct then and re-confirmed now, and the formal state flip was the right call.
   
   **Issue 1 (blocker) — PostgreSQL-CDC format guidance corrected, direction reversed**: the section now states the canonical two-part `schema.table` form (citing the connector's own error message), notes that three-part entries are accepted but the leading database segment is silently dropped without filtering, and points database selection at `database-names` instead. The comparison-table row now reads `schema.table (canonical)`. You were right that my round-1 "fix" pointed the wrong way — I corrected it from the docs and stopped at the option definitions without reading `PostgresSourceConfigFactory`'s actual split-handling. Parser behavior beats documentation; lesson applied.
   
   **Issue 2 (blocker) — template body actually fixed this time**: the Pattern template now uses `database-names = ["<database>"]` / `table-names = [...]` with a one-line inline note for the per-connector entry formats (`database.table` for MySQL-CDC, `schema.table` for PostgreSQL-CDC). The caveat comment that papered over the singular keys is gone along with the singular keys themselves.
   
   **Issue 3 — FieldMapper direction aligned**: `id = order_id`, consistent with `amount = total` (key = existing input field, value = new output name).
   
   **Issue 4 — wording softened**: `database-names` no longer claims to scope monitoring across the list; the text now says the connector connects to the first listed database.
   
   The PR description's PostgreSQL-CDC summary is also synced to match the shipped content (it still carried the round-1 wording, which would have become the squash commit message).
   
   Full suite: 76 passed at the new head.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][CLI] Add conditional-routing skill, PostgreSQL-CDC prerequisites, and wide-DAG wiring rules [seatunnel]

Posted by "SEZ9 (via GitHub)" <gi...@apache.org>.
SEZ9 commented on PR #11659:
URL: https://github.com/apache/seatunnel/pull/11659#issuecomment-5190874001

   @DanielLeens Thanks for the source-verified review — the PostgreSQL-CDC findings were all correct and are fixed in head `aaaf0ec`:
   
   **Issue 2 (blocker) — PostgreSQL-CDC option keys/format corrected** (Option A, verified against `CatalogOptions.java`, `JdbcSourceOptions.java`, `PostgresIncrementalSourceOptions.java`, and the shipped PostgreSQL-CDC doc):
   - `table-name "schema.table"` → **`table-names` (plural, list-typed) with fully qualified `database.schema.table` entries** (e.g. `["postgres_cdc.inventory.orders"]`)
   - `database-name` → **`database-names` (plural, list-typed)**
   - the fabricated `publication.name` top-level option → replaced with the real mechanism: **Debezium engine properties pass through the nested `debezium { }` map** (`debezium { publication.name = "my_pub" }`)
   - the comparison-table row now contrasts `table-names` entry formats (`database.table` vs `database.schema.table`), and the section ends by deferring to `get_connector_info` for exact keys, consistent with the rest of the file
   - the items you verified as correct (`wal_level = logical`, per-job `slot.name`, `decoding.plugin.name` default `pgoutput`) are unchanged
   
   You're right that the measured-effect table covers only the routing/chain tasks; the CDC section's key-name accuracy is now source-verified rather than benchmark-verified, and the fair follow-up is a CDC-subset benchmark run once this lands — noted for the umbrella tracking.
   
   **Issue 1 (recommended) — routing diagnostics now carry the real section**: `_validate_routing_pairs` messages use the actual `section.connector` location (`transform.Sql: plugin_input "wrong_label" ...`, `already used by transform.Sql`) instead of hard-coded `source.`/`sink.` prefixes, with two new regression tests covering transform-originated dangling and duplicate labels. Since the function lives in the routing-validation change, the same fix + tests are applied on #11658 (where that code originates) and carried here.
   
   On the `SkillRouter.match()` test suggestion: agreed it's the right lock-in for the non-displacement claim, and noted it's a pre-existing gap for all 9 skills rather than this diff — I'd prefer to add a proper `test_skills.py` covering all skills' trigger routing as a small follow-up rather than growing this PR's scope. Happy to fold a minimal two-case version in here instead if you'd rather see it now.
   
   Full suite: 76 passed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Improve][CLI] Add conditional-routing skill, PostgreSQL-CDC prerequisites, and wide-DAG wiring rules [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo merged PR #11659:
URL: https://github.com/apache/seatunnel/pull/11659


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org