You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "SEZ9 (via GitHub)" <gi...@apache.org> on 2026/08/05 06:48:45 UTC
[I] [Bug][CLI] Routing validation rejects valid transform-mediated topologies and is skipped entirely without pyhocon [seatunnel]
SEZ9 opened a new issue, #11657:
URL: https://github.com/apache/seatunnel/issues/11657
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue) and found no similar issues.
### What happened
Two related defects in the CLI's local routing validation (`validate_hocon` in `seatunnel_cli/agents.py`):
**1. Transform blocks are invisible to plugin_output/plugin_input pairing validation.**
`_extract_connector_blocks_raw` only scans the `source` and `sink` sections. `_validate_routing_pairs` therefore never sees labels emitted by transform blocks, and any valid config where a sink consumes a transform's output is falsely rejected:
```
ERROR: sink.Console: plugin_input "filtered" has no matching plugin_output in any source
```
A minimal valid config that reproduces the false rejection:
```hocon
env { job.mode = "BATCH" }
source { FakeSource {
row.num = 5
schema { fields { id = "bigint" } }
plugin_output = "raw" } }
transform {
Sql {
plugin_input = "raw"
plugin_output = "filtered"
query = "SELECT * FROM raw WHERE id > 1"
}
}
sink { Console { plugin_input = "filtered" } }
```
This affects every transform-mediated topology: chains (source → transform → sink) and predicate splits (one source feeding parallel SQL transforms routed to different sinks). Because `validate_hocon` also drives the CLI's auto-fix loop, the false error sends repair rounds chasing a non-existent problem — often "fixing" a correct config into a wrong one.
**2. Routing validation silently skipped when pyhocon is unavailable.**
The `_validate_routing_pairs` call sites live inside the two config-parsing branches (duplicate-connector raw path / pyhocon path). In an environment without pyhocon and without duplicate connector blocks, neither branch runs, and routing validation is skipped entirely — configs with dangling `plugin_input` labels validate as clean.
### What you expected to happen
- Labels emitted by transform blocks participate in routing validation.
- Genuinely dangling `plugin_input` references are still rejected.
- Routing validation runs regardless of parser availability, since it only needs the raw block text.
### SeaTunnel Version
dev
### SeaTunnel Config
See above.
### Running Command
```
seatunnel "route rows with amount >= 1000 to files and the rest to console" # then /check
```
### Error Exception
See above.
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
Python 3.10+ (seatunnel-cli)
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][CLI] Routing validation rejects valid transform-mediated topologies and is skipped entirely without pyhocon [seatunnel]
Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on issue #11657:
URL: https://github.com/apache/seatunnel/issues/11657#issuecomment-5191585537
Thanks for writing this up clearly.
I checked the current `apache/seatunnel` `dev` branch before classifying it as a normal main-repo bug, and I could not find the referenced `seatunnel_cli/agents.py`, `validate_hocon`, or a `seatunnel-cli/` module in this repository today. So the behavior you describe may belong to a companion CLI codebase or an in-flight branch/PR rather than the current main SeaTunnel tree.
Could you please link the exact branch, PR, or repository path that contains this validator logic? Once the source location is pinned down, the issue itself is well scoped: the first fix should preserve transform-mediated routing as valid and also behave correctly when `pyhocon` is unavailable, without broadening into unrelated CLI changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][CLI] Routing validation rejects valid transform-mediated topologies and is skipped entirely without pyhocon [seatunnel]
Posted by "corgy-w (via GitHub)" <gi...@apache.org>.
corgy-w closed issue #11657: [Bug][CLI] Routing validation rejects valid transform-mediated topologies and is skipped entirely without pyhocon
URL: https://github.com/apache/seatunnel/issues/11657
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org