You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "yzeng1618 (via GitHub)" <gi...@apache.org> on 2026/06/16 10:35:49 UTC
[PR] [Improve][Core] Support transform and value constraint option rules in CLI [seatunnel]
yzeng1618 opened a new pull request, #11109:
URL: https://github.com/apache/seatunnel/pull/11109
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
### Purpose of this pull request
https://github.com/apache/seatunnel/pull/10977
This PR improves SeaTunnel CLI connector metadata handling by supporting transform plugin option rules and value constraints.
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
### Does this PR introduce _any_ user-facing change?
Yes.
Previously, SeaTunnel CLI metadata lookup focused on source and sink plugins, so transform plugin options and value constraints could be missing from generated guidance.
After this change, CLI users can retrieve transform plugin metadata, and generated prompts/details include value constraints such as required non-blank transform SQL queries. This helps the CLI produce more accurate SeaTunnel job configurations.
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released SeaTunnel versions or within the unreleased branches such as dev.
If no, write 'No'.
If you are adding/modifying connector documents, please follow our new specifications: https://github.com/apache/seatunnel/issues/4544.
-->
### How was this patch tested?
Added unit tests
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
If you are adding E2E test cases, maybe refer to https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e/src/test/resources/mysqlcdc_to_mysql.conf, here is a good example.
-->
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the following files are updated:
1. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
2. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in [label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in [seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector [plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improve][Core] Support transform and value constraint option rules in CLI [seatunnel]
Posted by "yzeng1618 (via GitHub)" <gi...@apache.org>.
yzeng1618 commented on PR #11109:
URL: https://github.com/apache/seatunnel/pull/11109#issuecomment-4738447270
Thanks @SEZ9 for the careful review and the concrete suggestions! All of these are addressed in the latest head `ab80b9e` ("Harden CLI transform metadata constraints"). A point-by-point rundown:
**1. Test Coverage Enhancements**
- *Fault tolerance:* `_value_constraint_to_dict()` already guards the condition tree (`tree = constraint.get("conditionTree") or {}` plus an `isinstance(tree, dict)` check), so a malformed/missing tree degrades gracefully to just the `expression`. I've now added a dedicated edge-case test, `test_value_constraint_tolerates_missing_or_malformed_condition_tree`, covering a missing tree, a `None` tree, and a non-dict tree. For unexpected `expectValue` types I added `_constraint_value_to_text()`: strings pass through, structured values are serialized via `json.dumps`, and anything non-serializable falls back to `str()`. This is covered by `test_value_constraint_formats_complex_expect_value_as_json` (a list value renders as `["batch", "streaming"]`), with no unhandled exceptions on any branch.
- *Operator formatting:* Good catch. `_format_value_constraint()` now special-cases the `extension` operator, so it renders `- `query` must not be blank` instead of the awkward `- `query` extension must not be blank`. The prompt test asserts the natural phrasing and explicitly `assertNotIn`s the old one, so the LLM only ever sees the clean wording.
**2. Documentation Updates**
- Updated both `seatunnel-cli/README.md` and `seatunnel-cli/README.zh-CN.md`: a new "Transform Metadata" entry under Features, a clarified Architecture/bundled-metadata section stating that source, sink, and transform plugins all carry full option rules and value constraints, and an updated `/connectors` row.
- Added a note to the interactive `/connectors` output in `cli.py`: "Transform option rules and value constraints are supported during generation."
Happy to tweak the constraint phrasing further if you'd prefer different wording. Thanks again!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improve][Core] Support transform and value constraint option rules in CLI [seatunnel]
Posted by "SEZ9 (via GitHub)" <gi...@apache.org>.
SEZ9 commented on PR #11109:
URL: https://github.com/apache/seatunnel/pull/11109#issuecomment-4727828087
@yzeng1618
Great work on this PR! Extending the metadata lookup to include `transform` plugins and supporting `valueConstraints` is a crucial enhancement. It directly addresses the blind spots in the LLM prompt generation from the initial CLI implementation (#10789) and will significantly reduce runtime errors caused by missing constraints (like blank SQL queries).
The implementation looks solid overall. To make this feature even more robust and complete, here are a few suggestions regarding tests and documentation:
**1. Test Coverage Enhancements:**
While the current unit tests (`test_connector_metadata.py`) excellently cover the happy paths, I suggest adding a few edge-case tests for `_value_constraint_to_dict()` and `_format_value_constraint()`. Specifically:
- **Fault tolerance:** What happens if the `conditionTree` is malformed, missing entirely, or if `expectValue` is an unexpected type? Ensuring graceful degradation without throwing unhandled exceptions will make the CLI more resilient.
- **Operator formatting:** Currently, custom operators like `"extension"` might produce slightly awkward prompt strings (e.g., `` `query` extension must not be blank ``). We might want to ensure the LLM fully understands this phrasing, or eventually map these to more natural language.
**2. Documentation Updates:**
Since this is a user-facing improvement that enhances the capabilities of the CLI, we should update the documentation to reflect it:
- Please update `seatunnel-cli/README.md` and `seatunnel-cli/README.zh-CN.md`. Under the "Features" or "Architecture" sections, explicitly mention that the CLI now provides full metadata, option rules, and constraint validation for **Transform** plugins as well.
- It would also be helpful to add a brief note in the interactive `/connectors` command help section indicating that transform constraints are now fully supported.
Thanks Again!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improve][Core] Support transform and value constraint option rules in CLI [seatunnel]
Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11109:
URL: https://github.com/apache/seatunnel/pull/11109#issuecomment-4729040404
Thanks @SEZ9 for calling out the CLI metadata gap. I rechecked the PR state before replying: from Daniel's side there is still no newer code after the head I already reviewed (`c89c44a`), so I am not starting another full review round on unchanged code. If @yzeng1618 pushes an update or wants clarification on any of the remaining review items, I am happy to re-check the latest head.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improve][Core] Support transform and value constraint option rules in CLI [seatunnel]
Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo merged PR #11109:
URL: https://github.com/apache/seatunnel/pull/11109
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org