You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@seatunnel.apache.org by David Zollo <da...@gmail.com> on 2026/10/10 05:44:08 UTC
[DISCUSS] Introduce an ai-merge label for risk-bounded PRs
Hi guys,
I'd like to start a discussion about a small, opt-in mechanism for reducing
review load on PRs whose risk can be checked mechanically. The reference
implementation is in https://github.com/apache/seatunnel/pull/12706.
Nothing in it is active until we agree and enable it, and merging stays a
dry run until a repository variable is switched on.
*Motivation*
Many PRs are well-bounded and easy to verify: documentation fixes, extra
connector tests, bug fixes inside one connector, and new connectors that
follow the standard pattern. They wait for the same scarce committer
attention as engine or API changes. A new connector, for example, touches
the same six registration places every time (the module pom, <module>
entries, the seatunnel-dist dependency, plugin-mapping.properties,
config/plugin_config, and the labeler entry), and CI already checks that
registration is complete. At the same time, changes to the engine, the API
or checkpoint state need careful human review and must never be
fast-tracked.
*Proposal*
A new label, ai-merge. A PR may be merged without a further human code
review only if all of the following hold:
1. It belongs to one of five closed classes (below).
2. It hits none of the always-rejected conditions.
3. The label was applied by an authorised maintainer account, after the
current head commit was pushed. Any new push removes the label
automatically.
4. A designated committer has approved the exact head commit.
5. All required CI checks are green on that commit, there are no
unresolved review threads, and GitHub reports the PR as cleanly mergeable.
*The five classes*
Class What qualifies
docs Markdown, docs images, and brand-new example configs.
tests Connector unit tests and connector E2E, without weakening any
existing assertion, timeout or fixture.
connector A fix inside exactly one blast-radius unit (one connector module,
or one JDBC dialect), with a regression test in the same module, and docs
if it adds an option.
new-connector Exactly one brand-new connector module with its tests and
docs, plus the registration it needs: the module pom, <module> entries, the
seatunnel-dist dependency, plugin-mapping.properties, config/plugin_config,
the labeler entry and SPI service lines.
shared A small change to a shared connector module (connector-common, *-base,
JDBC core, the fake, assert and console connectors).
*How registration changes are accepted for a new connector*
- Registration edits must be purely additive (no removed line), and
every added line must be a plain registration of the new module and nothing
else.
- Service entries must name a class that the same PR adds.
- A pom or registration change without a brand-new module in the same PR
is still rejected.
- The new pom may only use artifacts that other poms in the repository
already use. No literal third-party version, no repositories, profiles,
antrun or exec plugins.
- In addition to the base checks, Code style (which runs
check_connector_registration.py) and Dependency licenses must be green.
*An honest limit on dependencies*
The existing license check covers core runtime jars, not the third-party
jars a connector bundles. So a new third-party jar or version in a
connector pom has no mechanical license or supply-chain review today, and
the gate sends those PRs to a human. When I replayed three real merged
connector PRs through the script, the Facebook Ads connector (#12275)
passed. The TikTok Ads (#12283) and PayPal (#12282) PRs were rejected
because their poms pin their own third-party versions. I think that is the
right outcome, but I'd like your view.
*Shared modules need an extra proof*
backend.yml selects tests by changed path, so a change to connector-common
only runs that module's tests, not the tests of the connectors that depend
on it. The shared class therefore stays rejected until we configure the CI
checks that exercise the dependents (AI_MERGE_WIDE_CHECKS). That would need
a small CI change to run the dependents on such PRs, which I would propose
as a separate PR. Even then it needs:
- at most 80 production lines;
- a test inside the same module;
- no changed public or protected signature.
*Always rejected*
- seatunnel-engine, seatunnel-api, seatunnel-core, seatunnel-common,
transforms, formats and translation.
- .github/** (apart from the additive labeler entry of a new connector)
and tools/**, so a PR can never change its own verifier.
- License files.
- Checkpoint, state, split and serializer classes, option or default
changes, and factory changes of existing connectors.
- Security-sensitive, concurrency and classloading code. Inside
brand-new connector files these are warnings instead, but process-level
calls such as System.exit stay blocked.
- Removed or weakened assertions, disabled tests, and any file deletion
or rename.
- PRs over the size caps, from first-time contributors, or containing
text addressed to a reviewer or AI.
*How it is enforced*
- tools/ai_merge_gate.py is a standard-library Python script. It only
reads the PR through the GitHub API and never checks out or runs PR code,
so it is safe under pull_request_target.
- The workflow removes the label on every push, validates who applied
it, and runs a 10-minute sweep of the full merge gate. A squash merge is
pinned to the evaluated commit, so GitHub rejects it if the head moved.
- It has 56 offline tests, each pinning one rule. I also checked that
breaking a rule makes the matching test fail.
*What this is not*
- It does not replace committer approval. A committer must approve the
exact head commit.
- The content checks are line-level heuristics, not code analysis. They
are deliberately conservative, and the AI review must still address the
warnings they raise (for example hot-path readers or writers, and type
mapping changes). I do not claim they catch every risky change.
- I have not exercised the GitHub API paths against a live PR yet, which
is why I propose starting in observe-only mode.
*Proposed rollout*
1. Phase 1: observe only. Post the eligibility report as a PR comment
and compare it with what reviewers decide. No label, no merging.
2. Phase 2: enable the label for docs, tests and single-connector fixes.
3. Phase 3: enable new connectors, then shared modules once the
dependents' CI proof exists.
4. After each phase, review the merged PRs. If a merged PR is reverted
or breaks dev, suspend that class until we have a retrospective.
*Questions I'd like your input on*
1. Is this acceptable under our project's review and merge expectations,
or should every PR still get at least one human code review from a
committer other than the approver?
2. Who should be the approvers and the label-applying accounts, and
should those be different people?
3. For new connectors, is "no new third-party jar or version without
human review" the right boundary, or should we also route new connectors to
a human whenever they handle credentials?
4. Should shared-module changes be allowed at all, given the extra CI
work they need, or stay human-only?
5. Is it acceptable for a bot to perform the merge, or should the
workflow only report "ready" and leave the final click to a committer? This
may need a check with ASF Infra.
6. Are the size caps reasonable (connector fix: at most 250 production
lines and 25 files; new connector: at most 8000 lines and 120 files)?
Please comment on the thread or on the PR. If there are no objections after
a reasonable discussion period, I propose we start with phase 1 only.
Best Regards,
David
Linkedin: https://www.linkedin.com/in/davidzollo
Re: [DISCUSS] Introduce an ai-merge label for risk-bounded PRs
Posted by David Zollo <da...@gmail.com>.
Hi guys,
Thanks, Zhiwei, Jast, Doyeon and Goutam, for the feedback. Following
my earlier revision, the proposal is now limited to advisory risk
triage. The ai-merge label and automated merging are withdrawn. Review
and merge decisions remain with committers under the existing project
process.
Based on the discussion, I suggest the following adjustments:
1. Assess the change without adding risk points simply because the
contributor is new. New-contributor information can help us offer
support, but should not increase the technical risk score.
2. Treat changes to CI behavior, workflow permissions and credential
handling as critical-risk changes requiring careful review. A
documentation change that mentions CI should not automatically receive
the same classification.
3. Keep the output lightweight: risk labels with the reasoning in the
job summary, rather than routine bot comments on every PR. We can
revisit notifications, including Slack, if the evaluation shows a
concrete need.
4. Keep five levels provisionally for manual evaluation. The
historical analysis did not clearly distinguish levels 1–4 by review
effort, so those distinctions should not be treated as validated. If
adjacent levels do not help reviewers, we can simplify them to three
before rollout.
Address the ASF workflow policy before enabling automation. Goutam,
thanks for pointing this out. My earlier statement that an API-only
implementation was safe under pull_request_target was too broad. The
policy says that trigger will be disabled from November 2026. Any
automated version needs a compliant design, such as pull_request plus
workflow_run, with minimal permissions and careful handling of
untrusted PR data. PR-controlled code must not run with write
credentials. I suggest discussing migration of the existing
add-label.yml and notify_test_workflow.yml separately to keep this
proposal focused.
Jast, the hardcoded approver configuration and merge-gate logic were
removed in my earlier revision. The separate question about
collaborator permissions in .asf.yaml remains a PMC matter. Please
clarify if that was the concern you intended to raise.
For the next step, I suggest a small manual evaluation on
representative open PRs, with no workflow enabled. We can share the
classifications, reasoning and disagreements here to identify missed
risks, unnecessary escalation and whether the labels actually help
reviewers. Workflow activation would remain a separate decision after
that evaluation and the outstanding implementation review findings are
addressed.
Does this capture the concerns raised so far? Examples of PRs that the
current rules would misclassify would be particularly helpful.
Best Regards,
David
Linkedin: https://www.linkedin.com/in/davidzollo
On Sun, Oct 11, 2026 at 1:46 AM Goutam <wo...@gmail.com> wrote:
>
> +1 to the triage-only version. few points/thoughts:
>
> 1. I see Infra disables pull_request_target for all ASF repos from November
> 2026 and already warns against using it on workflows that export
> GITHUB_TOKEN (https://infra.apache.org/github-actions-policy.html). We
> could build this on pull_request + workflow_run from the start and migrate
> add-label.yml and notify_test_workflow.yml in the same change. just a
> thought.
>
> 2. For First-time contributors, instead of raising a level, can we list
> this as an informational note? Githuib already flags new authors and paths
> already measure the risk of change itself.
>
> 3. 5 levels seem fine for collecting data. If 2-4 do not separate after a
> few weeks of real PRs, we can collapse to three levels overall ?
>
> Best regards,
> Goutam
>
> On Sat, Oct 10, 2026 07:13 AM, David Zollo <da...@gmail.com> wrote:
>
> > Hi all,
> >
> > Thanks Jast for the review. I'll answer your point first, then restate the
> > proposal, because it has changed substantially since my first mail. Please
> > disregard the merge mechanism described there.
> >
> > *On the hardcoded username*
> >
> > You are right. An early version of the PR defaulted a configuration value (
> > AI_MERGE_APPROVERS, the account whose approval would have allowed a merge)
> > to DanielLeens. A user list in code or defaults is the wrong place for
> > that, and it is gone. The current head of the PR (782c83247d) contains no
> > user name, no approver or label-actor list, and no merge step. I checked
> > the full PR diff for it. Labels and comments are written by the workflow's
> > own GITHUB_TOKEN, so the mechanism does not grant any person the right to
> > add labels or approve workflow runs. The PR also changes nothing in
> > .asf.yaml.
> >
> > Separately, I noticed that DanielLeens appears in the github.collaborators
> > list in .asf.yaml on dev, which I understand grants triage-level rights. If
> > that is the list you meant, it is independent of this proposal. This PR
> > neither depends on it nor changes it, and I think it is a matter for the
> > PMC rather than this thread. Jast, could you confirm which of the two you
> > meant?
> >
> > *What changed since my first mail*
> >
> > The first proposal let some PRs be merged without a further human code
> > review. That is gone. There is no ai-merge label, no sweep, no merge step,
> > and no approver or merge-related configuration. The workflow has permission
> > to write labels and comments only (contents: read, issues: write,
> > pull-requests:
> > write). Review and merge decisions stay entirely with committers.
> >
> > What remains is a risk label on every PR so reviewers can order the queue.
> > Reference implementation: https://github.com/apache/seatunnel/pull/12706
> >
> > *A correction to my first mail*
> >
> > I said that TikTok Ads (#12283) and PayPal (#12282) were rightly rejected
> > by my first design because their poms pin third-party versions. That was
> > wrong. I later checked: the pinned versions (httpclient 4.5.13, httpcore
> > 4.4.16, mockserver 5.14.0) are exactly the versions other poms in the
> > repository already use, and PayPal's only non-additive change was a line in
> > the labeler config. My first design scored on keywords instead of facts.
> > The new version scores whole-PR facts, for example an option key that only
> > moved to another file is not a removal, and an (artifact, version) pair is
> > only "new" if no existing pom uses it.
> >
> > *What it does*
> >
> > On every PR event (opened, pushed, edited, ready for review) it posts one
> > comment, updated in place, showing how the level was computed, and sets
> > exactly one label. Other labels are untouched.
> > Label Meaning
> > risk: 1-trivial No runtime effect: docs, examples, labeler metadata, tests
> > risk: 2-low Contained: a brand-new connector, or one existing connector
> > unit
> > risk: 3-medium Needs a focused read: sizeable change, missing tests or
> > docs, tooling, unclassified path
> > risk: 4-high Touches a contract, a shared connector module, the supply
> > chain, several units, or repository settings
> > risk: 5-critical Platform code: engine, api, core, common, config, formats,
> > transforms, translation, dist, build, CI, tools, license files
> >
> > Committers can then filter, for example label:"risk: 1-trivial" for a quick
> > pass or label:"risk: 5-critical" for PRs that need more than one set of
> > eyes.
> >
> > *How the level is computed*
> >
> > - *Base level* (the blast radius): 1 for docs and tests, 2 for a new
> > connector or one connector unit, 3 or 4 for tooling around the runtime,
> > 4
> > for a shared connector module, 5 for platform code.
> > - *Adjustments* come from facts about the whole PR, each rule counted
> > once. The result is capped at 4 unless the base is 5, so only platform
> > code
> > is critical.
> > - +2: an option key that no longer exists anywhere in the PR, or a
> > changed default or type; a modified checkpoint, split or state class
> > or
> > serialVersionUID; a dependency (artifact, version) that no existing
> > pom uses; an existing test weakened (fewer assertion lines in a
> > file, a
> > newly disabled test, looser numeric tolerance, shrunken fixture,
> > deleted
> > test file); a removed registration entry (module, dist dependency,
> > plugin-mapping.properties, plugin_config); an uninspectable diff;
> > text addressed to a reviewer or AI.
> > - +1: security, concurrency or classloading words in modified
> > existing code; more than 250 production lines (1500 for a new
> > connector);
> > production code changed without a test; several connector units; a
> > new
> > connector or new option without docs; an author who is not yet a
> > contributor.
> > - A PR to a branch other than dev gets a floor of 3.
> > - Lines that are only moved within a PR are ignored. Suggestive findings
> > (hot-path readers or writers, type-mapping changes, credentials words
> > in a
> > brand-new connector, no E2E test) are notes in the comment and never
> > change
> > the level.
> >
> > *What I checked*
> >
> > - 59 offline tests, one per rule. I broke ten rules one at a time and
> > confirmed each break turns a test red.
> > - A real replay: Facebook Ads (#12275), TikTok Ads (#12283) and PayPal
> > (#12282) now score risk: 2-low. The Zendesk sink (#12253) scores risk:
> > 3-medium.
> > - The last 150 merged PRs to dev (#12283 to #12689): 39 trivial, 29 low,
> > 31 medium, 22 high, 29 critical. Critical PRs averaged 4.6 reviews and
> > 4.9
> > comments, against 2.4 to 2.8 reviews for levels 1 to 4. These are raw
> > counts and may include bot activity.
> >
> > *Limits I want to be clear about*
> >
> > - Only level 5 separates clearly from the rest on review effort. Levels
> > 1 to 4 should be read as triage hints, not measured risk.
> > - The checks are line-level heuristics, not code analysis, and can be
> > wrong in both directions. The replay already exposed several wrong
> > scores
> > in my earlier versions, and I expect more once real PRs go through it.
> > - There was no revert in the 150-PR window, so I could not check whether
> > higher levels predict reverts.
> > - The label and comment writing has been tested offline only, not
> > against a live PR.
> > - A new third-party jar in a connector pom has no mechanical license
> > review today, as far as I can tell from checkLicense.sh. The scorer
> > raises the level for such a jar, but a human still has to review the
> > license.
> >
> > *Safety*
> >
> > The script is standard-library Python. It only reads the PR through the
> > GitHub API and never checks out or runs PR code, so it is safe under
> > pull_request_target. It contains no list of users. The worst failure is a
> > wrong label or comment, which anyone can ignore or remove.
> >
> > *Proposed rollout*
> >
> > 1. First, run the script by hand on a few open PRs and post the output
> > on the PR for comparison, with no workflow.
> > 2. Then enable the workflow for new PRs for a few weeks and collect
> > reports of wrong or noisy labels.
> > 3. After that, adjust the thresholds and decide whether the five levels
> > are right.
> >
> > *Questions for the community*
> >
> > 1. Is a risk label useful for how you review, or is it noise?
> > 2. Are five levels the right granularity, and should only platform code
> > be critical?
> > 3. Should a first-time contributor raise the level by one? It is a trust
> > signal, but it could feel unwelcoming.
> > 4. Are there rules you would add, drop or reweight, for example should
> > CI-only changes be critical?
> > 5. Is one bot comment per PR acceptable, or do you prefer labels only?
> > 6. Any concern with a workflow that has write access to issues and pull
> > requests, running from the base branch?
> >
> > Please comment here or on the PR. If there are no objections after a
> > reasonable discussion period, I propose we start with step 1 only.
> >
> >
> >
> >
> > Best Regards,
> > David
> > Linkedin: https://www.linkedin.com/in/davidzollo
> >
> >
> >
> > On Sat, Oct 10, 2026 at 9:03 AM zhangshenghang <sh...@apache.org>
> > wrote:
> >
> > > I think the overall plan is viable. However, I noticed "DanielLeens" is
> > > hardcoded in the whitelist, and this user should not be granted
> > permission
> > > to add labels or approve runs.
> > >
> > > Regards,
> > > Jast (Shenghang)
> > >
> > >
> > > zhiwei niu <zn...@gmail.com> 于2026年10月10日周六 14:22写道:
> > >
> > > > I wonder if we can create a new channel in Slack with AI notifications,
> > > > so that we can have a process connection and manually click on the
> > merge
> > > > without having to discuss compliance with ASF Infa?
> > > >
> > > > Best regards,
> > > > Zhiwei Niu
> > > >
> > > >
> > > > David Zollo <da...@gmail.com> 于2026年10月10日周六 13:44写道:
> > > >
> > > > > Hi guys,
> > > > >
> > > > > I'd like to start a discussion about a small, opt-in mechanism for
> > > > reducing
> > > > > review load on PRs whose risk can be checked mechanically. The
> > > reference
> > > > > implementation is in https://github.com/apache/seatunnel/pull/12706.
> > > > > Nothing in it is active until we agree and enable it, and merging
> > > stays a
> > > > > dry run until a repository variable is switched on.
> > > > >
> > > > > *Motivation*
> > > > >
> > > > > Many PRs are well-bounded and easy to verify: documentation fixes,
> > > extra
> > > > > connector tests, bug fixes inside one connector, and new connectors
> > > that
> > > > > follow the standard pattern. They wait for the same scarce committer
> > > > > attention as engine or API changes. A new connector, for example,
> > > touches
> > > > > the same six registration places every time (the module pom, <module>
> > > > > entries, the seatunnel-dist dependency, plugin-mapping.properties,
> > > > > config/plugin_config, and the labeler entry), and CI already checks
> > > that
> > > > > registration is complete. At the same time, changes to the engine,
> > the
> > > > API
> > > > > or checkpoint state need careful human review and must never be
> > > > > fast-tracked.
> > > > >
> > > > > *Proposal*
> > > > >
> > > > > A new label, ai-merge. A PR may be merged without a further human
> > code
> > > > > review only if all of the following hold:
> > > > >
> > > > > 1. It belongs to one of five closed classes (below).
> > > > > 2. It hits none of the always-rejected conditions.
> > > > > 3. The label was applied by an authorised maintainer account,
> > after
> > > > the
> > > > > current head commit was pushed. Any new push removes the label
> > > > > automatically.
> > > > > 4. A designated committer has approved the exact head commit.
> > > > > 5. All required CI checks are green on that commit, there are no
> > > > > unresolved review threads, and GitHub reports the PR as cleanly
> > > > > mergeable.
> > > > >
> > > > > *The five classes*
> > > > > Class What qualifies
> > > > > docs Markdown, docs images, and brand-new example configs.
> > > > > tests Connector unit tests and connector E2E, without weakening any
> > > > > existing assertion, timeout or fixture.
> > > > > connector A fix inside exactly one blast-radius unit (one connector
> > > > module,
> > > > > or one JDBC dialect), with a regression test in the same module, and
> > > docs
> > > > > if it adds an option.
> > > > > new-connector Exactly one brand-new connector module with its tests
> > and
> > > > > docs, plus the registration it needs: the module pom, <module>
> > entries,
> > > > the
> > > > > seatunnel-dist dependency, plugin-mapping.properties,
> > > > config/plugin_config,
> > > > > the labeler entry and SPI service lines.
> > > > > shared A small change to a shared connector module (connector-common,
> > > > > *-base,
> > > > > JDBC core, the fake, assert and console connectors).
> > > > >
> > > > > *How registration changes are accepted for a new connector*
> > > > >
> > > > > - Registration edits must be purely additive (no removed line),
> > and
> > > > > every added line must be a plain registration of the new module
> > and
> > > > > nothing
> > > > > else.
> > > > > - Service entries must name a class that the same PR adds.
> > > > > - A pom or registration change without a brand-new module in the
> > > same
> > > > PR
> > > > > is still rejected.
> > > > > - The new pom may only use artifacts that other poms in the
> > > repository
> > > > > already use. No literal third-party version, no repositories,
> > > > profiles,
> > > > > antrun or exec plugins.
> > > > > - In addition to the base checks, Code style (which runs
> > > > > check_connector_registration.py) and Dependency licenses must be
> > > > green.
> > > > >
> > > > > *An honest limit on dependencies*
> > > > >
> > > > > The existing license check covers core runtime jars, not the
> > > third-party
> > > > > jars a connector bundles. So a new third-party jar or version in a
> > > > > connector pom has no mechanical license or supply-chain review today,
> > > and
> > > > > the gate sends those PRs to a human. When I replayed three real
> > merged
> > > > > connector PRs through the script, the Facebook Ads connector (#12275)
> > > > > passed. The TikTok Ads (#12283) and PayPal (#12282) PRs were rejected
> > > > > because their poms pin their own third-party versions. I think that
> > is
> > > > the
> > > > > right outcome, but I'd like your view.
> > > > >
> > > > > *Shared modules need an extra proof*
> > > > >
> > > > > backend.yml selects tests by changed path, so a change to
> > > > connector-common
> > > > > only runs that module's tests, not the tests of the connectors that
> > > > depend
> > > > > on it. The shared class therefore stays rejected until we configure
> > the
> > > > CI
> > > > > checks that exercise the dependents (AI_MERGE_WIDE_CHECKS). That
> > would
> > > > need
> > > > > a small CI change to run the dependents on such PRs, which I would
> > > > propose
> > > > > as a separate PR. Even then it needs:
> > > > >
> > > > > - at most 80 production lines;
> > > > > - a test inside the same module;
> > > > > - no changed public or protected signature.
> > > > >
> > > > > *Always rejected*
> > > > >
> > > > > - seatunnel-engine, seatunnel-api, seatunnel-core,
> > seatunnel-common,
> > > > > transforms, formats and translation.
> > > > > - .github/** (apart from the additive labeler entry of a new
> > > > connector)
> > > > > and tools/**, so a PR can never change its own verifier.
> > > > > - License files.
> > > > > - Checkpoint, state, split and serializer classes, option or
> > default
> > > > > changes, and factory changes of existing connectors.
> > > > > - Security-sensitive, concurrency and classloading code. Inside
> > > > > brand-new connector files these are warnings instead, but
> > > > process-level
> > > > > calls such as System.exit stay blocked.
> > > > > - Removed or weakened assertions, disabled tests, and any file
> > > > deletion
> > > > > or rename.
> > > > > - PRs over the size caps, from first-time contributors, or
> > > containing
> > > > > text addressed to a reviewer or AI.
> > > > >
> > > > > *How it is enforced*
> > > > >
> > > > > - tools/ai_merge_gate.py is a standard-library Python script. It
> > > only
> > > > > reads the PR through the GitHub API and never checks out or runs
> > PR
> > > > > code,
> > > > > so it is safe under pull_request_target.
> > > > > - The workflow removes the label on every push, validates who
> > > applied
> > > > > it, and runs a 10-minute sweep of the full merge gate. A squash
> > > merge
> > > > is
> > > > > pinned to the evaluated commit, so GitHub rejects it if the head
> > > > moved.
> > > > > - It has 56 offline tests, each pinning one rule. I also checked
> > > that
> > > > > breaking a rule makes the matching test fail.
> > > > >
> > > > > *What this is not*
> > > > >
> > > > > - It does not replace committer approval. A committer must approve
> > > the
> > > > > exact head commit.
> > > > > - The content checks are line-level heuristics, not code analysis.
> > > > They
> > > > > are deliberately conservative, and the AI review must still
> > address
> > > > the
> > > > > warnings they raise (for example hot-path readers or writers, and
> > > type
> > > > > mapping changes). I do not claim they catch every risky change.
> > > > > - I have not exercised the GitHub API paths against a live PR yet,
> > > > which
> > > > > is why I propose starting in observe-only mode.
> > > > >
> > > > > *Proposed rollout*
> > > > >
> > > > > 1. Phase 1: observe only. Post the eligibility report as a PR
> > > comment
> > > > > and compare it with what reviewers decide. No label, no merging.
> > > > > 2. Phase 2: enable the label for docs, tests and single-connector
> > > > fixes.
> > > > > 3. Phase 3: enable new connectors, then shared modules once the
> > > > > dependents' CI proof exists.
> > > > > 4. After each phase, review the merged PRs. If a merged PR is
> > > reverted
> > > > > or breaks dev, suspend that class until we have a retrospective.
> > > > >
> > > > > *Questions I'd like your input on*
> > > > >
> > > > > 1. Is this acceptable under our project's review and merge
> > > > expectations,
> > > > > or should every PR still get at least one human code review from a
> > > > > committer other than the approver?
> > > > > 2. Who should be the approvers and the label-applying accounts,
> > and
> > > > > should those be different people?
> > > > > 3. For new connectors, is "no new third-party jar or version
> > without
> > > > > human review" the right boundary, or should we also route new
> > > > > connectors to
> > > > > a human whenever they handle credentials?
> > > > > 4. Should shared-module changes be allowed at all, given the extra
> > > CI
> > > > > work they need, or stay human-only?
> > > > > 5. Is it acceptable for a bot to perform the merge, or should the
> > > > > workflow only report "ready" and leave the final click to a
> > > committer?
> > > > > This
> > > > > may need a check with ASF Infra.
> > > > > 6. Are the size caps reasonable (connector fix: at most 250
> > > production
> > > > > lines and 25 files; new connector: at most 8000 lines and 120
> > > files)?
> > > > >
> > > > > Please comment on the thread or on the PR. If there are no objections
> > > > after
> > > > > a reasonable discussion period, I propose we start with phase 1 only.
> > > > >
> > > > > Best Regards,
> > > > >
> > > > > David
> > > > > Linkedin: https://www.linkedin.com/in/davidzollo
> > > > >
> > > >
> > >
> >
Re: [DISCUSS] Introduce an ai-merge label for risk-bounded PRs
Posted by Goutam <wo...@gmail.com>.
+1 to the triage-only version. few points/thoughts:
1. I see Infra disables pull_request_target for all ASF repos from November
2026 and already warns against using it on workflows that export
GITHUB_TOKEN (https://infra.apache.org/github-actions-policy.html). We
could build this on pull_request + workflow_run from the start and migrate
add-label.yml and notify_test_workflow.yml in the same change. just a
thought.
2. For First-time contributors, instead of raising a level, can we list
this as an informational note? Githuib already flags new authors and paths
already measure the risk of change itself.
3. 5 levels seem fine for collecting data. If 2-4 do not separate after a
few weeks of real PRs, we can collapse to three levels overall ?
Best regards,
Goutam
On Sat, Oct 10, 2026 07:13 AM, David Zollo <da...@gmail.com> wrote:
> Hi all,
>
> Thanks Jast for the review. I'll answer your point first, then restate the
> proposal, because it has changed substantially since my first mail. Please
> disregard the merge mechanism described there.
>
> *On the hardcoded username*
>
> You are right. An early version of the PR defaulted a configuration value (
> AI_MERGE_APPROVERS, the account whose approval would have allowed a merge)
> to DanielLeens. A user list in code or defaults is the wrong place for
> that, and it is gone. The current head of the PR (782c83247d) contains no
> user name, no approver or label-actor list, and no merge step. I checked
> the full PR diff for it. Labels and comments are written by the workflow's
> own GITHUB_TOKEN, so the mechanism does not grant any person the right to
> add labels or approve workflow runs. The PR also changes nothing in
> .asf.yaml.
>
> Separately, I noticed that DanielLeens appears in the github.collaborators
> list in .asf.yaml on dev, which I understand grants triage-level rights. If
> that is the list you meant, it is independent of this proposal. This PR
> neither depends on it nor changes it, and I think it is a matter for the
> PMC rather than this thread. Jast, could you confirm which of the two you
> meant?
>
> *What changed since my first mail*
>
> The first proposal let some PRs be merged without a further human code
> review. That is gone. There is no ai-merge label, no sweep, no merge step,
> and no approver or merge-related configuration. The workflow has permission
> to write labels and comments only (contents: read, issues: write,
> pull-requests:
> write). Review and merge decisions stay entirely with committers.
>
> What remains is a risk label on every PR so reviewers can order the queue.
> Reference implementation: https://github.com/apache/seatunnel/pull/12706
>
> *A correction to my first mail*
>
> I said that TikTok Ads (#12283) and PayPal (#12282) were rightly rejected
> by my first design because their poms pin third-party versions. That was
> wrong. I later checked: the pinned versions (httpclient 4.5.13, httpcore
> 4.4.16, mockserver 5.14.0) are exactly the versions other poms in the
> repository already use, and PayPal's only non-additive change was a line in
> the labeler config. My first design scored on keywords instead of facts.
> The new version scores whole-PR facts, for example an option key that only
> moved to another file is not a removal, and an (artifact, version) pair is
> only "new" if no existing pom uses it.
>
> *What it does*
>
> On every PR event (opened, pushed, edited, ready for review) it posts one
> comment, updated in place, showing how the level was computed, and sets
> exactly one label. Other labels are untouched.
> Label Meaning
> risk: 1-trivial No runtime effect: docs, examples, labeler metadata, tests
> risk: 2-low Contained: a brand-new connector, or one existing connector
> unit
> risk: 3-medium Needs a focused read: sizeable change, missing tests or
> docs, tooling, unclassified path
> risk: 4-high Touches a contract, a shared connector module, the supply
> chain, several units, or repository settings
> risk: 5-critical Platform code: engine, api, core, common, config, formats,
> transforms, translation, dist, build, CI, tools, license files
>
> Committers can then filter, for example label:"risk: 1-trivial" for a quick
> pass or label:"risk: 5-critical" for PRs that need more than one set of
> eyes.
>
> *How the level is computed*
>
> - *Base level* (the blast radius): 1 for docs and tests, 2 for a new
> connector or one connector unit, 3 or 4 for tooling around the runtime,
> 4
> for a shared connector module, 5 for platform code.
> - *Adjustments* come from facts about the whole PR, each rule counted
> once. The result is capped at 4 unless the base is 5, so only platform
> code
> is critical.
> - +2: an option key that no longer exists anywhere in the PR, or a
> changed default or type; a modified checkpoint, split or state class
> or
> serialVersionUID; a dependency (artifact, version) that no existing
> pom uses; an existing test weakened (fewer assertion lines in a
> file, a
> newly disabled test, looser numeric tolerance, shrunken fixture,
> deleted
> test file); a removed registration entry (module, dist dependency,
> plugin-mapping.properties, plugin_config); an uninspectable diff;
> text addressed to a reviewer or AI.
> - +1: security, concurrency or classloading words in modified
> existing code; more than 250 production lines (1500 for a new
> connector);
> production code changed without a test; several connector units; a
> new
> connector or new option without docs; an author who is not yet a
> contributor.
> - A PR to a branch other than dev gets a floor of 3.
> - Lines that are only moved within a PR are ignored. Suggestive findings
> (hot-path readers or writers, type-mapping changes, credentials words
> in a
> brand-new connector, no E2E test) are notes in the comment and never
> change
> the level.
>
> *What I checked*
>
> - 59 offline tests, one per rule. I broke ten rules one at a time and
> confirmed each break turns a test red.
> - A real replay: Facebook Ads (#12275), TikTok Ads (#12283) and PayPal
> (#12282) now score risk: 2-low. The Zendesk sink (#12253) scores risk:
> 3-medium.
> - The last 150 merged PRs to dev (#12283 to #12689): 39 trivial, 29 low,
> 31 medium, 22 high, 29 critical. Critical PRs averaged 4.6 reviews and
> 4.9
> comments, against 2.4 to 2.8 reviews for levels 1 to 4. These are raw
> counts and may include bot activity.
>
> *Limits I want to be clear about*
>
> - Only level 5 separates clearly from the rest on review effort. Levels
> 1 to 4 should be read as triage hints, not measured risk.
> - The checks are line-level heuristics, not code analysis, and can be
> wrong in both directions. The replay already exposed several wrong
> scores
> in my earlier versions, and I expect more once real PRs go through it.
> - There was no revert in the 150-PR window, so I could not check whether
> higher levels predict reverts.
> - The label and comment writing has been tested offline only, not
> against a live PR.
> - A new third-party jar in a connector pom has no mechanical license
> review today, as far as I can tell from checkLicense.sh. The scorer
> raises the level for such a jar, but a human still has to review the
> license.
>
> *Safety*
>
> The script is standard-library Python. It only reads the PR through the
> GitHub API and never checks out or runs PR code, so it is safe under
> pull_request_target. It contains no list of users. The worst failure is a
> wrong label or comment, which anyone can ignore or remove.
>
> *Proposed rollout*
>
> 1. First, run the script by hand on a few open PRs and post the output
> on the PR for comparison, with no workflow.
> 2. Then enable the workflow for new PRs for a few weeks and collect
> reports of wrong or noisy labels.
> 3. After that, adjust the thresholds and decide whether the five levels
> are right.
>
> *Questions for the community*
>
> 1. Is a risk label useful for how you review, or is it noise?
> 2. Are five levels the right granularity, and should only platform code
> be critical?
> 3. Should a first-time contributor raise the level by one? It is a trust
> signal, but it could feel unwelcoming.
> 4. Are there rules you would add, drop or reweight, for example should
> CI-only changes be critical?
> 5. Is one bot comment per PR acceptable, or do you prefer labels only?
> 6. Any concern with a workflow that has write access to issues and pull
> requests, running from the base branch?
>
> Please comment here or on the PR. If there are no objections after a
> reasonable discussion period, I propose we start with step 1 only.
>
>
>
>
> Best Regards,
> David
> Linkedin: https://www.linkedin.com/in/davidzollo
>
>
>
> On Sat, Oct 10, 2026 at 9:03 AM zhangshenghang <sh...@apache.org>
> wrote:
>
> > I think the overall plan is viable. However, I noticed "DanielLeens" is
> > hardcoded in the whitelist, and this user should not be granted
> permission
> > to add labels or approve runs.
> >
> > Regards,
> > Jast (Shenghang)
> >
> >
> > zhiwei niu <zn...@gmail.com> 于2026年10月10日周六 14:22写道:
> >
> > > I wonder if we can create a new channel in Slack with AI notifications,
> > > so that we can have a process connection and manually click on the
> merge
> > > without having to discuss compliance with ASF Infa?
> > >
> > > Best regards,
> > > Zhiwei Niu
> > >
> > >
> > > David Zollo <da...@gmail.com> 于2026年10月10日周六 13:44写道:
> > >
> > > > Hi guys,
> > > >
> > > > I'd like to start a discussion about a small, opt-in mechanism for
> > > reducing
> > > > review load on PRs whose risk can be checked mechanically. The
> > reference
> > > > implementation is in https://github.com/apache/seatunnel/pull/12706.
> > > > Nothing in it is active until we agree and enable it, and merging
> > stays a
> > > > dry run until a repository variable is switched on.
> > > >
> > > > *Motivation*
> > > >
> > > > Many PRs are well-bounded and easy to verify: documentation fixes,
> > extra
> > > > connector tests, bug fixes inside one connector, and new connectors
> > that
> > > > follow the standard pattern. They wait for the same scarce committer
> > > > attention as engine or API changes. A new connector, for example,
> > touches
> > > > the same six registration places every time (the module pom, <module>
> > > > entries, the seatunnel-dist dependency, plugin-mapping.properties,
> > > > config/plugin_config, and the labeler entry), and CI already checks
> > that
> > > > registration is complete. At the same time, changes to the engine,
> the
> > > API
> > > > or checkpoint state need careful human review and must never be
> > > > fast-tracked.
> > > >
> > > > *Proposal*
> > > >
> > > > A new label, ai-merge. A PR may be merged without a further human
> code
> > > > review only if all of the following hold:
> > > >
> > > > 1. It belongs to one of five closed classes (below).
> > > > 2. It hits none of the always-rejected conditions.
> > > > 3. The label was applied by an authorised maintainer account,
> after
> > > the
> > > > current head commit was pushed. Any new push removes the label
> > > > automatically.
> > > > 4. A designated committer has approved the exact head commit.
> > > > 5. All required CI checks are green on that commit, there are no
> > > > unresolved review threads, and GitHub reports the PR as cleanly
> > > > mergeable.
> > > >
> > > > *The five classes*
> > > > Class What qualifies
> > > > docs Markdown, docs images, and brand-new example configs.
> > > > tests Connector unit tests and connector E2E, without weakening any
> > > > existing assertion, timeout or fixture.
> > > > connector A fix inside exactly one blast-radius unit (one connector
> > > module,
> > > > or one JDBC dialect), with a regression test in the same module, and
> > docs
> > > > if it adds an option.
> > > > new-connector Exactly one brand-new connector module with its tests
> and
> > > > docs, plus the registration it needs: the module pom, <module>
> entries,
> > > the
> > > > seatunnel-dist dependency, plugin-mapping.properties,
> > > config/plugin_config,
> > > > the labeler entry and SPI service lines.
> > > > shared A small change to a shared connector module (connector-common,
> > > > *-base,
> > > > JDBC core, the fake, assert and console connectors).
> > > >
> > > > *How registration changes are accepted for a new connector*
> > > >
> > > > - Registration edits must be purely additive (no removed line),
> and
> > > > every added line must be a plain registration of the new module
> and
> > > > nothing
> > > > else.
> > > > - Service entries must name a class that the same PR adds.
> > > > - A pom or registration change without a brand-new module in the
> > same
> > > PR
> > > > is still rejected.
> > > > - The new pom may only use artifacts that other poms in the
> > repository
> > > > already use. No literal third-party version, no repositories,
> > > profiles,
> > > > antrun or exec plugins.
> > > > - In addition to the base checks, Code style (which runs
> > > > check_connector_registration.py) and Dependency licenses must be
> > > green.
> > > >
> > > > *An honest limit on dependencies*
> > > >
> > > > The existing license check covers core runtime jars, not the
> > third-party
> > > > jars a connector bundles. So a new third-party jar or version in a
> > > > connector pom has no mechanical license or supply-chain review today,
> > and
> > > > the gate sends those PRs to a human. When I replayed three real
> merged
> > > > connector PRs through the script, the Facebook Ads connector (#12275)
> > > > passed. The TikTok Ads (#12283) and PayPal (#12282) PRs were rejected
> > > > because their poms pin their own third-party versions. I think that
> is
> > > the
> > > > right outcome, but I'd like your view.
> > > >
> > > > *Shared modules need an extra proof*
> > > >
> > > > backend.yml selects tests by changed path, so a change to
> > > connector-common
> > > > only runs that module's tests, not the tests of the connectors that
> > > depend
> > > > on it. The shared class therefore stays rejected until we configure
> the
> > > CI
> > > > checks that exercise the dependents (AI_MERGE_WIDE_CHECKS). That
> would
> > > need
> > > > a small CI change to run the dependents on such PRs, which I would
> > > propose
> > > > as a separate PR. Even then it needs:
> > > >
> > > > - at most 80 production lines;
> > > > - a test inside the same module;
> > > > - no changed public or protected signature.
> > > >
> > > > *Always rejected*
> > > >
> > > > - seatunnel-engine, seatunnel-api, seatunnel-core,
> seatunnel-common,
> > > > transforms, formats and translation.
> > > > - .github/** (apart from the additive labeler entry of a new
> > > connector)
> > > > and tools/**, so a PR can never change its own verifier.
> > > > - License files.
> > > > - Checkpoint, state, split and serializer classes, option or
> default
> > > > changes, and factory changes of existing connectors.
> > > > - Security-sensitive, concurrency and classloading code. Inside
> > > > brand-new connector files these are warnings instead, but
> > > process-level
> > > > calls such as System.exit stay blocked.
> > > > - Removed or weakened assertions, disabled tests, and any file
> > > deletion
> > > > or rename.
> > > > - PRs over the size caps, from first-time contributors, or
> > containing
> > > > text addressed to a reviewer or AI.
> > > >
> > > > *How it is enforced*
> > > >
> > > > - tools/ai_merge_gate.py is a standard-library Python script. It
> > only
> > > > reads the PR through the GitHub API and never checks out or runs
> PR
> > > > code,
> > > > so it is safe under pull_request_target.
> > > > - The workflow removes the label on every push, validates who
> > applied
> > > > it, and runs a 10-minute sweep of the full merge gate. A squash
> > merge
> > > is
> > > > pinned to the evaluated commit, so GitHub rejects it if the head
> > > moved.
> > > > - It has 56 offline tests, each pinning one rule. I also checked
> > that
> > > > breaking a rule makes the matching test fail.
> > > >
> > > > *What this is not*
> > > >
> > > > - It does not replace committer approval. A committer must approve
> > the
> > > > exact head commit.
> > > > - The content checks are line-level heuristics, not code analysis.
> > > They
> > > > are deliberately conservative, and the AI review must still
> address
> > > the
> > > > warnings they raise (for example hot-path readers or writers, and
> > type
> > > > mapping changes). I do not claim they catch every risky change.
> > > > - I have not exercised the GitHub API paths against a live PR yet,
> > > which
> > > > is why I propose starting in observe-only mode.
> > > >
> > > > *Proposed rollout*
> > > >
> > > > 1. Phase 1: observe only. Post the eligibility report as a PR
> > comment
> > > > and compare it with what reviewers decide. No label, no merging.
> > > > 2. Phase 2: enable the label for docs, tests and single-connector
> > > fixes.
> > > > 3. Phase 3: enable new connectors, then shared modules once the
> > > > dependents' CI proof exists.
> > > > 4. After each phase, review the merged PRs. If a merged PR is
> > reverted
> > > > or breaks dev, suspend that class until we have a retrospective.
> > > >
> > > > *Questions I'd like your input on*
> > > >
> > > > 1. Is this acceptable under our project's review and merge
> > > expectations,
> > > > or should every PR still get at least one human code review from a
> > > > committer other than the approver?
> > > > 2. Who should be the approvers and the label-applying accounts,
> and
> > > > should those be different people?
> > > > 3. For new connectors, is "no new third-party jar or version
> without
> > > > human review" the right boundary, or should we also route new
> > > > connectors to
> > > > a human whenever they handle credentials?
> > > > 4. Should shared-module changes be allowed at all, given the extra
> > CI
> > > > work they need, or stay human-only?
> > > > 5. Is it acceptable for a bot to perform the merge, or should the
> > > > workflow only report "ready" and leave the final click to a
> > committer?
> > > > This
> > > > may need a check with ASF Infra.
> > > > 6. Are the size caps reasonable (connector fix: at most 250
> > production
> > > > lines and 25 files; new connector: at most 8000 lines and 120
> > files)?
> > > >
> > > > Please comment on the thread or on the PR. If there are no objections
> > > after
> > > > a reasonable discussion period, I propose we start with phase 1 only.
> > > >
> > > > Best Regards,
> > > >
> > > > David
> > > > Linkedin: https://www.linkedin.com/in/davidzollo
> > > >
> > >
> >
>
Re: [DISCUSS] Introduce an ai-merge label for risk-bounded PRs
Posted by Doyeon Kim <ek...@gmail.com>.
+1 to the revised proposal, thanks for the great work. Starting with
step 1 sounds good to me.
A few thoughts on the questions:
- Q3: I don't think a first-time contributor needs +1. GitHub already
shows a "First-time contributor" badge on the PR, so reviewers can
see it anyway.
- Q4: Agreed that CI changes should stay critical, especially when
they change how CI behaves.
- Q5: I'd prefer labels only. If the reasoning needs to be visible,
the workflow's job summary could hold it instead of a PR comment.
Best regards,
Doyeon Kim
On Sat, Oct 10, 2026 at 11:11 PM David Zollo <da...@gmail.com>
wrote:
> Hi all,
>
> Thanks Jast for the review. I'll answer your point first, then restate the
> proposal, because it has changed substantially since my first mail. Please
> disregard the merge mechanism described there.
>
> *On the hardcoded username*
>
> You are right. An early version of the PR defaulted a configuration value (
> AI_MERGE_APPROVERS, the account whose approval would have allowed a merge)
> to DanielLeens. A user list in code or defaults is the wrong place for
> that, and it is gone. The current head of the PR (782c83247d) contains no
> user name, no approver or label-actor list, and no merge step. I checked
> the full PR diff for it. Labels and comments are written by the workflow's
> own GITHUB_TOKEN, so the mechanism does not grant any person the right to
> add labels or approve workflow runs. The PR also changes nothing in
> .asf.yaml.
>
> Separately, I noticed that DanielLeens appears in the github.collaborators
> list in .asf.yaml on dev, which I understand grants triage-level rights. If
> that is the list you meant, it is independent of this proposal. This PR
> neither depends on it nor changes it, and I think it is a matter for the
> PMC rather than this thread. Jast, could you confirm which of the two you
> meant?
>
> *What changed since my first mail*
>
> The first proposal let some PRs be merged without a further human code
> review. That is gone. There is no ai-merge label, no sweep, no merge step,
> and no approver or merge-related configuration. The workflow has permission
> to write labels and comments only (contents: read, issues: write,
> pull-requests:
> write). Review and merge decisions stay entirely with committers.
>
> What remains is a risk label on every PR so reviewers can order the queue.
> Reference implementation: https://github.com/apache/seatunnel/pull/12706
>
> *A correction to my first mail*
>
> I said that TikTok Ads (#12283) and PayPal (#12282) were rightly rejected
> by my first design because their poms pin third-party versions. That was
> wrong. I later checked: the pinned versions (httpclient 4.5.13, httpcore
> 4.4.16, mockserver 5.14.0) are exactly the versions other poms in the
> repository already use, and PayPal's only non-additive change was a line in
> the labeler config. My first design scored on keywords instead of facts.
> The new version scores whole-PR facts, for example an option key that only
> moved to another file is not a removal, and an (artifact, version) pair is
> only "new" if no existing pom uses it.
>
> *What it does*
>
> On every PR event (opened, pushed, edited, ready for review) it posts one
> comment, updated in place, showing how the level was computed, and sets
> exactly one label. Other labels are untouched.
> Label Meaning
> risk: 1-trivial No runtime effect: docs, examples, labeler metadata, tests
> risk: 2-low Contained: a brand-new connector, or one existing connector
> unit
> risk: 3-medium Needs a focused read: sizeable change, missing tests or
> docs, tooling, unclassified path
> risk: 4-high Touches a contract, a shared connector module, the supply
> chain, several units, or repository settings
> risk: 5-critical Platform code: engine, api, core, common, config, formats,
> transforms, translation, dist, build, CI, tools, license files
>
> Committers can then filter, for example label:"risk: 1-trivial" for a quick
> pass or label:"risk: 5-critical" for PRs that need more than one set of
> eyes.
>
> *How the level is computed*
>
> - *Base level* (the blast radius): 1 for docs and tests, 2 for a new
> connector or one connector unit, 3 or 4 for tooling around the runtime,
> 4
> for a shared connector module, 5 for platform code.
> - *Adjustments* come from facts about the whole PR, each rule counted
> once. The result is capped at 4 unless the base is 5, so only platform
> code
> is critical.
> - +2: an option key that no longer exists anywhere in the PR, or a
> changed default or type; a modified checkpoint, split or state class
> or
> serialVersionUID; a dependency (artifact, version) that no existing
> pom uses; an existing test weakened (fewer assertion lines in a
> file, a
> newly disabled test, looser numeric tolerance, shrunken fixture,
> deleted
> test file); a removed registration entry (module, dist dependency,
> plugin-mapping.properties, plugin_config); an uninspectable diff;
> text addressed to a reviewer or AI.
> - +1: security, concurrency or classloading words in modified
> existing code; more than 250 production lines (1500 for a new
> connector);
> production code changed without a test; several connector units; a
> new
> connector or new option without docs; an author who is not yet a
> contributor.
> - A PR to a branch other than dev gets a floor of 3.
> - Lines that are only moved within a PR are ignored. Suggestive findings
> (hot-path readers or writers, type-mapping changes, credentials words
> in a
> brand-new connector, no E2E test) are notes in the comment and never
> change
> the level.
>
> *What I checked*
>
> - 59 offline tests, one per rule. I broke ten rules one at a time and
> confirmed each break turns a test red.
> - A real replay: Facebook Ads (#12275), TikTok Ads (#12283) and PayPal
> (#12282) now score risk: 2-low. The Zendesk sink (#12253) scores risk:
> 3-medium.
> - The last 150 merged PRs to dev (#12283 to #12689): 39 trivial, 29 low,
> 31 medium, 22 high, 29 critical. Critical PRs averaged 4.6 reviews and
> 4.9
> comments, against 2.4 to 2.8 reviews for levels 1 to 4. These are raw
> counts and may include bot activity.
>
> *Limits I want to be clear about*
>
> - Only level 5 separates clearly from the rest on review effort. Levels
> 1 to 4 should be read as triage hints, not measured risk.
> - The checks are line-level heuristics, not code analysis, and can be
> wrong in both directions. The replay already exposed several wrong
> scores
> in my earlier versions, and I expect more once real PRs go through it.
> - There was no revert in the 150-PR window, so I could not check whether
> higher levels predict reverts.
> - The label and comment writing has been tested offline only, not
> against a live PR.
> - A new third-party jar in a connector pom has no mechanical license
> review today, as far as I can tell from checkLicense.sh. The scorer
> raises the level for such a jar, but a human still has to review the
> license.
>
> *Safety*
>
> The script is standard-library Python. It only reads the PR through the
> GitHub API and never checks out or runs PR code, so it is safe under
> pull_request_target. It contains no list of users. The worst failure is a
> wrong label or comment, which anyone can ignore or remove.
>
> *Proposed rollout*
>
> 1. First, run the script by hand on a few open PRs and post the output
> on the PR for comparison, with no workflow.
> 2. Then enable the workflow for new PRs for a few weeks and collect
> reports of wrong or noisy labels.
> 3. After that, adjust the thresholds and decide whether the five levels
> are right.
>
> *Questions for the community*
>
> 1. Is a risk label useful for how you review, or is it noise?
> 2. Are five levels the right granularity, and should only platform code
> be critical?
> 3. Should a first-time contributor raise the level by one? It is a trust
> signal, but it could feel unwelcoming.
> 4. Are there rules you would add, drop or reweight, for example should
> CI-only changes be critical?
> 5. Is one bot comment per PR acceptable, or do you prefer labels only?
> 6. Any concern with a workflow that has write access to issues and pull
> requests, running from the base branch?
>
> Please comment here or on the PR. If there are no objections after a
> reasonable discussion period, I propose we start with step 1 only.
>
>
>
>
> Best Regards,
> David
> Linkedin: https://www.linkedin.com/in/davidzollo
>
>
>
> On Sat, Oct 10, 2026 at 9:03 AM zhangshenghang <sh...@apache.org>
> wrote:
>
> > I think the overall plan is viable. However, I noticed "DanielLeens" is
> > hardcoded in the whitelist, and this user should not be granted
> permission
> > to add labels or approve runs.
> >
> > Regards,
> > Jast (Shenghang)
> >
> >
> > zhiwei niu <zn...@gmail.com> 于2026年10月10日周六 14:22写道:
> >
> > > I wonder if we can create a new channel in Slack with AI notifications,
> > > so that we can have a process connection and manually click on the
> merge
> > > without having to discuss compliance with ASF Infa?
> > >
> > > Best regards,
> > > Zhiwei Niu
> > >
> > >
> > > David Zollo <da...@gmail.com> 于2026年10月10日周六 13:44写道:
> > >
> > > > Hi guys,
> > > >
> > > > I'd like to start a discussion about a small, opt-in mechanism for
> > > reducing
> > > > review load on PRs whose risk can be checked mechanically. The
> > reference
> > > > implementation is in https://github.com/apache/seatunnel/pull/12706.
> > > > Nothing in it is active until we agree and enable it, and merging
> > stays a
> > > > dry run until a repository variable is switched on.
> > > >
> > > > *Motivation*
> > > >
> > > > Many PRs are well-bounded and easy to verify: documentation fixes,
> > extra
> > > > connector tests, bug fixes inside one connector, and new connectors
> > that
> > > > follow the standard pattern. They wait for the same scarce committer
> > > > attention as engine or API changes. A new connector, for example,
> > touches
> > > > the same six registration places every time (the module pom, <module>
> > > > entries, the seatunnel-dist dependency, plugin-mapping.properties,
> > > > config/plugin_config, and the labeler entry), and CI already checks
> > that
> > > > registration is complete. At the same time, changes to the engine,
> the
> > > API
> > > > or checkpoint state need careful human review and must never be
> > > > fast-tracked.
> > > >
> > > > *Proposal*
> > > >
> > > > A new label, ai-merge. A PR may be merged without a further human
> code
> > > > review only if all of the following hold:
> > > >
> > > > 1. It belongs to one of five closed classes (below).
> > > > 2. It hits none of the always-rejected conditions.
> > > > 3. The label was applied by an authorised maintainer account,
> after
> > > the
> > > > current head commit was pushed. Any new push removes the label
> > > > automatically.
> > > > 4. A designated committer has approved the exact head commit.
> > > > 5. All required CI checks are green on that commit, there are no
> > > > unresolved review threads, and GitHub reports the PR as cleanly
> > > > mergeable.
> > > >
> > > > *The five classes*
> > > > Class What qualifies
> > > > docs Markdown, docs images, and brand-new example configs.
> > > > tests Connector unit tests and connector E2E, without weakening any
> > > > existing assertion, timeout or fixture.
> > > > connector A fix inside exactly one blast-radius unit (one connector
> > > module,
> > > > or one JDBC dialect), with a regression test in the same module, and
> > docs
> > > > if it adds an option.
> > > > new-connector Exactly one brand-new connector module with its tests
> and
> > > > docs, plus the registration it needs: the module pom, <module>
> entries,
> > > the
> > > > seatunnel-dist dependency, plugin-mapping.properties,
> > > config/plugin_config,
> > > > the labeler entry and SPI service lines.
> > > > shared A small change to a shared connector module (connector-common,
> > > > *-base,
> > > > JDBC core, the fake, assert and console connectors).
> > > >
> > > > *How registration changes are accepted for a new connector*
> > > >
> > > > - Registration edits must be purely additive (no removed line),
> and
> > > > every added line must be a plain registration of the new module
> and
> > > > nothing
> > > > else.
> > > > - Service entries must name a class that the same PR adds.
> > > > - A pom or registration change without a brand-new module in the
> > same
> > > PR
> > > > is still rejected.
> > > > - The new pom may only use artifacts that other poms in the
> > repository
> > > > already use. No literal third-party version, no repositories,
> > > profiles,
> > > > antrun or exec plugins.
> > > > - In addition to the base checks, Code style (which runs
> > > > check_connector_registration.py) and Dependency licenses must be
> > > green.
> > > >
> > > > *An honest limit on dependencies*
> > > >
> > > > The existing license check covers core runtime jars, not the
> > third-party
> > > > jars a connector bundles. So a new third-party jar or version in a
> > > > connector pom has no mechanical license or supply-chain review today,
> > and
> > > > the gate sends those PRs to a human. When I replayed three real
> merged
> > > > connector PRs through the script, the Facebook Ads connector (#12275)
> > > > passed. The TikTok Ads (#12283) and PayPal (#12282) PRs were rejected
> > > > because their poms pin their own third-party versions. I think that
> is
> > > the
> > > > right outcome, but I'd like your view.
> > > >
> > > > *Shared modules need an extra proof*
> > > >
> > > > backend.yml selects tests by changed path, so a change to
> > > connector-common
> > > > only runs that module's tests, not the tests of the connectors that
> > > depend
> > > > on it. The shared class therefore stays rejected until we configure
> the
> > > CI
> > > > checks that exercise the dependents (AI_MERGE_WIDE_CHECKS). That
> would
> > > need
> > > > a small CI change to run the dependents on such PRs, which I would
> > > propose
> > > > as a separate PR. Even then it needs:
> > > >
> > > > - at most 80 production lines;
> > > > - a test inside the same module;
> > > > - no changed public or protected signature.
> > > >
> > > > *Always rejected*
> > > >
> > > > - seatunnel-engine, seatunnel-api, seatunnel-core,
> seatunnel-common,
> > > > transforms, formats and translation.
> > > > - .github/** (apart from the additive labeler entry of a new
> > > connector)
> > > > and tools/**, so a PR can never change its own verifier.
> > > > - License files.
> > > > - Checkpoint, state, split and serializer classes, option or
> default
> > > > changes, and factory changes of existing connectors.
> > > > - Security-sensitive, concurrency and classloading code. Inside
> > > > brand-new connector files these are warnings instead, but
> > > process-level
> > > > calls such as System.exit stay blocked.
> > > > - Removed or weakened assertions, disabled tests, and any file
> > > deletion
> > > > or rename.
> > > > - PRs over the size caps, from first-time contributors, or
> > containing
> > > > text addressed to a reviewer or AI.
> > > >
> > > > *How it is enforced*
> > > >
> > > > - tools/ai_merge_gate.py is a standard-library Python script. It
> > only
> > > > reads the PR through the GitHub API and never checks out or runs
> PR
> > > > code,
> > > > so it is safe under pull_request_target.
> > > > - The workflow removes the label on every push, validates who
> > applied
> > > > it, and runs a 10-minute sweep of the full merge gate. A squash
> > merge
> > > is
> > > > pinned to the evaluated commit, so GitHub rejects it if the head
> > > moved.
> > > > - It has 56 offline tests, each pinning one rule. I also checked
> > that
> > > > breaking a rule makes the matching test fail.
> > > >
> > > > *What this is not*
> > > >
> > > > - It does not replace committer approval. A committer must approve
> > the
> > > > exact head commit.
> > > > - The content checks are line-level heuristics, not code analysis.
> > > They
> > > > are deliberately conservative, and the AI review must still
> address
> > > the
> > > > warnings they raise (for example hot-path readers or writers, and
> > type
> > > > mapping changes). I do not claim they catch every risky change.
> > > > - I have not exercised the GitHub API paths against a live PR yet,
> > > which
> > > > is why I propose starting in observe-only mode.
> > > >
> > > > *Proposed rollout*
> > > >
> > > > 1. Phase 1: observe only. Post the eligibility report as a PR
> > comment
> > > > and compare it with what reviewers decide. No label, no merging.
> > > > 2. Phase 2: enable the label for docs, tests and single-connector
> > > fixes.
> > > > 3. Phase 3: enable new connectors, then shared modules once the
> > > > dependents' CI proof exists.
> > > > 4. After each phase, review the merged PRs. If a merged PR is
> > reverted
> > > > or breaks dev, suspend that class until we have a retrospective.
> > > >
> > > > *Questions I'd like your input on*
> > > >
> > > > 1. Is this acceptable under our project's review and merge
> > > expectations,
> > > > or should every PR still get at least one human code review from a
> > > > committer other than the approver?
> > > > 2. Who should be the approvers and the label-applying accounts,
> and
> > > > should those be different people?
> > > > 3. For new connectors, is "no new third-party jar or version
> without
> > > > human review" the right boundary, or should we also route new
> > > > connectors to
> > > > a human whenever they handle credentials?
> > > > 4. Should shared-module changes be allowed at all, given the extra
> > CI
> > > > work they need, or stay human-only?
> > > > 5. Is it acceptable for a bot to perform the merge, or should the
> > > > workflow only report "ready" and leave the final click to a
> > committer?
> > > > This
> > > > may need a check with ASF Infra.
> > > > 6. Are the size caps reasonable (connector fix: at most 250
> > production
> > > > lines and 25 files; new connector: at most 8000 lines and 120
> > files)?
> > > >
> > > > Please comment on the thread or on the PR. If there are no objections
> > > after
> > > > a reasonable discussion period, I propose we start with phase 1 only.
> > > >
> > > > Best Regards,
> > > >
> > > > David
> > > > Linkedin: https://www.linkedin.com/in/davidzollo
> > > >
> > >
> >
>
Re: [DISCUSS] Introduce an ai-merge label for risk-bounded PRs
Posted by David Zollo <da...@gmail.com>.
Hi all,
Thanks Jast for the review. I'll answer your point first, then restate the
proposal, because it has changed substantially since my first mail. Please
disregard the merge mechanism described there.
*On the hardcoded username*
You are right. An early version of the PR defaulted a configuration value (
AI_MERGE_APPROVERS, the account whose approval would have allowed a merge)
to DanielLeens. A user list in code or defaults is the wrong place for
that, and it is gone. The current head of the PR (782c83247d) contains no
user name, no approver or label-actor list, and no merge step. I checked
the full PR diff for it. Labels and comments are written by the workflow's
own GITHUB_TOKEN, so the mechanism does not grant any person the right to
add labels or approve workflow runs. The PR also changes nothing in
.asf.yaml.
Separately, I noticed that DanielLeens appears in the github.collaborators
list in .asf.yaml on dev, which I understand grants triage-level rights. If
that is the list you meant, it is independent of this proposal. This PR
neither depends on it nor changes it, and I think it is a matter for the
PMC rather than this thread. Jast, could you confirm which of the two you
meant?
*What changed since my first mail*
The first proposal let some PRs be merged without a further human code
review. That is gone. There is no ai-merge label, no sweep, no merge step,
and no approver or merge-related configuration. The workflow has permission
to write labels and comments only (contents: read, issues: write,
pull-requests:
write). Review and merge decisions stay entirely with committers.
What remains is a risk label on every PR so reviewers can order the queue.
Reference implementation: https://github.com/apache/seatunnel/pull/12706
*A correction to my first mail*
I said that TikTok Ads (#12283) and PayPal (#12282) were rightly rejected
by my first design because their poms pin third-party versions. That was
wrong. I later checked: the pinned versions (httpclient 4.5.13, httpcore
4.4.16, mockserver 5.14.0) are exactly the versions other poms in the
repository already use, and PayPal's only non-additive change was a line in
the labeler config. My first design scored on keywords instead of facts.
The new version scores whole-PR facts, for example an option key that only
moved to another file is not a removal, and an (artifact, version) pair is
only "new" if no existing pom uses it.
*What it does*
On every PR event (opened, pushed, edited, ready for review) it posts one
comment, updated in place, showing how the level was computed, and sets
exactly one label. Other labels are untouched.
Label Meaning
risk: 1-trivial No runtime effect: docs, examples, labeler metadata, tests
risk: 2-low Contained: a brand-new connector, or one existing connector unit
risk: 3-medium Needs a focused read: sizeable change, missing tests or
docs, tooling, unclassified path
risk: 4-high Touches a contract, a shared connector module, the supply
chain, several units, or repository settings
risk: 5-critical Platform code: engine, api, core, common, config, formats,
transforms, translation, dist, build, CI, tools, license files
Committers can then filter, for example label:"risk: 1-trivial" for a quick
pass or label:"risk: 5-critical" for PRs that need more than one set of
eyes.
*How the level is computed*
- *Base level* (the blast radius): 1 for docs and tests, 2 for a new
connector or one connector unit, 3 or 4 for tooling around the runtime, 4
for a shared connector module, 5 for platform code.
- *Adjustments* come from facts about the whole PR, each rule counted
once. The result is capped at 4 unless the base is 5, so only platform code
is critical.
- +2: an option key that no longer exists anywhere in the PR, or a
changed default or type; a modified checkpoint, split or state class or
serialVersionUID; a dependency (artifact, version) that no existing
pom uses; an existing test weakened (fewer assertion lines in a file, a
newly disabled test, looser numeric tolerance, shrunken fixture, deleted
test file); a removed registration entry (module, dist dependency,
plugin-mapping.properties, plugin_config); an uninspectable diff;
text addressed to a reviewer or AI.
- +1: security, concurrency or classloading words in modified
existing code; more than 250 production lines (1500 for a new connector);
production code changed without a test; several connector units; a new
connector or new option without docs; an author who is not yet a
contributor.
- A PR to a branch other than dev gets a floor of 3.
- Lines that are only moved within a PR are ignored. Suggestive findings
(hot-path readers or writers, type-mapping changes, credentials words in a
brand-new connector, no E2E test) are notes in the comment and never change
the level.
*What I checked*
- 59 offline tests, one per rule. I broke ten rules one at a time and
confirmed each break turns a test red.
- A real replay: Facebook Ads (#12275), TikTok Ads (#12283) and PayPal
(#12282) now score risk: 2-low. The Zendesk sink (#12253) scores risk:
3-medium.
- The last 150 merged PRs to dev (#12283 to #12689): 39 trivial, 29 low,
31 medium, 22 high, 29 critical. Critical PRs averaged 4.6 reviews and 4.9
comments, against 2.4 to 2.8 reviews for levels 1 to 4. These are raw
counts and may include bot activity.
*Limits I want to be clear about*
- Only level 5 separates clearly from the rest on review effort. Levels
1 to 4 should be read as triage hints, not measured risk.
- The checks are line-level heuristics, not code analysis, and can be
wrong in both directions. The replay already exposed several wrong scores
in my earlier versions, and I expect more once real PRs go through it.
- There was no revert in the 150-PR window, so I could not check whether
higher levels predict reverts.
- The label and comment writing has been tested offline only, not
against a live PR.
- A new third-party jar in a connector pom has no mechanical license
review today, as far as I can tell from checkLicense.sh. The scorer
raises the level for such a jar, but a human still has to review the
license.
*Safety*
The script is standard-library Python. It only reads the PR through the
GitHub API and never checks out or runs PR code, so it is safe under
pull_request_target. It contains no list of users. The worst failure is a
wrong label or comment, which anyone can ignore or remove.
*Proposed rollout*
1. First, run the script by hand on a few open PRs and post the output
on the PR for comparison, with no workflow.
2. Then enable the workflow for new PRs for a few weeks and collect
reports of wrong or noisy labels.
3. After that, adjust the thresholds and decide whether the five levels
are right.
*Questions for the community*
1. Is a risk label useful for how you review, or is it noise?
2. Are five levels the right granularity, and should only platform code
be critical?
3. Should a first-time contributor raise the level by one? It is a trust
signal, but it could feel unwelcoming.
4. Are there rules you would add, drop or reweight, for example should
CI-only changes be critical?
5. Is one bot comment per PR acceptable, or do you prefer labels only?
6. Any concern with a workflow that has write access to issues and pull
requests, running from the base branch?
Please comment here or on the PR. If there are no objections after a
reasonable discussion period, I propose we start with step 1 only.
Best Regards,
David
Linkedin: https://www.linkedin.com/in/davidzollo
On Sat, Oct 10, 2026 at 9:03 AM zhangshenghang <sh...@apache.org> wrote:
> I think the overall plan is viable. However, I noticed "DanielLeens" is
> hardcoded in the whitelist, and this user should not be granted permission
> to add labels or approve runs.
>
> Regards,
> Jast (Shenghang)
>
>
> zhiwei niu <zn...@gmail.com> 于2026年10月10日周六 14:22写道:
>
> > I wonder if we can create a new channel in Slack with AI notifications,
> > so that we can have a process connection and manually click on the merge
> > without having to discuss compliance with ASF Infa?
> >
> > Best regards,
> > Zhiwei Niu
> >
> >
> > David Zollo <da...@gmail.com> 于2026年10月10日周六 13:44写道:
> >
> > > Hi guys,
> > >
> > > I'd like to start a discussion about a small, opt-in mechanism for
> > reducing
> > > review load on PRs whose risk can be checked mechanically. The
> reference
> > > implementation is in https://github.com/apache/seatunnel/pull/12706.
> > > Nothing in it is active until we agree and enable it, and merging
> stays a
> > > dry run until a repository variable is switched on.
> > >
> > > *Motivation*
> > >
> > > Many PRs are well-bounded and easy to verify: documentation fixes,
> extra
> > > connector tests, bug fixes inside one connector, and new connectors
> that
> > > follow the standard pattern. They wait for the same scarce committer
> > > attention as engine or API changes. A new connector, for example,
> touches
> > > the same six registration places every time (the module pom, <module>
> > > entries, the seatunnel-dist dependency, plugin-mapping.properties,
> > > config/plugin_config, and the labeler entry), and CI already checks
> that
> > > registration is complete. At the same time, changes to the engine, the
> > API
> > > or checkpoint state need careful human review and must never be
> > > fast-tracked.
> > >
> > > *Proposal*
> > >
> > > A new label, ai-merge. A PR may be merged without a further human code
> > > review only if all of the following hold:
> > >
> > > 1. It belongs to one of five closed classes (below).
> > > 2. It hits none of the always-rejected conditions.
> > > 3. The label was applied by an authorised maintainer account, after
> > the
> > > current head commit was pushed. Any new push removes the label
> > > automatically.
> > > 4. A designated committer has approved the exact head commit.
> > > 5. All required CI checks are green on that commit, there are no
> > > unresolved review threads, and GitHub reports the PR as cleanly
> > > mergeable.
> > >
> > > *The five classes*
> > > Class What qualifies
> > > docs Markdown, docs images, and brand-new example configs.
> > > tests Connector unit tests and connector E2E, without weakening any
> > > existing assertion, timeout or fixture.
> > > connector A fix inside exactly one blast-radius unit (one connector
> > module,
> > > or one JDBC dialect), with a regression test in the same module, and
> docs
> > > if it adds an option.
> > > new-connector Exactly one brand-new connector module with its tests and
> > > docs, plus the registration it needs: the module pom, <module> entries,
> > the
> > > seatunnel-dist dependency, plugin-mapping.properties,
> > config/plugin_config,
> > > the labeler entry and SPI service lines.
> > > shared A small change to a shared connector module (connector-common,
> > > *-base,
> > > JDBC core, the fake, assert and console connectors).
> > >
> > > *How registration changes are accepted for a new connector*
> > >
> > > - Registration edits must be purely additive (no removed line), and
> > > every added line must be a plain registration of the new module and
> > > nothing
> > > else.
> > > - Service entries must name a class that the same PR adds.
> > > - A pom or registration change without a brand-new module in the
> same
> > PR
> > > is still rejected.
> > > - The new pom may only use artifacts that other poms in the
> repository
> > > already use. No literal third-party version, no repositories,
> > profiles,
> > > antrun or exec plugins.
> > > - In addition to the base checks, Code style (which runs
> > > check_connector_registration.py) and Dependency licenses must be
> > green.
> > >
> > > *An honest limit on dependencies*
> > >
> > > The existing license check covers core runtime jars, not the
> third-party
> > > jars a connector bundles. So a new third-party jar or version in a
> > > connector pom has no mechanical license or supply-chain review today,
> and
> > > the gate sends those PRs to a human. When I replayed three real merged
> > > connector PRs through the script, the Facebook Ads connector (#12275)
> > > passed. The TikTok Ads (#12283) and PayPal (#12282) PRs were rejected
> > > because their poms pin their own third-party versions. I think that is
> > the
> > > right outcome, but I'd like your view.
> > >
> > > *Shared modules need an extra proof*
> > >
> > > backend.yml selects tests by changed path, so a change to
> > connector-common
> > > only runs that module's tests, not the tests of the connectors that
> > depend
> > > on it. The shared class therefore stays rejected until we configure the
> > CI
> > > checks that exercise the dependents (AI_MERGE_WIDE_CHECKS). That would
> > need
> > > a small CI change to run the dependents on such PRs, which I would
> > propose
> > > as a separate PR. Even then it needs:
> > >
> > > - at most 80 production lines;
> > > - a test inside the same module;
> > > - no changed public or protected signature.
> > >
> > > *Always rejected*
> > >
> > > - seatunnel-engine, seatunnel-api, seatunnel-core, seatunnel-common,
> > > transforms, formats and translation.
> > > - .github/** (apart from the additive labeler entry of a new
> > connector)
> > > and tools/**, so a PR can never change its own verifier.
> > > - License files.
> > > - Checkpoint, state, split and serializer classes, option or default
> > > changes, and factory changes of existing connectors.
> > > - Security-sensitive, concurrency and classloading code. Inside
> > > brand-new connector files these are warnings instead, but
> > process-level
> > > calls such as System.exit stay blocked.
> > > - Removed or weakened assertions, disabled tests, and any file
> > deletion
> > > or rename.
> > > - PRs over the size caps, from first-time contributors, or
> containing
> > > text addressed to a reviewer or AI.
> > >
> > > *How it is enforced*
> > >
> > > - tools/ai_merge_gate.py is a standard-library Python script. It
> only
> > > reads the PR through the GitHub API and never checks out or runs PR
> > > code,
> > > so it is safe under pull_request_target.
> > > - The workflow removes the label on every push, validates who
> applied
> > > it, and runs a 10-minute sweep of the full merge gate. A squash
> merge
> > is
> > > pinned to the evaluated commit, so GitHub rejects it if the head
> > moved.
> > > - It has 56 offline tests, each pinning one rule. I also checked
> that
> > > breaking a rule makes the matching test fail.
> > >
> > > *What this is not*
> > >
> > > - It does not replace committer approval. A committer must approve
> the
> > > exact head commit.
> > > - The content checks are line-level heuristics, not code analysis.
> > They
> > > are deliberately conservative, and the AI review must still address
> > the
> > > warnings they raise (for example hot-path readers or writers, and
> type
> > > mapping changes). I do not claim they catch every risky change.
> > > - I have not exercised the GitHub API paths against a live PR yet,
> > which
> > > is why I propose starting in observe-only mode.
> > >
> > > *Proposed rollout*
> > >
> > > 1. Phase 1: observe only. Post the eligibility report as a PR
> comment
> > > and compare it with what reviewers decide. No label, no merging.
> > > 2. Phase 2: enable the label for docs, tests and single-connector
> > fixes.
> > > 3. Phase 3: enable new connectors, then shared modules once the
> > > dependents' CI proof exists.
> > > 4. After each phase, review the merged PRs. If a merged PR is
> reverted
> > > or breaks dev, suspend that class until we have a retrospective.
> > >
> > > *Questions I'd like your input on*
> > >
> > > 1. Is this acceptable under our project's review and merge
> > expectations,
> > > or should every PR still get at least one human code review from a
> > > committer other than the approver?
> > > 2. Who should be the approvers and the label-applying accounts, and
> > > should those be different people?
> > > 3. For new connectors, is "no new third-party jar or version without
> > > human review" the right boundary, or should we also route new
> > > connectors to
> > > a human whenever they handle credentials?
> > > 4. Should shared-module changes be allowed at all, given the extra
> CI
> > > work they need, or stay human-only?
> > > 5. Is it acceptable for a bot to perform the merge, or should the
> > > workflow only report "ready" and leave the final click to a
> committer?
> > > This
> > > may need a check with ASF Infra.
> > > 6. Are the size caps reasonable (connector fix: at most 250
> production
> > > lines and 25 files; new connector: at most 8000 lines and 120
> files)?
> > >
> > > Please comment on the thread or on the PR. If there are no objections
> > after
> > > a reasonable discussion period, I propose we start with phase 1 only.
> > >
> > > Best Regards,
> > >
> > > David
> > > Linkedin: https://www.linkedin.com/in/davidzollo
> > >
> >
>
Re: [DISCUSS] Introduce an ai-merge label for risk-bounded PRs
Posted by zhangshenghang <sh...@apache.org>.
I think the overall plan is viable. However, I noticed "DanielLeens" is
hardcoded in the whitelist, and this user should not be granted permission
to add labels or approve runs.
Regards,
Jast (Shenghang)
zhiwei niu <zn...@gmail.com> 于2026年10月10日周六 14:22写道:
> I wonder if we can create a new channel in Slack with AI notifications,
> so that we can have a process connection and manually click on the merge
> without having to discuss compliance with ASF Infa?
>
> Best regards,
> Zhiwei Niu
>
>
> David Zollo <da...@gmail.com> 于2026年10月10日周六 13:44写道:
>
> > Hi guys,
> >
> > I'd like to start a discussion about a small, opt-in mechanism for
> reducing
> > review load on PRs whose risk can be checked mechanically. The reference
> > implementation is in https://github.com/apache/seatunnel/pull/12706.
> > Nothing in it is active until we agree and enable it, and merging stays a
> > dry run until a repository variable is switched on.
> >
> > *Motivation*
> >
> > Many PRs are well-bounded and easy to verify: documentation fixes, extra
> > connector tests, bug fixes inside one connector, and new connectors that
> > follow the standard pattern. They wait for the same scarce committer
> > attention as engine or API changes. A new connector, for example, touches
> > the same six registration places every time (the module pom, <module>
> > entries, the seatunnel-dist dependency, plugin-mapping.properties,
> > config/plugin_config, and the labeler entry), and CI already checks that
> > registration is complete. At the same time, changes to the engine, the
> API
> > or checkpoint state need careful human review and must never be
> > fast-tracked.
> >
> > *Proposal*
> >
> > A new label, ai-merge. A PR may be merged without a further human code
> > review only if all of the following hold:
> >
> > 1. It belongs to one of five closed classes (below).
> > 2. It hits none of the always-rejected conditions.
> > 3. The label was applied by an authorised maintainer account, after
> the
> > current head commit was pushed. Any new push removes the label
> > automatically.
> > 4. A designated committer has approved the exact head commit.
> > 5. All required CI checks are green on that commit, there are no
> > unresolved review threads, and GitHub reports the PR as cleanly
> > mergeable.
> >
> > *The five classes*
> > Class What qualifies
> > docs Markdown, docs images, and brand-new example configs.
> > tests Connector unit tests and connector E2E, without weakening any
> > existing assertion, timeout or fixture.
> > connector A fix inside exactly one blast-radius unit (one connector
> module,
> > or one JDBC dialect), with a regression test in the same module, and docs
> > if it adds an option.
> > new-connector Exactly one brand-new connector module with its tests and
> > docs, plus the registration it needs: the module pom, <module> entries,
> the
> > seatunnel-dist dependency, plugin-mapping.properties,
> config/plugin_config,
> > the labeler entry and SPI service lines.
> > shared A small change to a shared connector module (connector-common,
> > *-base,
> > JDBC core, the fake, assert and console connectors).
> >
> > *How registration changes are accepted for a new connector*
> >
> > - Registration edits must be purely additive (no removed line), and
> > every added line must be a plain registration of the new module and
> > nothing
> > else.
> > - Service entries must name a class that the same PR adds.
> > - A pom or registration change without a brand-new module in the same
> PR
> > is still rejected.
> > - The new pom may only use artifacts that other poms in the repository
> > already use. No literal third-party version, no repositories,
> profiles,
> > antrun or exec plugins.
> > - In addition to the base checks, Code style (which runs
> > check_connector_registration.py) and Dependency licenses must be
> green.
> >
> > *An honest limit on dependencies*
> >
> > The existing license check covers core runtime jars, not the third-party
> > jars a connector bundles. So a new third-party jar or version in a
> > connector pom has no mechanical license or supply-chain review today, and
> > the gate sends those PRs to a human. When I replayed three real merged
> > connector PRs through the script, the Facebook Ads connector (#12275)
> > passed. The TikTok Ads (#12283) and PayPal (#12282) PRs were rejected
> > because their poms pin their own third-party versions. I think that is
> the
> > right outcome, but I'd like your view.
> >
> > *Shared modules need an extra proof*
> >
> > backend.yml selects tests by changed path, so a change to
> connector-common
> > only runs that module's tests, not the tests of the connectors that
> depend
> > on it. The shared class therefore stays rejected until we configure the
> CI
> > checks that exercise the dependents (AI_MERGE_WIDE_CHECKS). That would
> need
> > a small CI change to run the dependents on such PRs, which I would
> propose
> > as a separate PR. Even then it needs:
> >
> > - at most 80 production lines;
> > - a test inside the same module;
> > - no changed public or protected signature.
> >
> > *Always rejected*
> >
> > - seatunnel-engine, seatunnel-api, seatunnel-core, seatunnel-common,
> > transforms, formats and translation.
> > - .github/** (apart from the additive labeler entry of a new
> connector)
> > and tools/**, so a PR can never change its own verifier.
> > - License files.
> > - Checkpoint, state, split and serializer classes, option or default
> > changes, and factory changes of existing connectors.
> > - Security-sensitive, concurrency and classloading code. Inside
> > brand-new connector files these are warnings instead, but
> process-level
> > calls such as System.exit stay blocked.
> > - Removed or weakened assertions, disabled tests, and any file
> deletion
> > or rename.
> > - PRs over the size caps, from first-time contributors, or containing
> > text addressed to a reviewer or AI.
> >
> > *How it is enforced*
> >
> > - tools/ai_merge_gate.py is a standard-library Python script. It only
> > reads the PR through the GitHub API and never checks out or runs PR
> > code,
> > so it is safe under pull_request_target.
> > - The workflow removes the label on every push, validates who applied
> > it, and runs a 10-minute sweep of the full merge gate. A squash merge
> is
> > pinned to the evaluated commit, so GitHub rejects it if the head
> moved.
> > - It has 56 offline tests, each pinning one rule. I also checked that
> > breaking a rule makes the matching test fail.
> >
> > *What this is not*
> >
> > - It does not replace committer approval. A committer must approve the
> > exact head commit.
> > - The content checks are line-level heuristics, not code analysis.
> They
> > are deliberately conservative, and the AI review must still address
> the
> > warnings they raise (for example hot-path readers or writers, and type
> > mapping changes). I do not claim they catch every risky change.
> > - I have not exercised the GitHub API paths against a live PR yet,
> which
> > is why I propose starting in observe-only mode.
> >
> > *Proposed rollout*
> >
> > 1. Phase 1: observe only. Post the eligibility report as a PR comment
> > and compare it with what reviewers decide. No label, no merging.
> > 2. Phase 2: enable the label for docs, tests and single-connector
> fixes.
> > 3. Phase 3: enable new connectors, then shared modules once the
> > dependents' CI proof exists.
> > 4. After each phase, review the merged PRs. If a merged PR is reverted
> > or breaks dev, suspend that class until we have a retrospective.
> >
> > *Questions I'd like your input on*
> >
> > 1. Is this acceptable under our project's review and merge
> expectations,
> > or should every PR still get at least one human code review from a
> > committer other than the approver?
> > 2. Who should be the approvers and the label-applying accounts, and
> > should those be different people?
> > 3. For new connectors, is "no new third-party jar or version without
> > human review" the right boundary, or should we also route new
> > connectors to
> > a human whenever they handle credentials?
> > 4. Should shared-module changes be allowed at all, given the extra CI
> > work they need, or stay human-only?
> > 5. Is it acceptable for a bot to perform the merge, or should the
> > workflow only report "ready" and leave the final click to a committer?
> > This
> > may need a check with ASF Infra.
> > 6. Are the size caps reasonable (connector fix: at most 250 production
> > lines and 25 files; new connector: at most 8000 lines and 120 files)?
> >
> > Please comment on the thread or on the PR. If there are no objections
> after
> > a reasonable discussion period, I propose we start with phase 1 only.
> >
> > Best Regards,
> >
> > David
> > Linkedin: https://www.linkedin.com/in/davidzollo
> >
>
Re: [DISCUSS] Introduce an ai-merge label for risk-bounded PRs
Posted by zhiwei niu <zn...@gmail.com>.
I wonder if we can create a new channel in Slack with AI notifications,
so that we can have a process connection and manually click on the merge
without having to discuss compliance with ASF Infa?
Best regards,
Zhiwei Niu
David Zollo <da...@gmail.com> 于2026年10月10日周六 13:44写道:
> Hi guys,
>
> I'd like to start a discussion about a small, opt-in mechanism for reducing
> review load on PRs whose risk can be checked mechanically. The reference
> implementation is in https://github.com/apache/seatunnel/pull/12706.
> Nothing in it is active until we agree and enable it, and merging stays a
> dry run until a repository variable is switched on.
>
> *Motivation*
>
> Many PRs are well-bounded and easy to verify: documentation fixes, extra
> connector tests, bug fixes inside one connector, and new connectors that
> follow the standard pattern. They wait for the same scarce committer
> attention as engine or API changes. A new connector, for example, touches
> the same six registration places every time (the module pom, <module>
> entries, the seatunnel-dist dependency, plugin-mapping.properties,
> config/plugin_config, and the labeler entry), and CI already checks that
> registration is complete. At the same time, changes to the engine, the API
> or checkpoint state need careful human review and must never be
> fast-tracked.
>
> *Proposal*
>
> A new label, ai-merge. A PR may be merged without a further human code
> review only if all of the following hold:
>
> 1. It belongs to one of five closed classes (below).
> 2. It hits none of the always-rejected conditions.
> 3. The label was applied by an authorised maintainer account, after the
> current head commit was pushed. Any new push removes the label
> automatically.
> 4. A designated committer has approved the exact head commit.
> 5. All required CI checks are green on that commit, there are no
> unresolved review threads, and GitHub reports the PR as cleanly
> mergeable.
>
> *The five classes*
> Class What qualifies
> docs Markdown, docs images, and brand-new example configs.
> tests Connector unit tests and connector E2E, without weakening any
> existing assertion, timeout or fixture.
> connector A fix inside exactly one blast-radius unit (one connector module,
> or one JDBC dialect), with a regression test in the same module, and docs
> if it adds an option.
> new-connector Exactly one brand-new connector module with its tests and
> docs, plus the registration it needs: the module pom, <module> entries, the
> seatunnel-dist dependency, plugin-mapping.properties, config/plugin_config,
> the labeler entry and SPI service lines.
> shared A small change to a shared connector module (connector-common,
> *-base,
> JDBC core, the fake, assert and console connectors).
>
> *How registration changes are accepted for a new connector*
>
> - Registration edits must be purely additive (no removed line), and
> every added line must be a plain registration of the new module and
> nothing
> else.
> - Service entries must name a class that the same PR adds.
> - A pom or registration change without a brand-new module in the same PR
> is still rejected.
> - The new pom may only use artifacts that other poms in the repository
> already use. No literal third-party version, no repositories, profiles,
> antrun or exec plugins.
> - In addition to the base checks, Code style (which runs
> check_connector_registration.py) and Dependency licenses must be green.
>
> *An honest limit on dependencies*
>
> The existing license check covers core runtime jars, not the third-party
> jars a connector bundles. So a new third-party jar or version in a
> connector pom has no mechanical license or supply-chain review today, and
> the gate sends those PRs to a human. When I replayed three real merged
> connector PRs through the script, the Facebook Ads connector (#12275)
> passed. The TikTok Ads (#12283) and PayPal (#12282) PRs were rejected
> because their poms pin their own third-party versions. I think that is the
> right outcome, but I'd like your view.
>
> *Shared modules need an extra proof*
>
> backend.yml selects tests by changed path, so a change to connector-common
> only runs that module's tests, not the tests of the connectors that depend
> on it. The shared class therefore stays rejected until we configure the CI
> checks that exercise the dependents (AI_MERGE_WIDE_CHECKS). That would need
> a small CI change to run the dependents on such PRs, which I would propose
> as a separate PR. Even then it needs:
>
> - at most 80 production lines;
> - a test inside the same module;
> - no changed public or protected signature.
>
> *Always rejected*
>
> - seatunnel-engine, seatunnel-api, seatunnel-core, seatunnel-common,
> transforms, formats and translation.
> - .github/** (apart from the additive labeler entry of a new connector)
> and tools/**, so a PR can never change its own verifier.
> - License files.
> - Checkpoint, state, split and serializer classes, option or default
> changes, and factory changes of existing connectors.
> - Security-sensitive, concurrency and classloading code. Inside
> brand-new connector files these are warnings instead, but process-level
> calls such as System.exit stay blocked.
> - Removed or weakened assertions, disabled tests, and any file deletion
> or rename.
> - PRs over the size caps, from first-time contributors, or containing
> text addressed to a reviewer or AI.
>
> *How it is enforced*
>
> - tools/ai_merge_gate.py is a standard-library Python script. It only
> reads the PR through the GitHub API and never checks out or runs PR
> code,
> so it is safe under pull_request_target.
> - The workflow removes the label on every push, validates who applied
> it, and runs a 10-minute sweep of the full merge gate. A squash merge is
> pinned to the evaluated commit, so GitHub rejects it if the head moved.
> - It has 56 offline tests, each pinning one rule. I also checked that
> breaking a rule makes the matching test fail.
>
> *What this is not*
>
> - It does not replace committer approval. A committer must approve the
> exact head commit.
> - The content checks are line-level heuristics, not code analysis. They
> are deliberately conservative, and the AI review must still address the
> warnings they raise (for example hot-path readers or writers, and type
> mapping changes). I do not claim they catch every risky change.
> - I have not exercised the GitHub API paths against a live PR yet, which
> is why I propose starting in observe-only mode.
>
> *Proposed rollout*
>
> 1. Phase 1: observe only. Post the eligibility report as a PR comment
> and compare it with what reviewers decide. No label, no merging.
> 2. Phase 2: enable the label for docs, tests and single-connector fixes.
> 3. Phase 3: enable new connectors, then shared modules once the
> dependents' CI proof exists.
> 4. After each phase, review the merged PRs. If a merged PR is reverted
> or breaks dev, suspend that class until we have a retrospective.
>
> *Questions I'd like your input on*
>
> 1. Is this acceptable under our project's review and merge expectations,
> or should every PR still get at least one human code review from a
> committer other than the approver?
> 2. Who should be the approvers and the label-applying accounts, and
> should those be different people?
> 3. For new connectors, is "no new third-party jar or version without
> human review" the right boundary, or should we also route new
> connectors to
> a human whenever they handle credentials?
> 4. Should shared-module changes be allowed at all, given the extra CI
> work they need, or stay human-only?
> 5. Is it acceptable for a bot to perform the merge, or should the
> workflow only report "ready" and leave the final click to a committer?
> This
> may need a check with ASF Infra.
> 6. Are the size caps reasonable (connector fix: at most 250 production
> lines and 25 files; new connector: at most 8000 lines and 120 files)?
>
> Please comment on the thread or on the PR. If there are no objections after
> a reasonable discussion period, I propose we start with phase 1 only.
>
> Best Regards,
>
> David
> Linkedin: https://www.linkedin.com/in/davidzollo
>