You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "pierrejeambrun (via GitHub)" <gi...@apache.org> on 2026/10/02 14:01:57 UTC

[PR] Fail instead of silently falling behind when a Lang-SDK vendored Dag schema drifts [airflow]

pierrejeambrun opened a new pull request, #74091:
URL: https://github.com/apache/airflow/pull/74091

   ## Why
   
   A core Dag-serialization schema change shipped recently with the Go SDK's vendored copy left behind, and nothing caught it: the Go and Java sync hooks were manual-only, and the check that runs on every commit (`check-go-sdk-generated-drift`) watches the *vendored copies*, not the Python source, so it never saw the schema change at all. The same class of drift already caused a two-release-old bug in the Go SDK's generated models (#73954).
   
   ## What
   
   - `sync-go-sdk-schemas` and `sync-java-sdk-dag-schema` now run on every commit (dropping `stages: ['manual']`), and are both triggered by the Python schema source itself, not just the vendored copy — matching how `sync-ts-sdk-dag-schema` already worked. Each refreshes the vendored copy and fails when it had to, so the refresh gets reviewed and committed rather than shipping unnoticed.
   - `check-go-sdk-generated-drift`'s trigger now also includes the Python schema sources, so it fires in the same commit as the sync hook above and catches generated Go code that still needs regenerating from the freshly-synced copy.
   - Java's underlying Gradle task (`SyncDagSchemaTask`) gains a `failOnRefresh` flag, off by default. `generateDagDsl`'s existing dependency keeps the flag off, so an ordinary in-repo build still refreshes a schema a developer is actively editing without breaking. The prek hook now calls a second task instance (`checkDagSchemaSync`) with the flag on.
   
   ## How I verified it
   
   Ran each hook directly with a deliberately corrupted vendored copy and confirmed it refreshes the file and fails:
   - `prek run sync-java-sdk-dag-schema --files airflow-core/src/airflow/serialization/schema.json`
   - `prek run sync-go-sdk-schemas --files airflow-core/src/airflow/serialization/schema.json`
   - `prek run check-go-sdk-generated-drift --files airflow-core/src/airflow/serialization/schema.json`
   
   Also confirmed `./java-sdk/gradlew :sdk:generateDagDsl` still refreshes silently and succeeds (no regression to the normal in-repo build), while `:sdk:checkDagSchemaSync` fails on the same drift.
   
   No newsfragment: CI/build tooling only, no user-facing behavior change.
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes — Claude Code (Opus 4.7)
   
   Generated-by: Claude Code (Opus 4.7) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fail instead of silently falling behind when a Lang-SDK vendored Dag schema drifts [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #74091:
URL: https://github.com/apache/airflow/pull/74091#issuecomment-5991344211

   Lets try this while we don't have the other 'periodic' check and see if this help / is too annoying for development.
   
   We can always revert if it's too big of a constraint for core update to have to update sdks too.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fail instead of silently falling behind when a Lang-SDK vendored Dag schema drifts [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun merged PR #74091:
URL: https://github.com/apache/airflow/pull/74091


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org