You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "sandynz (via GitHub)" <gi...@apache.org> on 2026/02/28 11:11:17 UTC

[PR] feat: dynamic matrix generation for e2e-sql CI workflow [shardingsphere]

sandynz opened a new pull request, #38274:
URL: https://github.com/apache/shardingsphere/pull/38274

   Fixes #38270.
   
   Changes proposed in this pull request:
     - feat: dynamic matrix generation for e2e-sql CI workflow
   
   ---
   
   Before committing this PR, I'm sure that I have checked the following options:
   - [ ] My code follows the [code of conduct](https://shardingsphere.apache.org/community/en/involved/conduct/code/) of this project.
   - [ ] I have self-reviewed the commit code.
   - [ ] I have (or in comment I request) added corresponding labels for the pull request.
   - [ ] I have passed maven check locally : `./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e`.
   - [ ] I have made corresponding changes to the documentation.
   - [ ] I have added corresponding unit tests for my changes.
   - [ ] I have updated the Release Notes of the current development version. For more details, see [Update Release Note](https://shardingsphere.apache.org/community/en/involved/contribute/contributor/)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: dynamic matrix generation for e2e-sql CI workflow [shardingsphere]

Posted by "sandynz (via GitHub)" <gi...@apache.org>.
sandynz commented on PR #38274:
URL: https://github.com/apache/shardingsphere/pull/38274#issuecomment-3977023239

   The e2e-sql workflow ran the full 119-job static matrix on every PR regardless of what changed. Replace it with a `dorny/paths-filter`-driven dynamic matrix that runs only the jobs relevant to the PR's changed files, reducing CI load by 50–90% for most PRs.
   
   ## Changes
   
   - **`.github/workflows/resources/filter/e2e-sql-filters.yml`** — 18 boolean path-filter labels across four dimensions:
     - Adapter: `adapter_proxy`, `adapter_jdbc`
     - Mode: `mode_standalone`, `mode_cluster`, `mode_core`
     - Database: `database_mysql`, `database_postgresql`
     - Feature/scenario: `feature_sharding`, `feature_encrypt`, `feature_readwrite_splitting`, `feature_shadow`, `feature_mask`, `feature_broadcast`, `feature_distsql`, `feature_sql_federation`
     - Full-fallback triggers: `core_infra`, `test_framework`, `pom_changes`
   
   - **`.github/workflows/resources/scripts/generate-e2e-sql-matrix.sh`** — reads the 18 filter outputs and emits a compact JSON matrix to `$GITHUB_OUTPUT` with three outputs: `matrix`, `has-jobs`, `need-proxy-image`. Logic:
     - `core_infra | test_framework | pom_changes = true` → emit full 119-job matrix unchanged
     - No relevant label changed → `has-jobs=false`, skip all downstream jobs
     - Otherwise: compute adapters/modes/databases from dimension labels; resolve scenarios from feature labels (smoke set `empty_rules, db, tbl, encrypt, readwrite_splitting, passthrough` when no feature label fires); apply all original 5 exclude rules and the MySQL 8.3.0 extra passthrough include
   
   - **`.github/workflows/e2e-sql.yml`** — replaces the static `matrix:` block with four jobs:
     1. `global-environment` — unchanged
     2. `detect-and-generate-matrix` — runs filter action + script, exposes `matrix`/`has-jobs`/`need-proxy-image`
     3. `build-e2e-image` — conditioned on `has-jobs == 'true' && need-proxy-image == 'true'`
     4. `e2e-sql` — uses `fromJSON(needs.detect-and-generate-matrix.outputs.matrix)` with `always()` guard so it handles the skipped build-image case cleanly
   
   Example output for a PR touching only `features/sharding/`:
   ```json
   {"include":[
     {"adapter":"proxy","mode":"Standalone","database":"MySQL","scenario":"db","additional-options":""},
     {"adapter":"proxy","mode":"Cluster","database":"MySQL","scenario":"sharding_and_encrypt","additional-options":""},
     ...
   ]}
   ```
   60 jobs instead of 119.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: dynamic matrix generation for e2e-sql CI workflow [shardingsphere]

Posted by "sandynz (via GitHub)" <gi...@apache.org>.
sandynz merged PR #38274:
URL: https://github.com/apache/shardingsphere/pull/38274


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org