You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "voonhous (via GitHub)" <gi...@apache.org> on 2026/09/11 15:49:55 UTC

[I] [CI] Java CI runner-minutes and critical runtime: PR tracker [hudi]

voonhous opened a new issue, #19910:
URL: https://github.com/apache/hudi/issues/19910

   Part of #19524. Baseline is master run [34314808833](https://github.com/apache/hudi/actions/runs/34314808833): 615 runner-minutes over 19 test jobs, longest job 77 min, two trips through the runner queue per run.
   
   | PR | change | runner-min per run | critical runtime | status |
   |---|---|---|---|---|
   | #19890 | flatten the job graph, path filter at the trigger | 615 -> 617 (unchanged) | second queue hop gone (median 1 min, p90 82 min) | measured, ready to merge |
   | #19892 | two surefire forks, DeltaStreamer split, jobs resharded by measured time | 615 -> 577 | longest job 77 -> 41 min | measured, ready to merge after #19890 |
   | #19906 | test Spark sessions at 2 shuffle partitions, AQE off | 577 -> ~547 (Spark lanes 447 -> 417, mean of two runs) | no change inside noise | measured, third run pending |
   | #19909 | one full build per module and lane: flink-2 filter, docker build trim, examples-spark trim | expected ~-34 | longest job 39 -> ~35 | first run in progress |
   | #19905 | Maven build cache | - | - | closed: the repository's Actions cache store keeps nothing |
   
   Expected once the four open PRs merge: about 615 -> 510 runner-minutes (-17%), longest job 77 -> ~35 min (-55%), one queue hop.
   
   Deferred, with reasons on record:
   - method-level test concurrency: https://github.com/apache/hudi/pull/18201#issuecomment-5633539153
   - path scoping between the Spark and engine workflows (~11% of runner-minutes, needs a path-to-lane map)
   - pinning six transitive version ranges (junit, junit-dep, json-smart, grpc x3) to remove metadata checks against jitpack.io and cloudera; one 16-minute stall observed
   - running test-common-and-other-modules outside its 2024 container (maintenance, not time)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [CI] Java CI runner-minutes and critical runtime: PR tracker [hudi]

Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on issue #19910:
URL: https://github.com/apache/hudi/issues/19910#issuecomment-5653906635

   Test-count parity, master against the campaign's end state, counted from every job log of [master run 34674848405](https://github.com/apache/hudi/actions/runs/34674848405) and [#19937's attempt 3](https://github.com/apache/hudi/actions/runs/34753265152/attempts/3) plus its engine run. The question was whether any test stopped running along the way.
   
   ScalaTest, identical:
   
   | | master | end state |
   |---|---|---|
   | passed / failed / canceled / ignored | 963 / 0 / 5 / 3 | 963 / 0 / 5 / 3 |
   | distinct test names | 986 | 986, the same set |
   
   Surefire, 0 failures on both sides, 1285 against 1287 classes, 9 classes with a different count and every difference explained:
   
   | | master | end state |
   |---|---|---|
   | tests run | 19267 | 17470 (Trino plugin excluded: its run is not part of master's Java CI) |
   | skipped | 214 | 215 |
   
   - 1808 duplicate runs removed: master ran the 113 hudi-flink test classes twice, in flink-2 and again in common-and-other; #19909 dropped the second run, the end state runs each once.
   - Utilities tests moved jobs: 1718 on master split between common-and-other and test-utilities, 1721 on the end state in the two utilities jobs, same classes.
   - TestHoodieDeltaStreamer is four classes since #19892: 197 against 199.
   - The bloom filter benchmark is out of unit runs by design in #19892: 1 test.
   - Two new unit tests from #19935 and #19936.
   - A few classes whose methods surefire's two forks report as two partial summaries; the totals match.
   
   No class runs fewer tests on the end state than on master. The same paragraph is in the findings list above.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [CI] Java CI runner-minutes and critical runtime: PR tracker [hudi]

Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on issue #19910:
URL: https://github.com/apache/hudi/issues/19910#issuecomment-5643134415

   Next lever, same shape as #19909: the integration-tests job builds and verifies far more than its integration tests use.
   
   What the job does today (run [34602459140](https://github.com/apache/hudi/actions/runs/34602459140), job step times): `mvn clean install` over the whole reactor, 66 modules including all 17 bundles (8.9 min), then `mvn verify -Pintegration-tests -pl !hudi-flink` over the same reactor (21 min). The verify step packages all 66 modules a second time, shading the 17 bundles again (34 shade executions in the job), then runs the integration tests that exist. Those live in four modules: hudi-cli (8 IT classes, 814 s), hudi-aws (2), hudi-azure (1), hudi-integ-test (its two demo suites are skipped and `@Disabled`; the job has no docker-compose activity).
   
   What the four modules need, from Maven's own resolution (`-pl hudi-cli,hudi-integ-test,hudi-aws,hudi-azure -am`): 30 modules, the core chain through hudi-utilities plus the utilities bundle (hudi-cli) and the hadoop-mr, hive-sync and spark bundles with two docker pom modules (hudi-integ-test).
   
   The 36 modules it would stop building are each built, and where a bundle shaded and run, in the job that owns them:
   
   | removed from integration-tests | already built and tested or validated in |
   |---|---|
   | hudi-flink, hudi-flink-client, the Flink datasource modules | test-flink-1 and test-flink-2 (build with -am, run their tests) |
   | hudi-flink-bundle and the other eleven bundles the ITs do not need | validate-bundles (shades every bundle, then runs them) |
   | hudi-examples-common, java, k8s | validate-bundles |
   | hudi-examples-spark | ut-other (with its quickstart test), validate-bundles |
   | hudi-examples-flink | test-flink-1 (with its unit tests), validate-bundles |
   | hudi-kafka-connect | test-common-and-other-modules (unit tests); validate-bundles (its bundle) |
   | hudi-datahub-sync, hudi-adb-sync | test-common-and-other-modules (unit tests) |
   | metaserver modules | test-common-and-other-modules (platform-service build); validate-bundles (metaserver bundle) |
   | the verify step's second packaging of all 66 modules | the same job's own build step |
   
   What stays: the 30-module build and all 14 integration-test classes the job runs today.
   
   Expected: the build step from 8.9 to about 5 minutes and the verify step from 21 to about 16, so the job from 33-38 to the high twenties, about 10 runner-minutes per run. It sits on the plateau with the Scala shards, so the wall clock only moves with it if those come down too. Verification before pushing, as for #19909: each `-pl` list validated against its command's reactor, and a pom scan showing nothing kept depends on anything no longer built.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [CI] Java CI runner-minutes and critical runtime: PR tracker [hudi]

Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on issue #19910:
URL: https://github.com/apache/hudi/issues/19910#issuecomment-5643850480

   #19919 measured, two attempts of the same commit against #19918's run (same code minus the change). Queue time excluded.
   
   integration-tests job:
   
   | step | #19918 run | #19919 attempt 1 | #19919 attempt 2 |
   |---|---|---|---|
   | Build Project (s) | 646 | 479 | 456 |
   | IT (s) | 1390 | 1028 | 971 |
   | modules built / verified | 66 / 64 | 30 / 4 | 30 / 4 |
   | job execution (min) | 37 | 28 | 25 |
   
   Whole run, 19 test jobs:
   
   | metric | #19918 run | #19919 attempt 1 | #19919 attempt 2 |
   |---|---|---|---|
   | total runner-minutes | 541 | 539 | 531 |
   | longest job (min) | 40 | 40 | 41 |
   
   Both IT-step samples are below every one of about 90 earlier samples of this job on master and PRs since 2026-09-04, whose lowest was 1258 s. The two attempts differ by 8 runner-minutes in the total with no code change, and single untouched jobs moved by up to 8 minutes either way, so the run total cannot resolve a 10-minute change in one job; the job's own steps are the measurement. Same 25 test classes and per-module totals in the IT step's log before and after. No effect on the wall clock: the Scala shards still set it.
   
   Runs: [attempt 1](https://github.com/apache/hudi/actions/runs/34670498928), [attempt 2](https://github.com/apache/hudi/actions/runs/34670498928/attempts/2); per-job tables on the PR.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [CI] Java CI runner-minutes and critical runtime: PR tracker [hudi]

Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on issue #19910:
URL: https://github.com/apache/hudi/issues/19910#issuecomment-5658468456

   Merge order. Each PR is rebased as the one below it merges; tick as they land.
   
   CI stack, bottom up:
   - [ ] #19890 flatten the job graph
   - [ ] #19892 two surefire forks, DeltaStreamer split, reshard by measured time
   - [ ] #19909 one full build per module and lane
   - [ ] #19918 slowest tests on every job's run page
   - [ ] #19919 integration-tests builds only its closure
   
   Off #19919, in either order:
   - [ ] #19931 job timeouts
   - [ ] #19933 hadoop-mr and java-client folded into utilities-rest
   - [ ] #19926 shared SparkContext for the SQL suites, opt-in
   - [ ] #19927 JVM state scoped for concurrent SQL suites
   - [ ] #19928 SQL suites two at a time on three shards (drop the DNM tag first)
   - [ ] #19932 other-2 joins the parallel shards (drop the DNM tag first; sheds its copies of the two stream fixes on the rebase after they merge)
   
   Independent of the stack, any time:
   - [ ] #19935 close the filesystem's stream on rewrap
   - [ ] #19936 close the outer file stream of inline reads
   
   Never merged: #19937 (measurement, closed), the proofs of concept #19921, #19923, #19925 (superseded, to close), and the closed experiments #19905 and #19906.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [CI] Java CI runner-minutes and critical runtime: PR tracker [hudi]

Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on issue #19910:
URL: https://github.com/apache/hudi/issues/19910#issuecomment-6364932061

   @yihua Amongst the open PRs, they're pretty much interlinked. All of them are pretty much dependent on each other, so i believe their all of equal priority. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [CI] Java CI runner-minutes and critical runtime: PR tracker [hudi]

Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on issue #19910:
URL: https://github.com/apache/hudi/issues/19910#issuecomment-6323449205

   @voonhous thanks for taking this up!  Among the changes, which PRs have the highest priority and leverage to review and merge first?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org