You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "voonhous (via GitHub)" <gi...@apache.org> on 2026/09/12 03:29:29 UTC
[PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
voonhous opened a new pull request, #19919:
URL: https://github.com/apache/hudi/pull/19919
### Describe the issue this Pull Request addresses
Part of the CI improvement epic #19524 (#19527), tracked on #19910. Stacked on #19918; the commits before "ci: build and verify only the IT modules in integration-tests" belong to that stack and drop out on rebase.
The integration-tests job builds all 66 modules of the reactor (8.9 min), then runs `mvn verify` over the same 66 modules, which packages every one of them a second time, the 17 bundles included, before running the integration tests (21 min). The integration tests live in four modules.
### Summary and Changelog
- `IT_MODULES` names the four modules with integration-test classes outside hudi-flink, whose ITs run in the flink jobs: hudi-cli, hudi-integ-test, hudi-aws, hudi-azure. The docker demo suites in hudi-integ-test are skipped and `@Disabled`, and the job has no docker-compose activity.
- The build step passes them with `-am`. Maven resolves that to 30 modules: the core chain through hudi-utilities, the utilities bundle for hudi-cli, and the hadoop-mr, hive-sync and spark bundles plus two docker pom modules for hudi-integ-test.
- The IT step verifies just the four, instead of the whole reactor minus hudi-flink.
Every module no longer built here is built, and where a bundle shaded and run, in the job that owns it:
| no longer built in integration-tests | already built and tested or validated in |
|---|---|
| hudi-flink, hudi-flink-client, the Flink datasource modules | test-flink-1 and test-flink-2 |
| hudi-flink-bundle and the other eleven bundles the ITs do not need | validate-bundles |
| hudi-examples-common, java, k8s | validate-bundles |
| hudi-examples-spark | ut-other and validate-bundles |
| hudi-examples-flink | test-flink-1 and validate-bundles |
| hudi-kafka-connect | test-common-and-other-modules (tests), validate-bundles (bundle) |
| hudi-datahub-sync, hudi-adb-sync | test-common-and-other-modules |
| metaserver modules | test-common-and-other-modules, validate-bundles |
| the verify step's second packaging of all 66 modules | this job's own build step |
What stays: the 30-module build and all 14 integration-test classes the job runs today (8 in hudi-cli, 2 in hudi-aws, 1 in hudi-azure, hudi-integ-test's skipped ones).
### Impact
Expected against #19892's run ([34499656559](https://github.com/apache/hudi/actions/runs/34499656559)): build step 8.9 to about 5 minutes, verify step 21 to about 16, the job from 33-38 minutes to the high twenties, about 10 runner-minutes per run. The job sits on the plateau with the Scala shards, so the wall clock follows only if those come down too. Measured numbers go here after the run.
### Risk Level
low
Both `-pl` lists validate against the job's profiles, and `-am` makes the build's reactor exactly the dependency closure of what verify runs, so nothing verify needs can be missing. If a module gained integration tests, `IT_MODULES` is the one line to extend; the failsafe class count to compare after this PR is 14.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5643224342
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/19919?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 80.18%. Comparing base ([`648996f`](https://app.codecov.io/gh/apache/hudi/commit/648996f002c3c9e24a1e2bf53f97fe03e8bc4c81?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e98fd38`](https://app.codecov.io/gh/apache/hudi/commit/e98fd38999e74c5053be30e827cba94c7b676e06?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 5 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #19919 +/- ##
============================================
- Coverage 80.20% 80.18% -0.03%
+ Complexity 34661 34594 -67
============================================
Files 2546 2546
Lines 142452 142463 +11
Branches 17330 17364 +34
============================================
- Hits 114260 114228 -32
- Misses 20281 20292 +11
- Partials 7911 7943 +32
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.82% <ø> (-0.03%)` | :arrow_down: |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.25% <ø> (-0.04%)` | :arrow_down: |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.52% <ø> (-0.06%)` | :arrow_down: |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.23% <ø> (-0.05%)` | :arrow_down: |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.13% <ø> (-0.05%)` | :arrow_down: |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `69.99% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.83% <ø> (+0.02%)` | :arrow_up: |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `75.99% <ø> (ø)` | |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.61% <ø> (ø)` | |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.34% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `80.99% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/19919/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.20% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/19919/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/19919/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.92% <ø> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
[see 67 files with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/19919/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5648799164
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
}, {
"hash" : "84838ed0edf7023fad776fdd3c15c90c1e784fb1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17053",
"triggerID" : "84838ed0edf7023fad776fdd3c15c90c1e784fb1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84838ed0edf7023fad776fdd3c15c90c1e784fb1 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17053)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5643688642
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e98fd38999e74c5053be30e827cba94c7b676e06 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5644150403
Attempt 3 of the same runs ([34670498897](https://github.com/apache/hudi/actions/runs/34670498897/attempts/3), [34670498928](https://github.com/apache/hudi/actions/runs/34670498928/attempts/3)), with the earlier attempts and #19918's run alongside. Queue time excluded.
| job | #19918 run (min) | attempt 1 (min) | attempt 2 (min) | attempt 3 (min) |
|---|---|---|---|---|
| integration-tests | 37 | 28 | 25 | 27 |
| test-common-and-other-modules | 34 | 34 | 35 | 34 |
| test-flink-1 | 32 | 31 | 31 | 29 |
| test-flink-2 | 23 | 23 | 23 | 23 |
| test-hudi-hadoop-mr-and-hudi-java-client | 11 | 10 | 11 | 11 |
| test-spark-client-and-hadoop-common | 33 | 34 | 30 | 35 |
| test-spark-java17-fta | 30 | 35 | 35 | 40 |
| test-spark-java17-ftb | 29 | 28 | 28 | 27 |
| test-spark-java17-scala-dml-1 | 40 | 38 | 38 | 39 |
| test-spark-java17-scala-dml-2 | 22 | 31 | 25 | 32 |
| test-spark-java17-scala-other-1 | 40 | 40 | 41 | 37 |
| test-spark-java17-scala-other-2 | 37 | 30 | 35 | 37 |
| test-spark-java17-ut-fat-and-ftc | 36 | 37 | 37 | 39 |
| test-spark-java17-ut-functional | 28 | 28 | 28 | 29 |
| test-spark-java17-ut-other | 29 | 32 | 26 | 32 |
| test-utilities-rest | 22 | 22 | 22 | 22 |
| test-utilities-streamer | 31 | 30 | 30 | 31 |
| validate-bundle-spark4 | 10 | 10 | 11 | 11 |
| validate-bundles | 16 | 16 | 19 | 18 |
| total | 541 | 539 | 531 | 551 |
| longest job | 40 | 40 | 41 | 40 |
integration-tests steps, attempt 3: Build Project 515 s (attempts 1 and 2: 479, 456; before: 646), IT 985 s (1028, 971; before: 1390). This attempt's total went up by 10 against #19918's run while the job it changes went down by 10: fta and scala-dml-2 both took 10 minutes longer than in #19918's run with no code change. That is the noise floor of the run total; the job's three samples all sit below every earlier sample of this job on record.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5647551426
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
}, {
"hash" : "84838ed0edf7023fad776fdd3c15c90c1e784fb1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17053",
"triggerID" : "84838ed0edf7023fad776fdd3c15c90c1e784fb1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e98fd38999e74c5053be30e827cba94c7b676e06 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034)
* 84838ed0edf7023fad776fdd3c15c90c1e784fb1 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17053)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5643213165
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e98fd38999e74c5053be30e827cba94c7b676e06 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5643720191
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e98fd38999e74c5053be30e827cba94c7b676e06 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5643240167
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e98fd38999e74c5053be30e827cba94c7b676e06 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5644126993
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e98fd38999e74c5053be30e827cba94c7b676e06 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5647514147
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
}, {
"hash" : "84838ed0edf7023fad776fdd3c15c90c1e784fb1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "84838ed0edf7023fad776fdd3c15c90c1e784fb1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e98fd38999e74c5053be30e827cba94c7b676e06 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034)
* 84838ed0edf7023fad776fdd3c15c90c1e784fb1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5643440699
Per-job execution time, this PR's run ([34670498897](https://github.com/apache/hudi/actions/runs/34670498897), [34670498928](https://github.com/apache/hudi/actions/runs/34670498928)) against #19918's run ([34668461677](https://github.com/apache/hudi/actions/runs/34668461677), [34668461644](https://github.com/apache/hudi/actions/runs/34668461644)). Same code apart from this change. Queue time excluded.
| job | #19918 run (min) | this PR (min) | delta (min) |
|---|---|---|---|
| integration-tests | 37 | 28 | -9 |
| test-common-and-other-modules | 34 | 34 | 0 |
| test-flink-1 | 32 | 31 | -1 |
| test-flink-2 | 23 | 23 | 0 |
| test-hudi-hadoop-mr-and-hudi-java-client | 11 | 10 | -1 |
| test-spark-client-and-hadoop-common | 33 | 34 | +1 |
| test-spark-java17-fta | 30 | 35 | +5 |
| test-spark-java17-ftb | 29 | 28 | -1 |
| test-spark-java17-scala-dml-1 | 40 | 38 | -2 |
| test-spark-java17-scala-dml-2 | 22 | 31 | +9 |
| test-spark-java17-scala-other-1 | 40 | 40 | 0 |
| test-spark-java17-scala-other-2 | 37 | 30 | -7 |
| test-spark-java17-ut-fat-and-ftc | 36 | 37 | +1 |
| test-spark-java17-ut-functional | 28 | 28 | 0 |
| test-spark-java17-ut-other | 29 | 32 | +3 |
| test-utilities-rest | 22 | 22 | 0 |
| test-utilities-streamer | 31 | 30 | -1 |
| validate-bundle-spark4 | 10 | 10 | 0 |
| validate-bundles | 16 | 16 | 0 |
| total | 541 | 539 | -2 |
Only integration-tests changed code-wise. Its build step went from 646 s to 479 s (66 modules to 30) and its IT step from 1390 s to 1028 s (64 modules verified to 4). The IT step's log lists the same 25 test classes as before (the 14 integration-test classes plus hudi-integ-test's 11 unit classes), with the same per-module totals. The rest of the delta is runner drift on untouched jobs, in both directions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5645349467
Attempt 4 of the same runs ([34670498897](https://github.com/apache/hudi/actions/runs/34670498897/attempts/4), [34670498928](https://github.com/apache/hudi/actions/runs/34670498928/attempts/4)), with all attempts and #19918's run alongside. Queue time excluded.
| job | #19918 run (min) | attempt 1 (min) | attempt 2 (min) | attempt 3 (min) | attempt 4 (min) |
|---|---|---|---|---|---|
| integration-tests | 37 | 28 | 25 | 27 | 28 |
| test-common-and-other-modules | 34 | 34 | 35 | 34 | 33 |
| test-flink-1 | 32 | 31 | 31 | 29 | 29 |
| test-flink-2 | 23 | 23 | 23 | 23 | 23 |
| test-hudi-hadoop-mr-and-hudi-java-client | 11 | 10 | 11 | 11 | 11 |
| test-spark-client-and-hadoop-common | 33 | 34 | 30 | 35 | 29 |
| test-spark-java17-fta | 30 | 35 | 35 | 40 | 35 |
| test-spark-java17-ftb | 29 | 28 | 28 | 27 | 28 |
| test-spark-java17-scala-dml-1 | 40 | 38 | 38 | 39 | 38 |
| test-spark-java17-scala-dml-2 | 22 | 31 | 25 | 32 | 27 |
| test-spark-java17-scala-other-1 | 40 | 40 | 41 | 37 | 39 |
| test-spark-java17-scala-other-2 | 37 | 30 | 35 | 37 | 36 |
| test-spark-java17-ut-fat-and-ftc | 36 | 37 | 37 | 39 | 39 |
| test-spark-java17-ut-functional | 28 | 28 | 28 | 29 | 28 |
| test-spark-java17-ut-other | 29 | 32 | 26 | 32 | 26 |
| test-utilities-rest | 22 | 22 | 22 | 22 | 20 |
| test-utilities-streamer | 31 | 30 | 30 | 31 | 21 |
| validate-bundle-spark4 | 10 | 10 | 11 | 11 | 10 |
| validate-bundles | 16 | 16 | 19 | 18 | 18 |
| total | 541 | 539 | 531 | 551 | 516 |
| longest job | 40 | 40 | 41 | 40 | 39 |
integration-tests steps, attempt 4: Build Project 495 s (earlier attempts 479, 456, 515; before: 646), IT 1006 s (1028, 971, 985; before: 1390). Four attempts of the same commit: the job it changes came in at 28, 25, 27 and 28 minutes against 37 before, and its IT step stayed below the lowest of about 90 earlier samples every time. The run total landed at 539, 531, 551 and 516 (mean 534 against 541), moved mostly by untouched jobs drawing faster or slower machines: this attempt utilities-streamer was 10 minutes faster and attempt 3 fta and scala-dml-2 were 10 minutes slower, with no code change either time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5643760407
Attempt 2 of the same runs ([34670498897](https://github.com/apache/hudi/actions/runs/34670498897/attempts/2), [34670498928](https://github.com/apache/hudi/actions/runs/34670498928/attempts/2)), against #19918's run as before. Queue time excluded.
| job | #19918 run (min) | attempt 1 (min) | attempt 2 (min) |
|---|---|---|---|
| integration-tests | 37 | 28 | 25 |
| test-common-and-other-modules | 34 | 34 | 35 |
| test-flink-1 | 32 | 31 | 31 |
| test-flink-2 | 23 | 23 | 23 |
| test-hudi-hadoop-mr-and-hudi-java-client | 11 | 10 | 11 |
| test-spark-client-and-hadoop-common | 33 | 34 | 30 |
| test-spark-java17-fta | 30 | 35 | 35 |
| test-spark-java17-ftb | 29 | 28 | 28 |
| test-spark-java17-scala-dml-1 | 40 | 38 | 38 |
| test-spark-java17-scala-dml-2 | 22 | 31 | 25 |
| test-spark-java17-scala-other-1 | 40 | 40 | 41 |
| test-spark-java17-scala-other-2 | 37 | 30 | 35 |
| test-spark-java17-ut-fat-and-ftc | 36 | 37 | 37 |
| test-spark-java17-ut-functional | 28 | 28 | 28 |
| test-spark-java17-ut-other | 29 | 32 | 26 |
| test-utilities-rest | 22 | 22 | 22 |
| test-utilities-streamer | 31 | 30 | 30 |
| validate-bundle-spark4 | 10 | 10 | 11 |
| validate-bundles | 16 | 16 | 19 |
| total | 541 | 539 | 531 |
integration-tests steps, attempt 2: Build Project 456 s (attempt 1: 479, before: 646), IT 971 s (attempt 1: 1028, before: 1390). The other 18 jobs moved by up to 8 minutes in either direction between the two attempts with no code change, which is the noise floor for the run total.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): [DNM] build and verify only the IT modules in integration-tests [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19919:
URL: https://github.com/apache/hudi/pull/19919#issuecomment-5643395077
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034",
"triggerID" : "e98fd38999e74c5053be30e827cba94c7b676e06",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e98fd38999e74c5053be30e827cba94c7b676e06 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17034)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org