You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2026/07/01 04:40:02 UTC
[PR] chore(ci): upload integration-test coverage to Codecov [hudi]
yihua opened a new pull request, #19130:
URL: https://github.com/apache/hudi/pull/19130
### Describe the issue this Pull Request addresses
The `integration-tests` GitHub Actions job runs the Hudi CLI and `hudi-integ-test` integration tests, but reported no code coverage to Codecov. The `integration-tests` Maven profile wired no JaCoCo agent (unlike the `unit-tests` / `functional-tests*` profiles), so no `.exec` data was produced, and the job had no coverage upload step. Because `hudi-cli` is excluded from the unit/functional test jobs, its production code was exercised only by these integration tests and its coverage was never reported.
### Summary and Changelog
Report integration-test coverage to Codecov, closing the `hudi-cli` blind spot.
- `pom.xml`: add a JaCoCo `prepare-agent` execution to the `integration-tests` profile, writing exec data under `target/jacoco-agent/` so the existing `scripts/jacoco/generate_merged_coverage_report.sh` picks it up. The agent is appended to the `argLine` property, which both failsafe (IT) and surefire (integ-test UT) already consume.
- `.github/workflows/bot.yml`: enable JaCoCo on the `UT integ-test` and `IT` steps (`-Djacoco.skip=false`), and add the standard "Generate merged coverage report" + "Upload coverage to Codecov" steps (flag `integration-tests`), mirroring the other test jobs.
Scope is limited to the `integration-tests` job; the Flink integration-test jobs are intentionally left out.
### Impact
CI only; no production code or public API change. The `integration-tests` job now uploads coverage under the `integration-tests` flag.
### Risk Level
low
CI-only change. It reuses the same JaCoCo agent plus merge/upload path already proven by the unit/functional test jobs. Failsafe/surefire memory settings are unchanged, since the agent is appended to the existing `argLine` property rather than replacing it.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): upload integration-test coverage to Codecov [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #19130:
URL: https://github.com/apache/hudi/pull/19130#issuecomment-4850550102
Update: the `integration-tests` job passed and the coverage pipeline ran end to end. Exec files were produced for the non-Flink IT modules (including `hudi-cli`, ~930 KB, and `hudi-integ-test`, ~762 KB), merged cleanly, and uploaded to Codecov under the `integration-tests` flag. No Calcite instrumentation issue surfaced on the non-Flink path.
Note: this is a fork PR, so the upload used Codecov's tokenless path (the `CODECOV_TOKEN` secret is not exposed to fork PRs); it succeeded here, and the token path applies on push to `master`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): upload integration-test coverage to Codecov [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19130:
URL: https://github.com/apache/hudi/pull/19130#discussion_r3503330862
##########
pom.xml:
##########
@@ -2339,6 +2339,24 @@
</execution>
</executions>
</plugin>
+ <!-- Attach the JaCoCo agent so the integration-test (failsafe) and integ-test unit
+ (surefire) runs emit coverage. The agent argument is appended to the argLine
+ property, which both plugins pick up. destFile matches the jacoco-agent path
+ that scripts/jacoco/generate_merged_coverage_report.sh globs for. -->
+ <plugin>
+ <groupId>org.jacoco</groupId>
+ <artifactId>jacoco-maven-plugin</artifactId>
+ <executions>
+ <execution>
+ <goals>
+ <goal>prepare-agent</goal>
Review Comment:
🤖 One difference from the unit/functional-test profiles worth verifying: those use `combine.self="append"`, so they keep the managed `<argLine>@{argLine}</argLine>`. Here the surefire (line 2307) and failsafe (line 2315) configs use `combine.self="override"`, which drops that managed `<argLine>` element — so the agent only reaches the forked JVMs via the implicit `property="argLine"` fallback rather than the late-bound `@{argLine}`. Could you confirm the `.exec` files actually come out non-empty and Codecov shows integration-tests coverage? A silent empty upload would still make the job pass green.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): upload integration-test coverage to Codecov [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19130:
URL: https://github.com/apache/hudi/pull/19130#issuecomment-4850322057
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ae08ebd6111cfef23f6715343a44252e3aafca0d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ae08ebd6111cfef23f6715343a44252e3aafca0d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ae08ebd6111cfef23f6715343a44252e3aafca0d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): upload integration-test coverage to Codecov [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #19130:
URL: https://github.com/apache/hudi/pull/19130#issuecomment-4850345340
Note on scope: this intentionally leaves the Flink jobs (`test-flink-1`, `test-flink-2`) out of coverage.
When Codecov was first wired up (#18230), jacoco was briefly enabled on the Flink jobs and then removed due to a jacoco + Apache Calcite incompatibility. The agent instruments bytecode with synthetic fields/methods, which breaks the reflection-based structural assertions in Calcite's `MetadataDef` constructor, failing with:
```
java.lang.NoClassDefFoundError: Could not initialize class org.apache.calcite.rel.metadata.DefaultRelMetadataProvider
at org.apache.calcite.plan.RelOptCluster.<init>(RelOptCluster.java:97)
at org.apache.flink.table.planner.calcite.FlinkRelOptClusterFactory$.create(FlinkRelOptClusterFactory.scala:36)
```
This PR keeps that constraint intact:
- Coverage is scoped to the `integration-tests` job, whose IT step runs `mvn verify -Pintegration-tests -pl !hudi-flink-datasource/hudi-flink`, so the Flink Calcite planner path is not exercised.
- The `prepare-agent` added to the `integration-tests` profile is a no-op unless a job overrides `-Djacoco.skip=false`. The Flink IT jobs keep the default `-Djacoco.skip` from `MVN_ARGS`, so they stay uninstrumented and the Calcite failure cannot recur.
The one thing to watch is any non-Flink IT that loads Calcite (e.g. Hive-based paths). Monitoring the `integration-tests` run on this PR to confirm it stays green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): upload integration-test coverage to Codecov [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan merged PR #19130:
URL: https://github.com/apache/hudi/pull/19130
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): upload integration-test coverage to Codecov [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19130:
URL: https://github.com/apache/hudi/pull/19130#issuecomment-4850839975
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ae08ebd6111cfef23f6715343a44252e3aafca0d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15066",
"triggerID" : "ae08ebd6111cfef23f6715343a44252e3aafca0d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ae08ebd6111cfef23f6715343a44252e3aafca0d Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15066)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore(ci): upload integration-test coverage to Codecov [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19130:
URL: https://github.com/apache/hudi/pull/19130#issuecomment-4850344812
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ae08ebd6111cfef23f6715343a44252e3aafca0d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15066",
"triggerID" : "ae08ebd6111cfef23f6715343a44252e3aafca0d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ae08ebd6111cfef23f6715343a44252e3aafca0d Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15066)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org