You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "exceptionfactory (via GitHub)" <gi...@apache.org> on 2023/11/07 20:45:44 UTC
[PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
exceptionfactory opened a new pull request, #7995:
URL: https://github.com/apache/nifi/pull/7995
# Summary
[NIFI-12192](https://issues.apache.org/jira/browse/NIFI-12192) Adds code coverage for unit tests using the [JaCoCo Maven Plugin](https://www.jacoco.org/jacoco/trunk/doc/maven.html) reporting to [Codecov](https://about.codecov.io/).
Codecov provides free reporting capabilities for open source projects and enables integration with GitHub workflows using the [Codecov Action](https://github.com/codecov/codecov-action).
The implementation of code coverage reporting updates the GitHub `ci-workflow` to run the Ubuntu Linux Java 21 job with JaCoCo coverage enabled. The JaCoCo plugin writes an aggregated report in XML, using the `nifi-code-coverage` module to aggregate coverage information based on an enumeration of dependencies. The Maven module is necessary for selecting and aggregating coverage reporting information. The Codecov Action reads the JaCoCo XML and sends it to Codecov for processing.
The JaCoCo prepare-agent goal updates the `argLine` property, which required removing custom locale information from the Ubuntu Linux job, instead relying on the default settings of the GitHub runner.
Updates include adding a Codecov badge to the README for displaying coverage on the main branch.
Current code coverage information is available in [Codecov for this feature branch](https://app.codecov.io/gh/exceptionfactory/nifi/tree/NIFI-12192).
# Tracking
Please complete the following tracking steps prior to pull request creation.
### Issue Tracking
- [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
### Pull Request Tracking
- [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
- [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
### Pull Request Formatting
- [X] Pull Request based on current revision of the `main` branch
- [X] Pull Request refers to a feature branch with one commit containing changes
# Verification
Please indicate the verification steps performed prior to pull request creation.
### Build
- [X] Build completed using `mvn clean install -P contrib-check`
- [X] JDK 21
### Licensing
- [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
- [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
### Documentation
- [ ] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on PR #7995:
URL: https://github.com/apache/nifi/pull/7995#issuecomment-1801898174
> As discussed elsewhere, would it be sensible to update the `integration-tests-ci` profile to skip all `ITRedisDistributedMapCacheClientService` ITs as part of this PR - more of them failed the first time in the PR checks, but then succeeded on the retry?
Thanks for the review and feedback @ChrisSamo632! I have also noticed that methods in that class have been unreliable in automated tests, so I changed the exclusion to apply to the entire `ITRedisDistributedMapCacheClientService` class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
Posted by "ChrisSamo632 (via GitHub)" <gi...@apache.org>.
ChrisSamo632 commented on PR #7995:
URL: https://github.com/apache/nifi/pull/7995#issuecomment-1801443699
@exceptionfactory thanks for the update, the code coverage will help improve confidence in the code base going forward.
I've suggeted a couple of options to think about for improving the use of JaCoCo - more for consideration than blockers to merging this PR.
As discussed elsewhere, would it be sensible to update the `integration-tests-ci` profile to skip all `ITRedisDistributedMapCacheClientService` ITs as part of this PR - more of them failed the first time in the PR checks, but then succeeded on the retry?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
Posted by "ChrisSamo632 (via GitHub)" <gi...@apache.org>.
ChrisSamo632 commented on code in PR #7995:
URL: https://github.com/apache/nifi/pull/7995#discussion_r1386290226
##########
pom.xml:
##########
@@ -29,6 +29,7 @@
<module>nifi-framework-api</module>
<module>nifi-server-api</module>
<module>nifi-bootstrap</module>
+ <module>nifi-code-coverage</module>
Review Comment:
I think the other possibie option, to avoid the new `nifi-code-coverage` sub-module, which may become out of step when new things are added to the repo, would be to add:
```xml
<properties>
<jacoco.skip>true</jacoco.skip>
</properties>
```
to all sub-modules where we don't want to collect coverage, e.g. NAR and Bundle sub-modules. So this would remove the need for a new sub-module to maintain, but would mean updating lots of other existing sub-modules instead, then people understanding when to include the `skip` property for new sub-modules in future.
I don't think either option is ideal, but thought I'd raise the alternative for consideration - I'd be happy with either considered option
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
Posted by "ChrisSamo632 (via GitHub)" <gi...@apache.org>.
ChrisSamo632 commented on code in PR #7995:
URL: https://github.com/apache/nifi/pull/7995#discussion_r1386277971
##########
.github/workflows/ci-workflow.yml:
##########
@@ -160,22 +159,19 @@ jobs:
${{ env.MAVEN_COMPILE_COMMAND }}
- name: Maven Verify
env:
- NIFI_CI_LOCALE: >-
- -Duser.language=en
- -Duser.country=AU
- SUREFIRE_OPTS: >-
Review Comment:
It might be possible to retain these args (thus allowing use of JaCoCo in other Actions in future if desired, without needing to change the `mvn` args, by using something like:
```xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>@{jaCoCoArgLine} -Xmx1024m</argLine>
</configuration>
</plugin>
```
within `<pluginManagement/>`, and (within `<build/>`):
```xml
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>jaCoCoArgLine</propertyName>
</configuration>
</execution>
</executions>
</plugin>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #7995:
URL: https://github.com/apache/nifi/pull/7995#discussion_r1386620132
##########
pom.xml:
##########
@@ -29,6 +29,7 @@
<module>nifi-framework-api</module>
<module>nifi-server-api</module>
<module>nifi-bootstrap</module>
+ <module>nifi-code-coverage</module>
Review Comment:
Thanks for the feedback. I considered the option of individually running the JaCoCo [report](https://www.jacoco.org/jacoco/trunk/doc/report-mojo.html) goal, but I went with the [report-aggregate](https://www.jacoco.org/jacoco/trunk/doc/report-aggregate-mojo.html) goal and aggregating module to avoid changing individual Maven configurations.
I agree that neither solution is perfect, as each approach requires an extra element to consider when adding new modules. However, after making recent changes to remove the need for JavaDoc and source plugin skipping in individual modules, I went with the centralized approach. Another positive of this option is that if we don't see coverage information for a particular module, there is only one place to check, versus going through multiple modules.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #7995:
URL: https://github.com/apache/nifi/pull/7995#issuecomment-1800230252
## Welcome to [Codecov](https://codecov.io?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) :tada:
Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.
Thanks for integrating Codecov - We've got you covered :open_umbrella:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #7995:
URL: https://github.com/apache/nifi/pull/7995#discussion_r1386624819
##########
.github/workflows/ci-workflow.yml:
##########
@@ -160,22 +159,19 @@ jobs:
${{ env.MAVEN_COMPILE_COMMAND }}
- name: Maven Verify
env:
- NIFI_CI_LOCALE: >-
- -Duser.language=en
- -Duser.country=AU
- SUREFIRE_OPTS: >-
Review Comment:
I considered similar options as described in the [prepare-agent](https://www.jacoco.org/jacoco/trunk/doc/prepare-agent-mojo.html) goal documentation, but part of the difficulty is that the build workflow was setting a System property, which JaCoCo also needs to set. We could change the System property in the build workflow to a different name, and then use that name for late property evaluation in the maven-surefire-plugin configuration, but that would have involved changes all of the build profiles. I kept the scope of changes narrow in this case, but we could revisit in the future if we need to introduce other arguments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] NIFI-12192 Add Codecov Reporting with JaCoCo [nifi]
Posted by "asfgit (via GitHub)" <gi...@apache.org>.
asfgit closed pull request #7995: NIFI-12192 Add Codecov Reporting with JaCoCo
URL: https://github.com/apache/nifi/pull/7995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org