You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Manit Gupta via dev <de...@beam.apache.org> on 2022/07/12 14:54:39 UTC
Generating Jacoco code coverage report for a subset of a Gradle Project in Beam
Hi Beam Community,
I am new to Beam, Gradle, and Jacoco. While I am learning all of these
things, there are some issue(s) I have been stuck with for the last couple
of days and would appreciate any help I can get.
I am trying to run JaCoco to generate the code coverage report for the
subset of a gradle project.
Specifically, I am working on the google-cloud-platform gradle project
inside Apache Beam:
https://github.com/apache/beam/tree/master/sdks/java/io/google-cloud-platform
I have come up with the following gradle tasks till now:
task spannerTests(type: Test) {
group = "Verification"
filter {
includeTestsMatching 'org.apache.beam.sdk.io.gcp.spanner.*'
excludeTestsMatching 'org.apache.beam.sdk.io.gcp.spanner.*IT'
}
doLast {
println 'Spanner Tests run'
}
}
def jacocoExcludes = [
'**/org/apache/beam/gradle/**',
'**/org/apache/beam/model/**',
'**/AutoValue_*'
]
task gcpCodeCoverageReport(type: JacocoReport, dependsOn: spannerTests) {
group = "Reporting"
classDirectories.setFrom(project.files(
project.fileTree(
dir: "${project.projectDir}",
exclude: jacocoExcludes,
)
)
)
executionData.setFrom(file("${buildDir}/jacoco/spannerTests.exec"))
reports {
html.enabled true
html.destination file("${buildDir}/reports/jacoco/gcp/")
}
}
I am facing two issues:
*Issue 1:*
With spannerTests tasks, I am able to filter and only run the tests inside
the spanner/ directory, which is what I want.
The problem starts is happening with gcpCodeCoverageReport task. Currently,
this task takes ${project.projectDir} directory as input to classDirectories.
Since this directory is the project directory, it contains the entire
source code under google-cloud-platform inside it. This is not what I want.
However, on attempting to modify this to:
project.fileTree(
dir:
"${project.projectDir}/src/main/java/org/apache/beam/sdk/io/gcp/spanner",
exclude: jacocoExcludes,
)
The jacoco plugin fails to identify any class files and just outputs an
empty report. What is the correct way to pick only the classes from the
spanner directory within gcp?
*Issue 2:*
Even for the coverage report for the entire google-cloud-platform project
(which I generated by switching to the default test target instead of the
spannerTests one in the gcpCodeCoverageReport task), I am seeing weird file
paths such as org.jacoco.agent.rt.internal present as part of the report.
I have tried removing these paths by adding them to jacocoExcludes, but
that does not work.
[image: Screen Shot 2022-07-12 at 7.49.09 PM.png]
Can anyone assist me in solving both of these issues?
TL;DR -
1. Is it possible to/How to specify a non-project directory for
classDirectories in jacoco?
2. How to exclude code paths like org.jacoco.agent.rt.internal from the
coverage report.
Sorry for the long post, I am just starting out with Apache Beam and
largely a beginner.
Thanks,
Manit
Re: Generating Jacoco code coverage report for a subset of a Gradle Project in Beam
Posted by Luke Cwik via dev <de...@beam.apache.org>.
I commented on the github link directly.
You typically have to declare the tasks inputs/outputs appropriately
otherwise you get said warnings. Most tasks handle this for you if you
configure them appropriately. It is likely your configuring the task in a
way that was unexpected or your missing an additional property. I don't
have any further suggestions without diving into the JacocoCoverageReport
gradle task in more detail.
On Tue, Jul 12, 2022 at 10:44 AM Manit Gupta <ma...@google.com> wrote:
> Adding the commit link with my changes so that it's easier to follow: Github
> link
> <https://github.com/manitgupta/beam/commit/cc78f73bd1be7e99551e5a71d9001e17e50b1985>
>
> Thanks,
> Manit
>
> On Tue, Jul 12, 2022 at 10:56 PM Manit Gupta <ma...@google.com>
> wrote:
>
>> Thanks Luke for the response,
>> What I am currently working on specific to Spanner. I am not sure we can
>> expand the scope to all of google-cloud-platform.
>>
>> I was able to follow a similar approach to solve both of the issues by
>> using includes and the excludes parameters provided by the fileTree API
>> like so:
>>
>> classDirectories.setFrom(project.files(
>> project.fileTree(
>> dir: "${project.projectDir}",
>> includes: spannerIncludes,
>> excludes: jacocoExcludes
>> )
>> )
>> )
>> spannerIncludes and jacocoExcludes are the ArrayLists for the classes
>> which I want to include and exclude respectively.
>>
>> While my final task works as intended now, I am seeing some warnings
>> around implicit dependencies when I add the following line to build.gradle
>> of google-cloud-platform project:
>>
>> build.finalizedBy "spannerCodeCoverageVerification"
>>
>> Sample Run
>>
>> ❯ ./gradlew -p sdks/java/io/google-cloud-platform build
>> Configuration on demand is an incubating feature.
>>
>> > Task :sdks:java:io:google-cloud-platform:spannerCodeCoverageReport
>> Execution optimizations have been disabled for task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageReport' to ensure
>> correctness due to the following reasons:
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageReport' uses this
>> output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:spotlessJava'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageReport' uses this
>> output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:startScripts'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageReport' uses this
>> output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:startShadowScripts'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageReport' uses this
>> output of task ':sdks:java:io:google-cloud-platform:javadoc' without
>> declaring an explicit or implicit dependency. This can lead to incorrect
>> results being produced, depending on what order the tasks are executed.
>> Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageReport' uses this
>> output of task ':sdks:java:io:google-cloud-platform:spotlessJava' without
>> declaring an explicit or implicit dependency. This can lead to incorrect
>> results being produced, depending on what order the tasks are executed.
>> Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageReport' uses this
>> output of task ':sdks:java:io:google-cloud-platform:test' without declaring
>> an explicit or implicit dependency. This can lead to incorrect results
>> being produced, depending on what order the tasks are executed. Please
>> refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>>
>> > Task
>> :sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification FAILED
>> Execution optimizations have been disabled for task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' to
>> ensure correctness due to the following reasons:
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' uses
>> this output of task
>> ':sdks:java:extensions:schemaio-expansion-service:compileJava' without
>> declaring an explicit or implicit dependency. This can lead to incorrect
>> results being produced, depending on what order the tasks are executed.
>> Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' uses
>> this output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:spotlessJava'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' uses
>> this output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:startScripts'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' uses
>> this output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:startShadowScripts'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' uses
>> this output of task ':sdks:java:io:jdbc:compileJava' without declaring an
>> explicit or implicit dependency. This can lead to incorrect results being
>> produced, depending on what order the tasks are executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' uses
>> this output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:spotlessJava'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' uses
>> this output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:startScripts'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> - Gradle detected a problem with the following location:
>> '/Users/manitgupta/beam/sdks/java/io/google-cloud-platform'. Reason: Task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification' uses
>> this output of task
>> ':sdks:java:io:google-cloud-platform:expansion-service:startShadowScripts'
>> without declaring an explicit or implicit dependency. This can lead to
>> incorrect results being produced, depending on what order the tasks are
>> executed. Please refer to
>> https://docs.gradle.org/7.4/userguide/validation_problems.html#implicit_dependency
>> for more details about this problem.
>> [ant:jacocoReport] Rule violated for bundle google-cloud-platform:
>> instructions covered ratio is 0.88, but expected minimum is 0.95
>>
>> FAILURE: Build failed with an exception.
>>
>> * What went wrong:
>> Execution failed for task
>> ':sdks:java:io:google-cloud-platform:spannerCodeCoverageVerification'.
>> > Rule violated for bundle google-cloud-platform: instructions covered
>> ratio is 0.88, but expected minimum is 0.95
>>
>> Given the definition of the tasks in the earlier email, I don't fully
>> understand how my newly created task is having implicit dependencies on
>> unrelated tasks such as spotlessJava or
>> expansion-service:startShadowScripts.
>>
>> Can someone elaborate on this?
>> Thanks!
>>
>> On Tue, Jul 12, 2022 at 10:32 PM Luke Cwik <lc...@google.com> wrote:
>>
>>> Wouldn't we want code coverage for all of google-cloud-platform?
>>>
>>> 1: This has to point at compiled classes, not the source set.
>>> 2: Configure the includes/excludes parameter on the jacoco test task
>>> extension like:
>>> task spannerTests(type: Test) {
>>> ...
>>> jacoco {
>>> includes = [...]
>>> excludes = [...]
>>> }
>>> }
>>>
>>> See:
>>>
>>> https://docs.gradle.org/current/dsl/org.gradle.testing.jacoco.plugins.JacocoTaskExtension.html#org.gradle.testing.jacoco.plugins.JacocoTaskExtension:excludes
>>>
>>> On Tue, Jul 12, 2022 at 9:35 AM Manit Gupta via dev <de...@beam.apache.org>
>>> wrote:
>>>
>>>> Hi Beam Community,
>>>>
>>>> I am new to Beam, Gradle, and Jacoco. While I am learning all of these
>>>> things, there are some issue(s) I have been stuck with for the last couple
>>>> of days and would appreciate any help I can get.
>>>>
>>>> I am trying to run JaCoco to generate the code coverage report for the
>>>> subset of a gradle project.
>>>> Specifically, I am working on the google-cloud-platform gradle project
>>>> inside Apache Beam:
>>>>
>>>> https://github.com/apache/beam/tree/master/sdks/java/io/google-cloud-platform
>>>>
>>>> I have come up with the following gradle tasks till now:
>>>>
>>>> task spannerTests(type: Test) {
>>>> group = "Verification"
>>>> filter {
>>>> includeTestsMatching 'org.apache.beam.sdk.io.gcp.spanner.*'
>>>> excludeTestsMatching 'org.apache.beam.sdk.io.gcp.spanner.*IT'
>>>> }
>>>> doLast {
>>>> println 'Spanner Tests run'
>>>> }
>>>> }
>>>>
>>>> def jacocoExcludes = [
>>>> '**/org/apache/beam/gradle/**',
>>>> '**/org/apache/beam/model/**',
>>>> '**/AutoValue_*'
>>>> ]
>>>>
>>>> task gcpCodeCoverageReport(type: JacocoReport, dependsOn: spannerTests)
>>>> {
>>>> group = "Reporting"
>>>> classDirectories.setFrom(project.files(
>>>> project.fileTree(
>>>> dir: "${project.projectDir}",
>>>> exclude: jacocoExcludes,
>>>> )
>>>> )
>>>> )
>>>> executionData.setFrom(file("${buildDir}/jacoco/spannerTests.exec"))
>>>> reports {
>>>> html.enabled true
>>>> html.destination file("${buildDir}/reports/jacoco/gcp/")
>>>> }
>>>> }
>>>>
>>>> I am facing two issues:
>>>>
>>>> *Issue 1:*
>>>> With spannerTests tasks, I am able to filter and only run the tests
>>>> inside the spanner/ directory, which is what I want.
>>>>
>>>> The problem starts is happening with gcpCodeCoverageReport task.
>>>> Currently, this task takes ${project.projectDir} directory as input to
>>>> classDirectories. Since this directory is the project directory, it
>>>> contains the entire source code under google-cloud-platform inside it.
>>>> This is not what I want.
>>>>
>>>> However, on attempting to modify this to:
>>>>
>>>> project.fileTree(
>>>> dir:
>>>> "${project.projectDir}/src/main/java/org/apache/beam/sdk/io/gcp/spanner",
>>>> exclude: jacocoExcludes,
>>>> )
>>>>
>>>> The jacoco plugin fails to identify any class files and just outputs an
>>>> empty report. What is the correct way to pick only the classes from the
>>>> spanner directory within gcp?
>>>>
>>>> *Issue 2:*
>>>>
>>>> Even for the coverage report for the entire google-cloud-platform
>>>> project (which I generated by switching to the default test target
>>>> instead of the spannerTests one in the gcpCodeCoverageReport task), I
>>>> am seeing weird file paths such as org.jacoco.agent.rt.internal present
>>>> as part of the report. I have tried removing these paths by adding
>>>> them to jacocoExcludes, but that does not work.
>>>>
>>>> [image: Screen Shot 2022-07-12 at 7.49.09 PM.png]
>>>>
>>>> Can anyone assist me in solving both of these issues?
>>>>
>>>> TL;DR -
>>>>
>>>> 1. Is it possible to/How to specify a non-project directory for
>>>> classDirectories in jacoco?
>>>> 2. How to exclude code paths like org.jacoco.agent.rt.internal from
>>>> the coverage report.
>>>>
>>>> Sorry for the long post, I am just starting out with Apache Beam and
>>>> largely a beginner.
>>>>
>>>> Thanks,
>>>> Manit
>>>>
>>>
Re: Generating Jacoco code coverage report for a subset of a Gradle Project in Beam
Posted by Luke Cwik via dev <de...@beam.apache.org>.
Wouldn't we want code coverage for all of google-cloud-platform?
1: This has to point at compiled classes, not the source set.
2: Configure the includes/excludes parameter on the jacoco test task
extension like:
task spannerTests(type: Test) {
...
jacoco {
includes = [...]
excludes = [...]
}
}
See:
https://docs.gradle.org/current/dsl/org.gradle.testing.jacoco.plugins.JacocoTaskExtension.html#org.gradle.testing.jacoco.plugins.JacocoTaskExtension:excludes
On Tue, Jul 12, 2022 at 9:35 AM Manit Gupta via dev <de...@beam.apache.org>
wrote:
> Hi Beam Community,
>
> I am new to Beam, Gradle, and Jacoco. While I am learning all of these
> things, there are some issue(s) I have been stuck with for the last couple
> of days and would appreciate any help I can get.
>
> I am trying to run JaCoco to generate the code coverage report for the
> subset of a gradle project.
> Specifically, I am working on the google-cloud-platform gradle project
> inside Apache Beam:
>
> https://github.com/apache/beam/tree/master/sdks/java/io/google-cloud-platform
>
> I have come up with the following gradle tasks till now:
>
> task spannerTests(type: Test) {
> group = "Verification"
> filter {
> includeTestsMatching 'org.apache.beam.sdk.io.gcp.spanner.*'
> excludeTestsMatching 'org.apache.beam.sdk.io.gcp.spanner.*IT'
> }
> doLast {
> println 'Spanner Tests run'
> }
> }
>
> def jacocoExcludes = [
> '**/org/apache/beam/gradle/**',
> '**/org/apache/beam/model/**',
> '**/AutoValue_*'
> ]
>
> task gcpCodeCoverageReport(type: JacocoReport, dependsOn: spannerTests) {
> group = "Reporting"
> classDirectories.setFrom(project.files(
> project.fileTree(
> dir: "${project.projectDir}",
> exclude: jacocoExcludes,
> )
> )
> )
> executionData.setFrom(file("${buildDir}/jacoco/spannerTests.exec"))
> reports {
> html.enabled true
> html.destination file("${buildDir}/reports/jacoco/gcp/")
> }
> }
>
> I am facing two issues:
>
> *Issue 1:*
> With spannerTests tasks, I am able to filter and only run the tests
> inside the spanner/ directory, which is what I want.
>
> The problem starts is happening with gcpCodeCoverageReport task.
> Currently, this task takes ${project.projectDir} directory as input to
> classDirectories. Since this directory is the project directory, it
> contains the entire source code under google-cloud-platform inside it.
> This is not what I want.
>
> However, on attempting to modify this to:
>
> project.fileTree(
> dir:
> "${project.projectDir}/src/main/java/org/apache/beam/sdk/io/gcp/spanner",
> exclude: jacocoExcludes,
> )
>
> The jacoco plugin fails to identify any class files and just outputs an
> empty report. What is the correct way to pick only the classes from the
> spanner directory within gcp?
>
> *Issue 2:*
>
> Even for the coverage report for the entire google-cloud-platform project
> (which I generated by switching to the default test target instead of the
> spannerTests one in the gcpCodeCoverageReport task), I am seeing weird
> file paths such as org.jacoco.agent.rt.internal present as part of the
> report. I have tried removing these paths by adding them to jacocoExcludes,
> but that does not work.
>
> [image: Screen Shot 2022-07-12 at 7.49.09 PM.png]
>
> Can anyone assist me in solving both of these issues?
>
> TL;DR -
>
> 1. Is it possible to/How to specify a non-project directory for
> classDirectories in jacoco?
> 2. How to exclude code paths like org.jacoco.agent.rt.internal from
> the coverage report.
>
> Sorry for the long post, I am just starting out with Apache Beam and
> largely a beginner.
>
> Thanks,
> Manit
>