You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Tyson Hamilton <ty...@google.com> on 2020/05/20 16:40:54 UTC
Code Coverage Tracking
Hello,
I noticed on the github page, there is a badge that reports 100% coverage. This seems suspect, and sure enough after a couple clicks coverall shows that only one file is tracked. A more interesting page is the Builds page [1] that shows the impact of specific PRs.
It would be really nice if there was a way to get a coverage breakdown for Beam by directory, even nicer if that could be displayed like the post-commit test coverage table (though that may be a bit much). I'd also love to see these build coverage feel metrics in the actual PRs if possible.
I'm not familiar with coverall, didn't find any information on cwiki regarding the configuration for Beam, and was wondering if anyone has information?
[1]: https://coveralls.io/repos/140391/builds
Re: Code Coverage Tracking
Posted by Luke Cwik <lc...@google.com>.
Its integrated and available as part of the Jenkins UI:
https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PreCommit_Java_Cron/lastBuild/jacoco/
On Tue, May 26, 2020 at 8:10 AM Tyson Hamilton <ty...@google.com> wrote:
> Huh.. I just realized it is closed already, now I'm confused.
>
> On 2020/05/26 15:08:21, Tyson Hamilton <ty...@google.com> wrote:
> > Yes, this would be a good choice for the fixit. The tracking issue
> exists already too:
> >
> > https://issues.apache.org/jira/browse/BEAM-1132
> >
> > On 2020/05/26 14:00:00, Kenneth Knowles <ke...@apache.org> wrote:
> > > We tried coveralls (I also used it in a few past projects) and it did
> not
> > > really work for us. A service that I really like is codecov.io (and
> they
> > > have a chrome plugin to overlay coverage on PRs) but the granularity of
> > > GitHub permissions was a problem in the past (most GitHub apps require
> > > blanket "write" permission because finer granularity is not offered).
> > >
> > > The truth is that the local coverage data/reports are quite usable
> > > HTML-based things and just need to be uploaded somewhere. The services
> are
> > > nice-to-have, and only really matter if they add something beyond a
> > > presentation of the data.
> > >
> > > For a multi-module gradle project, the coverage data is per module. You
> > > will need to gather it together to get a holistic view. There are some
> > > gradle plugins to do this.
> > >
> > > All of the above would be a good choice for the fixit week, perhaps?
> > >
> > > Kenn
> > >
> > > On Wed, May 20, 2020 at 9:41 AM Tyson Hamilton <ty...@google.com>
> wrote:
> > >
> > > > Hello,
> > > >
> > > > I noticed on the github page, there is a badge that reports 100%
> coverage.
> > > > This seems suspect, and sure enough after a couple clicks coverall
> shows
> > > > that only one file is tracked. A more interesting page is the Builds
> page
> > > > [1] that shows the impact of specific PRs.
> > > >
> > > > It would be really nice if there was a way to get a coverage
> breakdown for
> > > > Beam by directory, even nicer if that could be displayed like the
> > > > post-commit test coverage table (though that may be a bit much). I'd
> also
> > > > love to see these build coverage feel metrics in the actual PRs if
> possible.
> > > >
> > > > I'm not familiar with coverall, didn't find any information on cwiki
> > > > regarding the configuration for Beam, and was wondering if anyone has
> > > > information?
> > > >
> > > >
> > > > [1]: https://coveralls.io/repos/140391/builds
> > > >
> > >
> >
>
Re: Code Coverage Tracking
Posted by Tyson Hamilton <ty...@google.com>.
Huh.. I just realized it is closed already, now I'm confused.
On 2020/05/26 15:08:21, Tyson Hamilton <ty...@google.com> wrote:
> Yes, this would be a good choice for the fixit. The tracking issue exists already too:
>
> https://issues.apache.org/jira/browse/BEAM-1132
>
> On 2020/05/26 14:00:00, Kenneth Knowles <ke...@apache.org> wrote:
> > We tried coveralls (I also used it in a few past projects) and it did not
> > really work for us. A service that I really like is codecov.io (and they
> > have a chrome plugin to overlay coverage on PRs) but the granularity of
> > GitHub permissions was a problem in the past (most GitHub apps require
> > blanket "write" permission because finer granularity is not offered).
> >
> > The truth is that the local coverage data/reports are quite usable
> > HTML-based things and just need to be uploaded somewhere. The services are
> > nice-to-have, and only really matter if they add something beyond a
> > presentation of the data.
> >
> > For a multi-module gradle project, the coverage data is per module. You
> > will need to gather it together to get a holistic view. There are some
> > gradle plugins to do this.
> >
> > All of the above would be a good choice for the fixit week, perhaps?
> >
> > Kenn
> >
> > On Wed, May 20, 2020 at 9:41 AM Tyson Hamilton <ty...@google.com> wrote:
> >
> > > Hello,
> > >
> > > I noticed on the github page, there is a badge that reports 100% coverage.
> > > This seems suspect, and sure enough after a couple clicks coverall shows
> > > that only one file is tracked. A more interesting page is the Builds page
> > > [1] that shows the impact of specific PRs.
> > >
> > > It would be really nice if there was a way to get a coverage breakdown for
> > > Beam by directory, even nicer if that could be displayed like the
> > > post-commit test coverage table (though that may be a bit much). I'd also
> > > love to see these build coverage feel metrics in the actual PRs if possible.
> > >
> > > I'm not familiar with coverall, didn't find any information on cwiki
> > > regarding the configuration for Beam, and was wondering if anyone has
> > > information?
> > >
> > >
> > > [1]: https://coveralls.io/repos/140391/builds
> > >
> >
>
Re: Code Coverage Tracking
Posted by Tyson Hamilton <ty...@google.com>.
Yes, this would be a good choice for the fixit. The tracking issue exists already too:
https://issues.apache.org/jira/browse/BEAM-1132
On 2020/05/26 14:00:00, Kenneth Knowles <ke...@apache.org> wrote:
> We tried coveralls (I also used it in a few past projects) and it did not
> really work for us. A service that I really like is codecov.io (and they
> have a chrome plugin to overlay coverage on PRs) but the granularity of
> GitHub permissions was a problem in the past (most GitHub apps require
> blanket "write" permission because finer granularity is not offered).
>
> The truth is that the local coverage data/reports are quite usable
> HTML-based things and just need to be uploaded somewhere. The services are
> nice-to-have, and only really matter if they add something beyond a
> presentation of the data.
>
> For a multi-module gradle project, the coverage data is per module. You
> will need to gather it together to get a holistic view. There are some
> gradle plugins to do this.
>
> All of the above would be a good choice for the fixit week, perhaps?
>
> Kenn
>
> On Wed, May 20, 2020 at 9:41 AM Tyson Hamilton <ty...@google.com> wrote:
>
> > Hello,
> >
> > I noticed on the github page, there is a badge that reports 100% coverage.
> > This seems suspect, and sure enough after a couple clicks coverall shows
> > that only one file is tracked. A more interesting page is the Builds page
> > [1] that shows the impact of specific PRs.
> >
> > It would be really nice if there was a way to get a coverage breakdown for
> > Beam by directory, even nicer if that could be displayed like the
> > post-commit test coverage table (though that may be a bit much). I'd also
> > love to see these build coverage feel metrics in the actual PRs if possible.
> >
> > I'm not familiar with coverall, didn't find any information on cwiki
> > regarding the configuration for Beam, and was wondering if anyone has
> > information?
> >
> >
> > [1]: https://coveralls.io/repos/140391/builds
> >
>
Re: Code Coverage Tracking
Posted by Kenneth Knowles <ke...@apache.org>.
We tried coveralls (I also used it in a few past projects) and it did not
really work for us. A service that I really like is codecov.io (and they
have a chrome plugin to overlay coverage on PRs) but the granularity of
GitHub permissions was a problem in the past (most GitHub apps require
blanket "write" permission because finer granularity is not offered).
The truth is that the local coverage data/reports are quite usable
HTML-based things and just need to be uploaded somewhere. The services are
nice-to-have, and only really matter if they add something beyond a
presentation of the data.
For a multi-module gradle project, the coverage data is per module. You
will need to gather it together to get a holistic view. There are some
gradle plugins to do this.
All of the above would be a good choice for the fixit week, perhaps?
Kenn
On Wed, May 20, 2020 at 9:41 AM Tyson Hamilton <ty...@google.com> wrote:
> Hello,
>
> I noticed on the github page, there is a badge that reports 100% coverage.
> This seems suspect, and sure enough after a couple clicks coverall shows
> that only one file is tracked. A more interesting page is the Builds page
> [1] that shows the impact of specific PRs.
>
> It would be really nice if there was a way to get a coverage breakdown for
> Beam by directory, even nicer if that could be displayed like the
> post-commit test coverage table (though that may be a bit much). I'd also
> love to see these build coverage feel metrics in the actual PRs if possible.
>
> I'm not familiar with coverall, didn't find any information on cwiki
> regarding the configuration for Beam, and was wondering if anyone has
> information?
>
>
> [1]: https://coveralls.io/repos/140391/builds
>