You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dubbo.apache.org by htynkn <ht...@163.com> on 2018/04/10 02:21:12 UTC
[Suggestion] PR failing if test coverage went down
hey team,
In pull request template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), this is one item to ask all pr to
"Write necessary unit-test to verify your logic correction"
Codecov has feature to fail PR if test coverage went down. Which means for any pull request, if it add some code without any unit test, the PR will fail
This can help to increase test coverage.
Any thoughts?
Relate links:
1.https://docs.codecov.io/docs/commit-status
2.https://github.com/codecov/support/issues/351
Re: [Suggestion] PR failing if test coverage went down
Posted by ht...@gmail.com,
ht...@gmail.com.
Cool.
I'm busy recently and may not have time to make this change.
So I'm going to raise a issue in github repo and put everything together, if anyone interested in this, he/she can help with this.
On 2018/04/16 04:04:39, shang zonghai <yi...@hotmail.com> wrote:
> +1 for considering small changes.
>
> We need resources to improve existing unit tests and increase unit coverage.
>
> Best Regards！
> yiji
Re: [Suggestion] PR failing if test coverage went down
Posted by shang zonghai <yi...@hotmail.com>.
+1 for considering small changes.
We need resources to improve existing unit tests and increase unit coverage.
Best Regards！
yiji
Re: [Suggestion] PR failing if test coverage went down
Posted by Huxing Zhang <hu...@apache.org>.
+1 for considering small changes.
Pull request is welcome!
On Mon, Apr 16, 2018 at 10:56 AM, htynkn@gmail.com <ht...@gmail.com> wrote:
> If we just want to make sure we don't reject any document fix or tiny fix. X should be 0.05
> If we want to accept some small change, we can consider 0.1 to 0.3
>
> Regards,
> Yunkun Huang
>
> On 2018/04/16 01:36:17, Huxing Zhang <hu...@apache.org> wrote:
>> Hi,
>>
>> On Thu, Apr 12, 2018 at 11:07 AM, htynkn@gmail.com <ht...@gmail.com> wrote:
>> > yeah. I understand your concern.
>> >
>> > This is a common requirement for coverage calc service. Codecov support this feature.
>> > In codecov.yml, we can config threshold value which allow the coverage to drop by X%, and posting a success status.
>>
>> That is awesome!
>>
>> What is your suggestion of X?
>>
>> >
>> > Regards,
>> > Yunkun Huang
>> >
>> > On 2018/04/10 06:36:03, Huxing Zhang <hu...@apache.org> wrote:
>> >> Hi,
>> >>
>> >> Thanks for the suggestion.
>> >>
>> >> I generally agree with the idea that we should prevent a pull request
>> >> to be merged if the code coverage gets lower.
>> >>
>> >> But my concern is the code coverage might have a minor drop even when
>> >> just fixing typo[1], which makes confused.
>> >>
>> >> Unless we can figure it out, or I think it will be better if we can
>> >> set a threshold.
>> >>
>> >> For example, if the code coverage drops >=0.1%, prevent the PR to be merged.
>> >>
>> >> [1] https://github.com/apache/incubator-dubbo/pull/1562
>> >>
>> >> On Tue, Apr 10, 2018 at 10:21 AM, htynkn <ht...@163.com> wrote:
>> >> > hey team,
>> >> >
>> >> >
>> >> > In pull request template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), this is one item to ask all pr to
>> >> >
>> >> >
>> >> > "Write necessary unit-test to verify your logic correction"
>> >> >
>> >> >
>> >> > Codecov has feature to fail PR if test coverage went down. Which means for any pull request, if it add some code without any unit test, the PR will fail
>> >> > This can help to increase test coverage.
>> >> >
>> >> >
>> >> > Any thoughts?
>> >> >
>> >> >
>> >> > Relate links:
>> >> > 1.https://docs.codecov.io/docs/commit-status
>> >> > 2.https://github.com/codecov/support/issues/351
>> >> >
>> >>
>> >> --
>> >> Best Regards！
>> >> Huxing
>> >>
>>
>>
>>
>> --
>> Best Regards！
>> Huxing
>>
--
Best Regards！
Huxing
Re: [Suggestion] PR failing if test coverage went down
Posted by ht...@gmail.com,
ht...@gmail.com.
If we just want to make sure we don't reject any document fix or tiny fix. X should be 0.05
If we want to accept some small change, we can consider 0.1 to 0.3
Regards,
Yunkun Huang
On 2018/04/16 01:36:17, Huxing Zhang <hu...@apache.org> wrote:
> Hi,
>
> On Thu, Apr 12, 2018 at 11:07 AM, htynkn@gmail.com <ht...@gmail.com> wrote:
> > yeah. I understand your concern.
> >
> > This is a common requirement for coverage calc service. Codecov support this feature.
> > In codecov.yml, we can config threshold value which allow the coverage to drop by X%, and posting a success status.
>
> That is awesome!
>
> What is your suggestion of X?
>
> >
> > Regards,
> > Yunkun Huang
> >
> > On 2018/04/10 06:36:03, Huxing Zhang <hu...@apache.org> wrote:
> >> Hi,
> >>
> >> Thanks for the suggestion.
> >>
> >> I generally agree with the idea that we should prevent a pull request
> >> to be merged if the code coverage gets lower.
> >>
> >> But my concern is the code coverage might have a minor drop even when
> >> just fixing typo[1], which makes confused.
> >>
> >> Unless we can figure it out, or I think it will be better if we can
> >> set a threshold.
> >>
> >> For example, if the code coverage drops >=0.1%, prevent the PR to be merged.
> >>
> >> [1] https://github.com/apache/incubator-dubbo/pull/1562
> >>
> >> On Tue, Apr 10, 2018 at 10:21 AM, htynkn <ht...@163.com> wrote:
> >> > hey team,
> >> >
> >> >
> >> > In pull request template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), this is one item to ask all pr to
> >> >
> >> >
> >> > "Write necessary unit-test to verify your logic correction"
> >> >
> >> >
> >> > Codecov has feature to fail PR if test coverage went down. Which means for any pull request, if it add some code without any unit test, the PR will fail
> >> > This can help to increase test coverage.
> >> >
> >> >
> >> > Any thoughts?
> >> >
> >> >
> >> > Relate links:
> >> > 1.https://docs.codecov.io/docs/commit-status
> >> > 2.https://github.com/codecov/support/issues/351
> >> >
> >>
> >> --
> >> Best Regards！
> >> Huxing
> >>
>
>
>
> --
> Best Regards！
> Huxing
>
Re: [Suggestion] PR failing if test coverage went down
Posted by Huxing Zhang <hu...@apache.org>.
Hi,
On Thu, Apr 12, 2018 at 11:07 AM, htynkn@gmail.com <ht...@gmail.com> wrote:
> yeah. I understand your concern.
>
> This is a common requirement for coverage calc service. Codecov support this feature.
> In codecov.yml, we can config threshold value which allow the coverage to drop by X%, and posting a success status.
That is awesome!
What is your suggestion of X?
>
> Regards,
> Yunkun Huang
>
> On 2018/04/10 06:36:03, Huxing Zhang <hu...@apache.org> wrote:
>> Hi,
>>
>> Thanks for the suggestion.
>>
>> I generally agree with the idea that we should prevent a pull request
>> to be merged if the code coverage gets lower.
>>
>> But my concern is the code coverage might have a minor drop even when
>> just fixing typo[1], which makes confused.
>>
>> Unless we can figure it out, or I think it will be better if we can
>> set a threshold.
>>
>> For example, if the code coverage drops >=0.1%, prevent the PR to be merged.
>>
>> [1] https://github.com/apache/incubator-dubbo/pull/1562
>>
>> On Tue, Apr 10, 2018 at 10:21 AM, htynkn <ht...@163.com> wrote:
>> > hey team,
>> >
>> >
>> > In pull request template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), this is one item to ask all pr to
>> >
>> >
>> > "Write necessary unit-test to verify your logic correction"
>> >
>> >
>> > Codecov has feature to fail PR if test coverage went down. Which means for any pull request, if it add some code without any unit test, the PR will fail
>> > This can help to increase test coverage.
>> >
>> >
>> > Any thoughts?
>> >
>> >
>> > Relate links:
>> > 1.https://docs.codecov.io/docs/commit-status
>> > 2.https://github.com/codecov/support/issues/351
>> >
>>
>> --
>> Best Regards！
>> Huxing
>>
--
Best Regards！
Huxing
Re: [Suggestion] PR failing if test coverage went down
Posted by ht...@gmail.com,
ht...@gmail.com.
yeah. I understand your concern.
This is a common requirement for coverage calc service. Codecov support this feature.
In codecov.yml, we can config threshold value which allow the coverage to drop by X%, and posting a success status.
Regards,
Yunkun Huang
On 2018/04/10 06:36:03, Huxing Zhang <hu...@apache.org> wrote:
> Hi,
>
> Thanks for the suggestion.
>
> I generally agree with the idea that we should prevent a pull request
> to be merged if the code coverage gets lower.
>
> But my concern is the code coverage might have a minor drop even when
> just fixing typo[1], which makes confused.
>
> Unless we can figure it out, or I think it will be better if we can
> set a threshold.
>
> For example, if the code coverage drops >=0.1%, prevent the PR to be merged.
>
> [1] https://github.com/apache/incubator-dubbo/pull/1562
>
> On Tue, Apr 10, 2018 at 10:21 AM, htynkn <ht...@163.com> wrote:
> > hey team,
> >
> >
> > In pull request template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), this is one item to ask all pr to
> >
> >
> > "Write necessary unit-test to verify your logic correction"
> >
> >
> > Codecov has feature to fail PR if test coverage went down. Which means for any pull request, if it add some code without any unit test, the PR will fail
> > This can help to increase test coverage.
> >
> >
> > Any thoughts?
> >
> >
> > Relate links:
> > 1.https://docs.codecov.io/docs/commit-status
> > 2.https://github.com/codecov/support/issues/351
> >
>
> --
> Best Regards！
> Huxing
>
Re: [Suggestion] PR failing if test coverage went down
Posted by Huxing Zhang <hu...@apache.org>.
Hi,
Thanks for the suggestion.
I generally agree with the idea that we should prevent a pull request
to be merged if the code coverage gets lower.
But my concern is the code coverage might have a minor drop even when
just fixing typo[1], which makes confused.
Unless we can figure it out, or I think it will be better if we can
set a threshold.
For example, if the code coverage drops >=0.1%, prevent the PR to be merged.
[1] https://github.com/apache/incubator-dubbo/pull/1562
On Tue, Apr 10, 2018 at 10:21 AM, htynkn <ht...@163.com> wrote:
> hey team,
>
>
> In pull request template(https://github.com/apache/incubator-dubbo/blob/master/PULL_REQUEST_TEMPLATE.md), this is one item to ask all pr to
>
>
> "Write necessary unit-test to verify your logic correction"
>
>
> Codecov has feature to fail PR if test coverage went down. Which means for any pull request, if it add some code without any unit test, the PR will fail
> This can help to increase test coverage.
>
>
> Any thoughts?
>
>
> Relate links:
> 1.https://docs.codecov.io/docs/commit-status
> 2.https://github.com/codecov/support/issues/351
>
--
Best Regards！
Huxing