You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Dongjoon Hyun <do...@apache.org> on 2016/01/27 02:17:19 UTC
"The overhead of reviewing and testing"
Hi,
In these days, does REEF project have any limitations or minimum
requirements about contributions?
I've never thought I made a burden to REEF community, but, when I made the
following four issues and PRs yesterday and I received a phrase, "The
overhead of reviewing and testing", from a reviewer. Of course, I know the
good intention of that, but I'm just wondering if that is really true.
[REEF-1150] Use `static` members properly by removing access by `instance`
reference
[REEF-1151] Remove illformed `String.format` usage in
TestClassHierarchyRoundTrip
[REEF-1152] Remove duplicate plugin declaration in reef-vortex.
[REEF-1153] Fix typos in class/function names
In fact, during last eight weeks, I'm the one who consumed 22 PRs. I don't
think REEF community is overwhelmed by 'the overhead of reviewing and
testing' at all. The size of queue of PRs is very short.
$ git log --since=8.weeks --pretty=format:"%ce" | sort | uniq -c | sort -nr
| head -n 5
22 dongjoon@apache.org
20 weimer@apache.org
18 mariia@apache.org
11 afchung90@gmail.com
6 yunseong@apache.org
Let's go back to the first question. Actually, we don't have anything like
that. If there is any limitation or minimum requirements for PRs, that
will be a hurdle for REEF newcomers.
IMHO, if we really face `the overhead of reviewing and testing`, it's not a
problem of contributors of those PRs. It's a problem of lack of reviewer
pool. Every committers (including PMCs) should try to do reviews. That is
the difference between a committer and a contributor.
On more thing, you can give my PRs '-1' after fast reviews, too. :)
Warmly,
Dongjoon.
Re: "The overhead of reviewing and testing"
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-01-27 10:13, Andrew Chung wrote:
> 2. blocking a PR based on work outside of the scope of the PR itself or
> posing hard questions to a contributor (automating typo detection, in this
> case) may make the new contributors feel unwelcome.
Ah, thanks! I completely missed that.
> [...]
> Ideally, I would prefer harder discussions to be moved to the mailing list
> and/or opening separate JIRA items for it and move the discussion there
+1 We've done that in the past. It allows the change at hand to move
forward. It also recognizes the need for a more sustainable solution. In
those cases, we've filled the JIRA before merging the PR. That way, we
make sure that important issues aren't forgotten when the PR is closed.
Markus
Re: "The overhead of reviewing and testing"
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-01-27 16:49, Dongjoon Hyun wrote:
> I mean '[MINOR]' tag is adapted to REEF after resolving REEF-111 (.Net CI)
> and REEF-1040 (WatchTest failures)?
+1 from my side. We should define what [MINOR] shall mean, such that
contributors have a clear idea when to use it.
Gon: Have you ever written down what it means in your projects? Can we
steal that definition for REEF's contributor guide?
Markus
Re: "The overhead of reviewing and testing"
Posted by Dongjoon Hyun <do...@apache.org>.
I mean '[MINOR]' tag is adapted to REEF after resolving REEF-111 (.Net CI)
and REEF-1040 (WatchTest failures)?
Warmly,
Dongjoon.
On Wed, Jan 27, 2016 at 4:48 PM, Dongjoon Hyun <do...@apache.org> wrote:
> That sounds great!
>
> Then, the only remaining hurdles for that is REEF-111 and REEF-1040 ?
>
> Dongjoon.
>
> On Wed, Jan 27, 2016 at 4:41 PM, Markus Weimer <ma...@weimo.de> wrote:
>
>> On 2016-01-27 16:02, Byung-Gon Chun wrote:
>>
>>> I see. Thanks for letting us know.
>>> The CI setup's really valuable to the community.
>>>
>>
>> + A Million
>>
>> Once we have it, the minimal effort of testing and merging trivial
>> changes will go down from half an hour to just a few minutes, like it is
>> for pure Java changes right now.
>>
>> Markus
>>
>
>
Re: "The overhead of reviewing and testing"
Posted by Dongjoon Hyun <do...@apache.org>.
That sounds great!
Then, the only remaining hurdles for that is REEF-111 and REEF-1040 ?
Dongjoon.
On Wed, Jan 27, 2016 at 4:41 PM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-01-27 16:02, Byung-Gon Chun wrote:
>
>> I see. Thanks for letting us know.
>> The CI setup's really valuable to the community.
>>
>
> + A Million
>
> Once we have it, the minimal effort of testing and merging trivial changes
> will go down from half an hour to just a few minutes, like it is for pure
> Java changes right now.
>
> Markus
>
Re: "The overhead of reviewing and testing"
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-01-27 16:02, Byung-Gon Chun wrote:
> I see. Thanks for letting us know.
> The CI setup's really valuable to the community.
+ A Million
Once we have it, the minimal effort of testing and merging trivial
changes will go down from half an hour to just a few minutes, like it is
for pure Java changes right now.
Markus
Re: "The overhead of reviewing and testing"
Posted by Byung-Gon Chun <bg...@gmail.com>.
I see. Thanks for letting us know.
The CI setup's really valuable to the community.
Thanks.
-Gon
On Thu, Jan 28, 2016 at 8:37 AM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-01-27 15:34, Byung-Gon Chun wrote:
>
>> Right. Any way we can set up CI for the .NET side?
>>
>
> Mariia is on it. The tricky bit is that we require a CI service that can
> build .NET *and* Java, as our .NET bits depend on the Java bits...
>
> Markus
>
--
Byung-Gon Chun
Re: "The overhead of reviewing and testing"
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-01-27 15:34, Byung-Gon Chun wrote:
> Right. Any way we can set up CI for the .NET side?
Mariia is on it. The tricky bit is that we require a CI service that can
build .NET *and* Java, as our .NET bits depend on the Java bits...
Markus
Re: "The overhead of reviewing and testing"
Posted by Byung-Gon Chun <bg...@gmail.com>.
Right. Any way we can set up CI for the .NET side?
On Thu, Jan 28, 2016 at 8:33 AM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-01-27 15:24, Byung-Gon Chun wrote:
>
>> Back to the [MINOR] tag. I think this has a value. If we have small
>> code or comment changes [...], we can treat the tag as a signal not
>> to run tests locally if the CI tests pass.
>>
>
> Agreed. The overhead with small code changes today mostly stems from the
> relative immaturity of our .NET build. We don't have CI for it, which
> means that every code change that *could* affect the .NET side needs to
> be tested locally before we can merge it.
>
> Markus
>
--
Byung-Gon Chun
Re: "The overhead of reviewing and testing"
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-01-27 15:24, Byung-Gon Chun wrote:
> Back to the [MINOR] tag. I think this has a value. If we have small
> code or comment changes [...], we can treat the tag as a signal not
> to run tests locally if the CI tests pass.
Agreed. The overhead with small code changes today mostly stems from the
relative immaturity of our .NET build. We don't have CI for it, which
means that every code change that *could* affect the .NET side needs to
be tested locally before we can merge it.
Markus
Re: "The overhead of reviewing and testing"
Posted by Byung-Gon Chun <bg...@gmail.com>.
On Thu, Jan 28, 2016 at 4:08 AM, Markus Weimer <ma...@weimo.de> wrote:
> On 2016-01-27 10:53, Dongjoon Hyun wrote:
>
>> Specially, item 4 needed `Deprecation process` due to the change of
>> `public` functions.
>>
>
> Yes. I believe *that* overhead is what sparked the whole conversation.
> Fortunately, we have production level users even in the current 0.x days.
> At the same time, our code base shows signs of rapid growth, which prompt
> us to do all sorts of cleanup work which sometimes breaks users.
>
> So far, we've made the promise not to break any `public` API in between
> releases. This creates a constant tension between the cost of maintaining
> this promise and the agility of our development. This is most annoying in
> the case of trivial changes like a mere rename, which can really spiral out
> of control:
>
> * in 0.x, introduce the new name, deprecate the old one and document
> it all. Maintain two sets of tests, one per name. File a JIRA to
> remove the old code in release 0.x+1, link it all up correctly.
> * in 0.x+1, remove the old code and tests of it.
>
> With the current release frequency, the two steps happen with at least a
> month in between them. This makes it very hard for people to muster up the
> motivation and discipline to follow through on those.
>
The deprecation process is really painful. At the end of the day, we're
discouraging people to pay attention to such code parts. Thanks to Dongjoon
and others who've been taking the painful tasks!
>
> For the .NET side, we've decided to break this promise in 0.14 as there
> were just too many small, breaking changes to first deprecate and then fix
> in 0.15. Instead, we just collect them as breaking changes for the release
> notes.
>
> Maybe we need to make that the rule instead of the exception? As in: we
> break APIs in between releases, but document these breakages? Especially
> until a 1.0, that could be an OK thing to do?
I'm fine with the suggestion. What do the clients of REEF in production
think?
For new public APIs, we're using "@Unstable" annotations extensively if the
APIs might change in the future. Breaking @Unstable public APIs should be
fine.
It's good to have a release note that describes changes. But this will give
more work to release managers.
Back to the [MINOR] tag. I think this has a value. If we have small code or
comment changes (e.g., fix typos not involving important public APIs or
deprecated methods), we can treat the tag as a signal not to run tests
locally if the CI tests pass. To me, running tests locally is the most
time-consuming part for small changes. Reviewing small changes and merging
them can be quickly done.
The tag is declared by a contributor. The reviewer can check the changes
are actually minor or not in the review process.
>
>
> Markus
>
--
Byung-Gon Chun
Re: "The overhead of reviewing and testing"
Posted by Markus Weimer <ma...@weimo.de>.
On 2016-01-27 10:53, Dongjoon Hyun wrote:
> Specially, item 4 needed `Deprecation process` due to the change of
> `public` functions.
Yes. I believe *that* overhead is what sparked the whole conversation.
Fortunately, we have production level users even in the current 0.x
days. At the same time, our code base shows signs of rapid growth, which
prompt us to do all sorts of cleanup work which sometimes breaks users.
So far, we've made the promise not to break any `public` API in between
releases. This creates a constant tension between the cost of
maintaining this promise and the agility of our development. This is
most annoying in the case of trivial changes like a mere rename, which
can really spiral out of control:
* in 0.x, introduce the new name, deprecate the old one and document
it all. Maintain two sets of tests, one per name. File a JIRA to
remove the old code in release 0.x+1, link it all up correctly.
* in 0.x+1, remove the old code and tests of it.
With the current release frequency, the two steps happen with at least a
month in between them. This makes it very hard for people to muster up
the motivation and discipline to follow through on those.
For the .NET side, we've decided to break this promise in 0.14 as there
were just too many small, breaking changes to first deprecate and then
fix in 0.15. Instead, we just collect them as breaking changes for the
release notes.
Maybe we need to make that the rule instead of the exception? As in: we
break APIs in between releases, but document these breakages? Especially
until a 1.0, that could be an OK thing to do?
Markus
Re: "The overhead of reviewing and testing"
Posted by Dongjoon Hyun <do...@apache.org>.
Thank you, Julia.
That guideline sounds reasonable and good. I also usually do not mix of
them.
By the way, unfortunately, all the following items are not MINOR, then.
1. [REEF-1150] Use `static` members properly by removing access by
`instance` reference 2. [REEF-1151] Remove illformed `String.format` usage
in TestClassHierarchyRoundTrip
3. [REEF-1152] Remove duplicate plugin declaration in reef-vortex.
4. [REEF-1153] Fix typos in class/function names
Specially, item 4 needed `Deprecation process` due to the change of
`public` functions.
Dongjoon.
On Wed, Jan 27, 2016 at 10:42 AM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com> wrote:
> The cost for a review includes
> -Review code content
> -Checkout the branch, build and testing
> -Rebase
> -Merge/push
>
> We had quite some discussions before. Mainly on the other side, when
> content covers too broader areas, it makes code review difficult. So we
> tend to split a big PR into a few smaller one so that it is easy for
> reviewer. For contributor, it decreases the working efficiency as keep
> merging takes time.
> However, when a PR is too small, like fix a typo or couple of lines
> change, the reviewer would take much longer time for testing and merge
> compared with the small change.
> So we just need to find a balance point when creating a PR, except for
> blocking issues or the change nature only involves one line like version
> change.
> One principle that Markus always insists is to separate typo, style and
> syntax change from feature changes. That makes sense because the former
> usually involves a lot of files. If we know it is syntax only, the review
> can be pretty quick.
>
> Thanks,
> Julia
>
>
> -----Original Message-----
> From: Dongjoon Hyun [mailto:dongjoon@apache.org]
> Sent: Wednesday, January 27, 2016 10:24 AM
> To: dev@reef.apache.org
> Subject: Re: "The overhead of reviewing and testing"
>
> Thank you, Gon and Andrew.
>
> I agree with your opinions and suggestions.
>
> Especially, for the '[MINOR]' tag, '[MINOR]' would be great if that reduce
> the burden of reviewers really. Is it used before really? So far, I cannot
> find any commit having '[MINOR]' in its title.
>
> If we have a documentation about HOW TO REVIEW the MINOR tags, that would
> be really helpful for us. And, if possible, any guideline about what is
> minor tag? I'm not sure about this part. Usually, the MINOR tag is the
> intention of contributor, not it of reviewer.
>
> I hope it reduces the number of JIRA issues and maybe some testing if we
> trust our CI.
> For example, the following 1~4 are really small minor implementation
> changes. Can we skip manually testing step for these during reviews if they
> have explicit '[MINOR]' tag? Of course, one of our three CI should passes.
> The purpose of running three REEF CI was 'Speculative Testing <
> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail-archives.apache.org%2fmod_mbox%2freef-dev%2f201510.mbox%2f%253CCAFu6yH3kiX%252BYpHPVGETXoN91MYMfckQviqM7a8%252BHzQRCNROdLA%2540mail.gmail.com%253E&data=01%7c01%7cQiuhe.Wang%40microsoft.com%7ca6a27fa1aacc4886834708d32746f95e%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=xRBmnSI5ybOrVbPMRQfmrW5BWr49Xjb%2f2fQ8G4UKwXg%3d
> >
> '.
>
> 1. [REEF-1150] Use `static` members properly by removing access by
> `instance` reference 2. [REEF-1151] Remove illformed `String.format` usage
> in TestClassHierarchyRoundTrip 3. [REEF-1152] Remove duplicate plugin
> declaration in reef-vortex.
> 4. [REEF-1153] Fix typos in class/function names
>
>
>
>
> On Wed, Jan 27, 2016 at 10:13 AM, Andrew Chung <af...@gmail.com>
> wrote:
>
> > Thanks for your comment Dongjoon!
> >
> > I can understand your argument in that:
> >
> > 1. REEF would like to have minor tasks that don't require extensive
> > knowledge of the codebase such as fixing typos such that we can
> > allow newer
> > contributors to get the gratification of contributing to the
> > codebase and
> > hopefully induce them to making more contributions in the future and
> > 2. blocking a PR based on work outside of the scope of the PR itself
> or
> > posing hard questions to a contributor (automating typo detection,
> > in this
> > case) may make the new contributors feel unwelcome.
> >
> > I can also understand Mariia's argument as to how certain simple
> > processes should be automated.
> >
> > Personally, I am fine with reviewing and merging smaller PRs, but
> > automation is ultimately something that would make everyone's life
> easier.
> > Ideally, I would prefer harder discussions to be moved to the mailing
> > list and/or opening separate JIRA items for it and move the discussion
> > there instead of posing questions directly in the same PR as such to
> > avoid making the contribution feel unwelcome, even though it may not
> > be the intention of the reviewer -- after all, as contributors to the
> > open source project, we are ultimately all trying to improve the
> > codebase quality. :)
> >
> > What does everyone else think?
> >
> > Thanks,
> > Andrew
> >
> > On Wed, Jan 27, 2016 at 1:49 AM, Byung-Gon Chun <bg...@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > We had related discussion in August 2015.
> > >
> > >
> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmail-
> > archives.apache.org%2fmod_mbox%2fincubator-reef-dev%2f201508.mbox%2f%2
> > 53C55DB2CDE.40804%40weimo.de%253E&data=01%7c01%7cQiuhe.Wang%40microsof
> > t.com%7ca6a27fa1aacc4886834708d32746f95e%7c72f988bf86f141af91ab2d7cd01
> > 1db47%7c1&sdata=QCaa4YPwR8k4vZtjv3e0a4MELAnoxFEsWTHAlN7c1go%3d
> > >
> > > The [MINOR] tag has worked really well for SNU internal projects. I
> > > strongly recommend the reef community adopts it. :-)
> > >
> > > Thanks!
> > > -Gon
> > >
> > >
> > >
> > > On Wed, Jan 27, 2016 at 5:11 PM, Dongjoon Hyun <do...@apache.org>
> > > wrote:
> > >
> > > > Oh, I missed your word, "We've had this discussion from the other
> > > > side before".
> > > > It's not a different story. Sorry!
> > > >
> > > > It's too late night. I had better go to bed before making another
> > > mistake.
> > > >
> > > > Dongjoon.
> > > >
> > > >
> > > > On Tue, Jan 26, 2016 at 11:51 PM, Dongjoon Hyun
> > > > <do...@apache.org>
> > > > wrote:
> > > >
> > > > > Thank you for your generous comments. Actually, what I meant to
> > > > > focus
> > > is
> > > > > the vice versa.
> > > > >
> > > > > "The reviewer (in a review cycle) should focus on only the PR
> > > > > code
> > > itself
> > > > > like Blind Paper Review Process."
> > > > >
> > > > > - Why does a reviewer consider a contributor's other PRs?
> > > > > - Why does a reviewer ask something beyond the scope of the PR?
> > > > > (As we know, contributors are not REEF's or PMC's full-time
> > > employees.)
> > > > >
> > > > > I dream that an ideal environment where any tiny contributions
> > > > > from
> > > > anyone
> > > > > are welcome equally(here, blindly) anytime.
> > > > >
> > > > > Although the content of PR does not improve REEF, a reviewer
> > > > > should
> > > say a
> > > > > warm comment like 'Sorry, but thanks for making a PR', and give
> > > > > -1. I
> > > > think
> > > > > that's natural in Open Source Communities. (The PR code will be
> > > > withdrawed
> > > > > or closed.)
> > > > >
> > > > > I know that REEF community consists of strong developers and has
> > > > > warm atmosphere in general. I wrote here because I was surprise
> > > > > about the concept of `overhead`. I really wanted to listen
> > > > > others' opinions,
> > too.
> > > > > Thank you, Markus.
> > > > >
> > > > > Dongjoon.
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Jan 26, 2016 at 10:58 PM, Markus Weimer
> > > > > <ma...@weimo.de>
> > > wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> thanks for bringing this up. It is an interesting, different
> > > > >> angle
> > on
> > > > the
> > > > >> review process. We've had this discussion from the other side
> > before:
> > > > >> contributors felt that the review cycle put a impediment onto
> them.
> > > > >>
> > > > >> I believe both arguments have a kernel of validity, but I also
> > believe
> > > > >> that reviews are super-crucial not only for code quality, but
> > > > >> even
> > > more
> > > > so
> > > > >> for a shared understanding of the code. As you said, every
> > > > >> committer
> > > and
> > > > >> PMC member should use the opportunity to stay current with
> > > > >> REEF's development via the review process. To be honest, that
> > > > >> is often my
> > > > reason
> > > > >> for picking specific PRs: I want to know what's happening in
> > > > >> that
> > part
> > > > of
> > > > >> the code.
> > > > >>
> > > > >> That said, I think we should use every tool and idea available
> > > > >> to
> > > remove
> > > > >> friction from the process. Basic coding standard checks and
> > > > >> test
> > runs
> > > > >> shouldn't be the work of human brains, but CPUs :) The work
> > > > >> Mariia
> > and
> > > > you
> > > > >> have been doing goes a long way towards that. There are some
> > > > >> gaping
> > > > holes
> > > > >> (*ahem* .NET tests) which should be addressed soon.
> > > > >>
> > > > >> But I am rambling: What do other's think? Is the PR review
> > > > >> process
> > too
> > > > >> burdensome? What (beyond the basic continuous integration
> > > > >> setup) can
> > > we
> > > > do
> > > > >> to make it easier?
> > > > >>
> > > > >> Thanks again for bringing this up!
> > > > >>
> > > > >> Markus
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Byung-Gon Chun
> > >
> >
>
RE: "The overhead of reviewing and testing"
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
The cost for a review includes
-Review code content
-Checkout the branch, build and testing
-Rebase
-Merge/push
We had quite some discussions before. Mainly on the other side, when content covers too broader areas, it makes code review difficult. So we tend to split a big PR into a few smaller one so that it is easy for reviewer. For contributor, it decreases the working efficiency as keep merging takes time.
However, when a PR is too small, like fix a typo or couple of lines change, the reviewer would take much longer time for testing and merge compared with the small change.
So we just need to find a balance point when creating a PR, except for blocking issues or the change nature only involves one line like version change.
One principle that Markus always insists is to separate typo, style and syntax change from feature changes. That makes sense because the former usually involves a lot of files. If we know it is syntax only, the review can be pretty quick.
Thanks,
Julia
-----Original Message-----
From: Dongjoon Hyun [mailto:dongjoon@apache.org]
Sent: Wednesday, January 27, 2016 10:24 AM
To: dev@reef.apache.org
Subject: Re: "The overhead of reviewing and testing"
Thank you, Gon and Andrew.
I agree with your opinions and suggestions.
Especially, for the '[MINOR]' tag, '[MINOR]' would be great if that reduce the burden of reviewers really. Is it used before really? So far, I cannot find any commit having '[MINOR]' in its title.
If we have a documentation about HOW TO REVIEW the MINOR tags, that would be really helpful for us. And, if possible, any guideline about what is minor tag? I'm not sure about this part. Usually, the MINOR tag is the intention of contributor, not it of reviewer.
I hope it reduces the number of JIRA issues and maybe some testing if we trust our CI.
For example, the following 1~4 are really small minor implementation changes. Can we skip manually testing step for these during reviews if they have explicit '[MINOR]' tag? Of course, one of our three CI should passes.
The purpose of running three REEF CI was 'Speculative Testing <https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmail-archives.apache.org%2fmod_mbox%2freef-dev%2f201510.mbox%2f%253CCAFu6yH3kiX%252BYpHPVGETXoN91MYMfckQviqM7a8%252BHzQRCNROdLA%2540mail.gmail.com%253E&data=01%7c01%7cQiuhe.Wang%40microsoft.com%7ca6a27fa1aacc4886834708d32746f95e%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=xRBmnSI5ybOrVbPMRQfmrW5BWr49Xjb%2f2fQ8G4UKwXg%3d>
'.
1. [REEF-1150] Use `static` members properly by removing access by `instance` reference 2. [REEF-1151] Remove illformed `String.format` usage in TestClassHierarchyRoundTrip 3. [REEF-1152] Remove duplicate plugin declaration in reef-vortex.
4. [REEF-1153] Fix typos in class/function names
On Wed, Jan 27, 2016 at 10:13 AM, Andrew Chung <af...@gmail.com> wrote:
> Thanks for your comment Dongjoon!
>
> I can understand your argument in that:
>
> 1. REEF would like to have minor tasks that don't require extensive
> knowledge of the codebase such as fixing typos such that we can
> allow newer
> contributors to get the gratification of contributing to the
> codebase and
> hopefully induce them to making more contributions in the future and
> 2. blocking a PR based on work outside of the scope of the PR itself or
> posing hard questions to a contributor (automating typo detection,
> in this
> case) may make the new contributors feel unwelcome.
>
> I can also understand Mariia's argument as to how certain simple
> processes should be automated.
>
> Personally, I am fine with reviewing and merging smaller PRs, but
> automation is ultimately something that would make everyone's life easier.
> Ideally, I would prefer harder discussions to be moved to the mailing
> list and/or opening separate JIRA items for it and move the discussion
> there instead of posing questions directly in the same PR as such to
> avoid making the contribution feel unwelcome, even though it may not
> be the intention of the reviewer -- after all, as contributors to the
> open source project, we are ultimately all trying to improve the
> codebase quality. :)
>
> What does everyone else think?
>
> Thanks,
> Andrew
>
> On Wed, Jan 27, 2016 at 1:49 AM, Byung-Gon Chun <bg...@gmail.com> wrote:
>
> > Hi,
> >
> > We had related discussion in August 2015.
> >
> >
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fmail-
> archives.apache.org%2fmod_mbox%2fincubator-reef-dev%2f201508.mbox%2f%2
> 53C55DB2CDE.40804%40weimo.de%253E&data=01%7c01%7cQiuhe.Wang%40microsof
> t.com%7ca6a27fa1aacc4886834708d32746f95e%7c72f988bf86f141af91ab2d7cd01
> 1db47%7c1&sdata=QCaa4YPwR8k4vZtjv3e0a4MELAnoxFEsWTHAlN7c1go%3d
> >
> > The [MINOR] tag has worked really well for SNU internal projects. I
> > strongly recommend the reef community adopts it. :-)
> >
> > Thanks!
> > -Gon
> >
> >
> >
> > On Wed, Jan 27, 2016 at 5:11 PM, Dongjoon Hyun <do...@apache.org>
> > wrote:
> >
> > > Oh, I missed your word, "We've had this discussion from the other
> > > side before".
> > > It's not a different story. Sorry!
> > >
> > > It's too late night. I had better go to bed before making another
> > mistake.
> > >
> > > Dongjoon.
> > >
> > >
> > > On Tue, Jan 26, 2016 at 11:51 PM, Dongjoon Hyun
> > > <do...@apache.org>
> > > wrote:
> > >
> > > > Thank you for your generous comments. Actually, what I meant to
> > > > focus
> > is
> > > > the vice versa.
> > > >
> > > > "The reviewer (in a review cycle) should focus on only the PR
> > > > code
> > itself
> > > > like Blind Paper Review Process."
> > > >
> > > > - Why does a reviewer consider a contributor's other PRs?
> > > > - Why does a reviewer ask something beyond the scope of the PR?
> > > > (As we know, contributors are not REEF's or PMC's full-time
> > employees.)
> > > >
> > > > I dream that an ideal environment where any tiny contributions
> > > > from
> > > anyone
> > > > are welcome equally(here, blindly) anytime.
> > > >
> > > > Although the content of PR does not improve REEF, a reviewer
> > > > should
> > say a
> > > > warm comment like 'Sorry, but thanks for making a PR', and give
> > > > -1. I
> > > think
> > > > that's natural in Open Source Communities. (The PR code will be
> > > withdrawed
> > > > or closed.)
> > > >
> > > > I know that REEF community consists of strong developers and has
> > > > warm atmosphere in general. I wrote here because I was surprise
> > > > about the concept of `overhead`. I really wanted to listen
> > > > others' opinions,
> too.
> > > > Thank you, Markus.
> > > >
> > > > Dongjoon.
> > > >
> > > >
> > > >
> > > > On Tue, Jan 26, 2016 at 10:58 PM, Markus Weimer
> > > > <ma...@weimo.de>
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> thanks for bringing this up. It is an interesting, different
> > > >> angle
> on
> > > the
> > > >> review process. We've had this discussion from the other side
> before:
> > > >> contributors felt that the review cycle put a impediment onto them.
> > > >>
> > > >> I believe both arguments have a kernel of validity, but I also
> believe
> > > >> that reviews are super-crucial not only for code quality, but
> > > >> even
> > more
> > > so
> > > >> for a shared understanding of the code. As you said, every
> > > >> committer
> > and
> > > >> PMC member should use the opportunity to stay current with
> > > >> REEF's development via the review process. To be honest, that
> > > >> is often my
> > > reason
> > > >> for picking specific PRs: I want to know what's happening in
> > > >> that
> part
> > > of
> > > >> the code.
> > > >>
> > > >> That said, I think we should use every tool and idea available
> > > >> to
> > remove
> > > >> friction from the process. Basic coding standard checks and
> > > >> test
> runs
> > > >> shouldn't be the work of human brains, but CPUs :) The work
> > > >> Mariia
> and
> > > you
> > > >> have been doing goes a long way towards that. There are some
> > > >> gaping
> > > holes
> > > >> (*ahem* .NET tests) which should be addressed soon.
> > > >>
> > > >> But I am rambling: What do other's think? Is the PR review
> > > >> process
> too
> > > >> burdensome? What (beyond the basic continuous integration
> > > >> setup) can
> > we
> > > do
> > > >> to make it easier?
> > > >>
> > > >> Thanks again for bringing this up!
> > > >>
> > > >> Markus
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Byung-Gon Chun
> >
>
Re: "The overhead of reviewing and testing"
Posted by Dongjoon Hyun <do...@apache.org>.
Thank you, Gon and Andrew.
I agree with your opinions and suggestions.
Especially, for the '[MINOR]' tag, '[MINOR]' would be great if that reduce
the burden of reviewers really. Is it used before really? So far, I cannot
find any commit having '[MINOR]' in its title.
If we have a documentation about HOW TO REVIEW the MINOR tags, that would
be really helpful for us. And, if possible, any guideline about what is
minor tag? I'm not sure about this part. Usually, the MINOR tag is the
intention of contributor, not it of reviewer.
I hope it reduces the number of JIRA issues and maybe some testing if we
trust our CI.
For example, the following 1~4 are really small minor implementation
changes. Can we skip manually testing step for these during reviews if they
have explicit '[MINOR]' tag? Of course, one of our three CI should passes.
The purpose of running three REEF CI was 'Speculative Testing
<http://mail-archives.apache.org/mod_mbox/reef-dev/201510.mbox/%3CCAFu6yH3kiX%2BYpHPVGETXoN91MYMfckQviqM7a8%2BHzQRCNROdLA%40mail.gmail.com%3E>
'.
1. [REEF-1150] Use `static` members properly by removing access by
`instance` reference
2. [REEF-1151] Remove illformed `String.format` usage in
TestClassHierarchyRoundTrip
3. [REEF-1152] Remove duplicate plugin declaration in reef-vortex.
4. [REEF-1153] Fix typos in class/function names
On Wed, Jan 27, 2016 at 10:13 AM, Andrew Chung <af...@gmail.com> wrote:
> Thanks for your comment Dongjoon!
>
> I can understand your argument in that:
>
> 1. REEF would like to have minor tasks that don't require extensive
> knowledge of the codebase such as fixing typos such that we can allow
> newer
> contributors to get the gratification of contributing to the codebase
> and
> hopefully induce them to making more contributions in the future and
> 2. blocking a PR based on work outside of the scope of the PR itself or
> posing hard questions to a contributor (automating typo detection, in
> this
> case) may make the new contributors feel unwelcome.
>
> I can also understand Mariia's argument as to how certain simple processes
> should be automated.
>
> Personally, I am fine with reviewing and merging smaller PRs, but
> automation is ultimately something that would make everyone's life easier.
> Ideally, I would prefer harder discussions to be moved to the mailing list
> and/or opening separate JIRA items for it and move the discussion there
> instead of posing questions directly in the same PR as such to avoid making
> the contribution feel unwelcome, even though it may not be the intention of
> the reviewer -- after all, as contributors to the open source project, we
> are ultimately all trying to improve the codebase quality. :)
>
> What does everyone else think?
>
> Thanks,
> Andrew
>
> On Wed, Jan 27, 2016 at 1:49 AM, Byung-Gon Chun <bg...@gmail.com> wrote:
>
> > Hi,
> >
> > We had related discussion in August 2015.
> >
> >
> https://mail-archives.apache.org/mod_mbox/incubator-reef-dev/201508.mbox/%3C55DB2CDE.40804@weimo.de%3E
> >
> > The [MINOR] tag has worked really well for SNU internal projects. I
> > strongly recommend the reef community adopts it. :-)
> >
> > Thanks!
> > -Gon
> >
> >
> >
> > On Wed, Jan 27, 2016 at 5:11 PM, Dongjoon Hyun <do...@apache.org>
> > wrote:
> >
> > > Oh, I missed your word, "We've had this discussion from the other side
> > > before".
> > > It's not a different story. Sorry!
> > >
> > > It's too late night. I had better go to bed before making another
> > mistake.
> > >
> > > Dongjoon.
> > >
> > >
> > > On Tue, Jan 26, 2016 at 11:51 PM, Dongjoon Hyun <do...@apache.org>
> > > wrote:
> > >
> > > > Thank you for your generous comments. Actually, what I meant to focus
> > is
> > > > the vice versa.
> > > >
> > > > "The reviewer (in a review cycle) should focus on only the PR code
> > itself
> > > > like Blind Paper Review Process."
> > > >
> > > > - Why does a reviewer consider a contributor's other PRs?
> > > > - Why does a reviewer ask something beyond the scope of the PR?
> > > > (As we know, contributors are not REEF's or PMC's full-time
> > employees.)
> > > >
> > > > I dream that an ideal environment where any tiny contributions from
> > > anyone
> > > > are welcome equally(here, blindly) anytime.
> > > >
> > > > Although the content of PR does not improve REEF, a reviewer should
> > say a
> > > > warm comment like 'Sorry, but thanks for making a PR', and give -1. I
> > > think
> > > > that's natural in Open Source Communities. (The PR code will be
> > > withdrawed
> > > > or closed.)
> > > >
> > > > I know that REEF community consists of strong developers and has warm
> > > > atmosphere in general. I wrote here because I was surprise about the
> > > > concept of `overhead`. I really wanted to listen others' opinions,
> too.
> > > > Thank you, Markus.
> > > >
> > > > Dongjoon.
> > > >
> > > >
> > > >
> > > > On Tue, Jan 26, 2016 at 10:58 PM, Markus Weimer <ma...@weimo.de>
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> thanks for bringing this up. It is an interesting, different angle
> on
> > > the
> > > >> review process. We've had this discussion from the other side
> before:
> > > >> contributors felt that the review cycle put a impediment onto them.
> > > >>
> > > >> I believe both arguments have a kernel of validity, but I also
> believe
> > > >> that reviews are super-crucial not only for code quality, but even
> > more
> > > so
> > > >> for a shared understanding of the code. As you said, every committer
> > and
> > > >> PMC member should use the opportunity to stay current with REEF's
> > > >> development via the review process. To be honest, that is often my
> > > reason
> > > >> for picking specific PRs: I want to know what's happening in that
> part
> > > of
> > > >> the code.
> > > >>
> > > >> That said, I think we should use every tool and idea available to
> > remove
> > > >> friction from the process. Basic coding standard checks and test
> runs
> > > >> shouldn't be the work of human brains, but CPUs :) The work Mariia
> and
> > > you
> > > >> have been doing goes a long way towards that. There are some gaping
> > > holes
> > > >> (*ahem* .NET tests) which should be addressed soon.
> > > >>
> > > >> But I am rambling: What do other's think? Is the PR review process
> too
> > > >> burdensome? What (beyond the basic continuous integration setup) can
> > we
> > > do
> > > >> to make it easier?
> > > >>
> > > >> Thanks again for bringing this up!
> > > >>
> > > >> Markus
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Byung-Gon Chun
> >
>
Re: "The overhead of reviewing and testing"
Posted by Andrew Chung <af...@gmail.com>.
Thanks for your comment Dongjoon!
I can understand your argument in that:
1. REEF would like to have minor tasks that don't require extensive
knowledge of the codebase such as fixing typos such that we can allow newer
contributors to get the gratification of contributing to the codebase and
hopefully induce them to making more contributions in the future and
2. blocking a PR based on work outside of the scope of the PR itself or
posing hard questions to a contributor (automating typo detection, in this
case) may make the new contributors feel unwelcome.
I can also understand Mariia's argument as to how certain simple processes
should be automated.
Personally, I am fine with reviewing and merging smaller PRs, but
automation is ultimately something that would make everyone's life easier.
Ideally, I would prefer harder discussions to be moved to the mailing list
and/or opening separate JIRA items for it and move the discussion there
instead of posing questions directly in the same PR as such to avoid making
the contribution feel unwelcome, even though it may not be the intention of
the reviewer -- after all, as contributors to the open source project, we
are ultimately all trying to improve the codebase quality. :)
What does everyone else think?
Thanks,
Andrew
On Wed, Jan 27, 2016 at 1:49 AM, Byung-Gon Chun <bg...@gmail.com> wrote:
> Hi,
>
> We had related discussion in August 2015.
>
> https://mail-archives.apache.org/mod_mbox/incubator-reef-dev/201508.mbox/%3C55DB2CDE.40804@weimo.de%3E
>
> The [MINOR] tag has worked really well for SNU internal projects. I
> strongly recommend the reef community adopts it. :-)
>
> Thanks!
> -Gon
>
>
>
> On Wed, Jan 27, 2016 at 5:11 PM, Dongjoon Hyun <do...@apache.org>
> wrote:
>
> > Oh, I missed your word, "We've had this discussion from the other side
> > before".
> > It's not a different story. Sorry!
> >
> > It's too late night. I had better go to bed before making another
> mistake.
> >
> > Dongjoon.
> >
> >
> > On Tue, Jan 26, 2016 at 11:51 PM, Dongjoon Hyun <do...@apache.org>
> > wrote:
> >
> > > Thank you for your generous comments. Actually, what I meant to focus
> is
> > > the vice versa.
> > >
> > > "The reviewer (in a review cycle) should focus on only the PR code
> itself
> > > like Blind Paper Review Process."
> > >
> > > - Why does a reviewer consider a contributor's other PRs?
> > > - Why does a reviewer ask something beyond the scope of the PR?
> > > (As we know, contributors are not REEF's or PMC's full-time
> employees.)
> > >
> > > I dream that an ideal environment where any tiny contributions from
> > anyone
> > > are welcome equally(here, blindly) anytime.
> > >
> > > Although the content of PR does not improve REEF, a reviewer should
> say a
> > > warm comment like 'Sorry, but thanks for making a PR', and give -1. I
> > think
> > > that's natural in Open Source Communities. (The PR code will be
> > withdrawed
> > > or closed.)
> > >
> > > I know that REEF community consists of strong developers and has warm
> > > atmosphere in general. I wrote here because I was surprise about the
> > > concept of `overhead`. I really wanted to listen others' opinions, too.
> > > Thank you, Markus.
> > >
> > > Dongjoon.
> > >
> > >
> > >
> > > On Tue, Jan 26, 2016 at 10:58 PM, Markus Weimer <ma...@weimo.de>
> wrote:
> > >
> > >> Hi,
> > >>
> > >> thanks for bringing this up. It is an interesting, different angle on
> > the
> > >> review process. We've had this discussion from the other side before:
> > >> contributors felt that the review cycle put a impediment onto them.
> > >>
> > >> I believe both arguments have a kernel of validity, but I also believe
> > >> that reviews are super-crucial not only for code quality, but even
> more
> > so
> > >> for a shared understanding of the code. As you said, every committer
> and
> > >> PMC member should use the opportunity to stay current with REEF's
> > >> development via the review process. To be honest, that is often my
> > reason
> > >> for picking specific PRs: I want to know what's happening in that part
> > of
> > >> the code.
> > >>
> > >> That said, I think we should use every tool and idea available to
> remove
> > >> friction from the process. Basic coding standard checks and test runs
> > >> shouldn't be the work of human brains, but CPUs :) The work Mariia and
> > you
> > >> have been doing goes a long way towards that. There are some gaping
> > holes
> > >> (*ahem* .NET tests) which should be addressed soon.
> > >>
> > >> But I am rambling: What do other's think? Is the PR review process too
> > >> burdensome? What (beyond the basic continuous integration setup) can
> we
> > do
> > >> to make it easier?
> > >>
> > >> Thanks again for bringing this up!
> > >>
> > >> Markus
> > >>
> > >
> > >
> >
>
>
>
> --
> Byung-Gon Chun
>
Re: "The overhead of reviewing and testing"
Posted by Byung-Gon Chun <bg...@gmail.com>.
Hi,
We had related discussion in August 2015.
https://mail-archives.apache.org/mod_mbox/incubator-reef-dev/201508.mbox/%3C55DB2CDE.40804@weimo.de%3E
The [MINOR] tag has worked really well for SNU internal projects. I
strongly recommend the reef community adopts it. :-)
Thanks!
-Gon
On Wed, Jan 27, 2016 at 5:11 PM, Dongjoon Hyun <do...@apache.org> wrote:
> Oh, I missed your word, "We've had this discussion from the other side
> before".
> It's not a different story. Sorry!
>
> It's too late night. I had better go to bed before making another mistake.
>
> Dongjoon.
>
>
> On Tue, Jan 26, 2016 at 11:51 PM, Dongjoon Hyun <do...@apache.org>
> wrote:
>
> > Thank you for your generous comments. Actually, what I meant to focus is
> > the vice versa.
> >
> > "The reviewer (in a review cycle) should focus on only the PR code itself
> > like Blind Paper Review Process."
> >
> > - Why does a reviewer consider a contributor's other PRs?
> > - Why does a reviewer ask something beyond the scope of the PR?
> > (As we know, contributors are not REEF's or PMC's full-time employees.)
> >
> > I dream that an ideal environment where any tiny contributions from
> anyone
> > are welcome equally(here, blindly) anytime.
> >
> > Although the content of PR does not improve REEF, a reviewer should say a
> > warm comment like 'Sorry, but thanks for making a PR', and give -1. I
> think
> > that's natural in Open Source Communities. (The PR code will be
> withdrawed
> > or closed.)
> >
> > I know that REEF community consists of strong developers and has warm
> > atmosphere in general. I wrote here because I was surprise about the
> > concept of `overhead`. I really wanted to listen others' opinions, too.
> > Thank you, Markus.
> >
> > Dongjoon.
> >
> >
> >
> > On Tue, Jan 26, 2016 at 10:58 PM, Markus Weimer <ma...@weimo.de> wrote:
> >
> >> Hi,
> >>
> >> thanks for bringing this up. It is an interesting, different angle on
> the
> >> review process. We've had this discussion from the other side before:
> >> contributors felt that the review cycle put a impediment onto them.
> >>
> >> I believe both arguments have a kernel of validity, but I also believe
> >> that reviews are super-crucial not only for code quality, but even more
> so
> >> for a shared understanding of the code. As you said, every committer and
> >> PMC member should use the opportunity to stay current with REEF's
> >> development via the review process. To be honest, that is often my
> reason
> >> for picking specific PRs: I want to know what's happening in that part
> of
> >> the code.
> >>
> >> That said, I think we should use every tool and idea available to remove
> >> friction from the process. Basic coding standard checks and test runs
> >> shouldn't be the work of human brains, but CPUs :) The work Mariia and
> you
> >> have been doing goes a long way towards that. There are some gaping
> holes
> >> (*ahem* .NET tests) which should be addressed soon.
> >>
> >> But I am rambling: What do other's think? Is the PR review process too
> >> burdensome? What (beyond the basic continuous integration setup) can we
> do
> >> to make it easier?
> >>
> >> Thanks again for bringing this up!
> >>
> >> Markus
> >>
> >
> >
>
--
Byung-Gon Chun
Re: "The overhead of reviewing and testing"
Posted by Dongjoon Hyun <do...@apache.org>.
Oh, I missed your word, "We've had this discussion from the other side
before".
It's not a different story. Sorry!
It's too late night. I had better go to bed before making another mistake.
Dongjoon.
On Tue, Jan 26, 2016 at 11:51 PM, Dongjoon Hyun <do...@apache.org> wrote:
> Thank you for your generous comments. Actually, what I meant to focus is
> the vice versa.
>
> "The reviewer (in a review cycle) should focus on only the PR code itself
> like Blind Paper Review Process."
>
> - Why does a reviewer consider a contributor's other PRs?
> - Why does a reviewer ask something beyond the scope of the PR?
> (As we know, contributors are not REEF's or PMC's full-time employees.)
>
> I dream that an ideal environment where any tiny contributions from anyone
> are welcome equally(here, blindly) anytime.
>
> Although the content of PR does not improve REEF, a reviewer should say a
> warm comment like 'Sorry, but thanks for making a PR', and give -1. I think
> that's natural in Open Source Communities. (The PR code will be withdrawed
> or closed.)
>
> I know that REEF community consists of strong developers and has warm
> atmosphere in general. I wrote here because I was surprise about the
> concept of `overhead`. I really wanted to listen others' opinions, too.
> Thank you, Markus.
>
> Dongjoon.
>
>
>
> On Tue, Jan 26, 2016 at 10:58 PM, Markus Weimer <ma...@weimo.de> wrote:
>
>> Hi,
>>
>> thanks for bringing this up. It is an interesting, different angle on the
>> review process. We've had this discussion from the other side before:
>> contributors felt that the review cycle put a impediment onto them.
>>
>> I believe both arguments have a kernel of validity, but I also believe
>> that reviews are super-crucial not only for code quality, but even more so
>> for a shared understanding of the code. As you said, every committer and
>> PMC member should use the opportunity to stay current with REEF's
>> development via the review process. To be honest, that is often my reason
>> for picking specific PRs: I want to know what's happening in that part of
>> the code.
>>
>> That said, I think we should use every tool and idea available to remove
>> friction from the process. Basic coding standard checks and test runs
>> shouldn't be the work of human brains, but CPUs :) The work Mariia and you
>> have been doing goes a long way towards that. There are some gaping holes
>> (*ahem* .NET tests) which should be addressed soon.
>>
>> But I am rambling: What do other's think? Is the PR review process too
>> burdensome? What (beyond the basic continuous integration setup) can we do
>> to make it easier?
>>
>> Thanks again for bringing this up!
>>
>> Markus
>>
>
>
Re: "The overhead of reviewing and testing"
Posted by Dongjoon Hyun <do...@apache.org>.
Thank you for your generous comments. Actually, what I meant to focus is
the vice versa.
"The reviewer (in a review cycle) should focus on only the PR code itself
like Blind Paper Review Process."
- Why does a reviewer consider a contributor's other PRs?
- Why does a reviewer ask something beyond the scope of the PR?
(As we know, contributors are not REEF's or PMC's full-time employees.)
I dream that an ideal environment where any tiny contributions from anyone
are welcome equally(here, blindly) anytime.
Although the content of PR does not improve REEF, a reviewer should say a
warm comment like 'Sorry, but thanks for making a PR', and give -1. I think
that's natural in Open Source Communities. (The PR code will be withdrawed
or closed.)
I know that REEF community consists of strong developers and has warm
atmosphere in general. I wrote here because I was surprise about the
concept of `overhead`. I really wanted to listen others' opinions, too.
Thank you, Markus.
Dongjoon.
On Tue, Jan 26, 2016 at 10:58 PM, Markus Weimer <ma...@weimo.de> wrote:
> Hi,
>
> thanks for bringing this up. It is an interesting, different angle on the
> review process. We've had this discussion from the other side before:
> contributors felt that the review cycle put a impediment onto them.
>
> I believe both arguments have a kernel of validity, but I also believe
> that reviews are super-crucial not only for code quality, but even more so
> for a shared understanding of the code. As you said, every committer and
> PMC member should use the opportunity to stay current with REEF's
> development via the review process. To be honest, that is often my reason
> for picking specific PRs: I want to know what's happening in that part of
> the code.
>
> That said, I think we should use every tool and idea available to remove
> friction from the process. Basic coding standard checks and test runs
> shouldn't be the work of human brains, but CPUs :) The work Mariia and you
> have been doing goes a long way towards that. There are some gaping holes
> (*ahem* .NET tests) which should be addressed soon.
>
> But I am rambling: What do other's think? Is the PR review process too
> burdensome? What (beyond the basic continuous integration setup) can we do
> to make it easier?
>
> Thanks again for bringing this up!
>
> Markus
>
Re: "The overhead of reviewing and testing"
Posted by Markus Weimer <ma...@weimo.de>.
Hi,
thanks for bringing this up. It is an interesting, different angle on
the review process. We've had this discussion from the other side
before: contributors felt that the review cycle put a impediment onto them.
I believe both arguments have a kernel of validity, but I also believe
that reviews are super-crucial not only for code quality, but even more
so for a shared understanding of the code. As you said, every committer
and PMC member should use the opportunity to stay current with REEF's
development via the review process. To be honest, that is often my
reason for picking specific PRs: I want to know what's happening in that
part of the code.
That said, I think we should use every tool and idea available to remove
friction from the process. Basic coding standard checks and test runs
shouldn't be the work of human brains, but CPUs :) The work Mariia and
you have been doing goes a long way towards that. There are some gaping
holes (*ahem* .NET tests) which should be addressed soon.
But I am rambling: What do other's think? Is the PR review process too
burdensome? What (beyond the basic continuous integration setup) can we
do to make it easier?
Thanks again for bringing this up!
Markus