You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafodion.apache.org by "Varnau, Steve (Trafodion)" <st...@hp.com> on 2015/06/03 18:34:58 UTC
Proposal: Trafodion Code Development Process in Apache
Trafodion Dev,
I'm working on the process for how we'll manage code changes once our code migrates to Apache git server. Our current process won't work, so changes will be needed. I have a proposal for how it will work, but am looking for feedback.
We currently submit code changes to Gerrit site (review.trafodion.org) for code review and automated testing. It enforces certain workflow criteria of reviews and tests and merges the code automatically. While Gerrit is highly configurable, it does require that it run against the canonical repo and then mirrors it elsewhere. That basic assumption is incompatible with the canonical repo being on the Apache server and committers having the only write access.
Also, one of the most confusing things about our current usage of git is the way Gerrit requires changes to be packaged in single commit and with separate change-IDs in the comments. Changing our process will have the side benefit of avoiding those issues.
Other Apache projects tend to submit code contributions in one three ways: Jira (attaching a patch file), ReviewBoard, GitHub pull-request
Some projects use two of the three methods. In general, each project has specific instructions on which method you should use.
While github is not running on the Apache infrastructure, Apache infra team provides integrations, so that github activity is mirrored over to Apache Jira and/or mail lists. That satisfies the Apache requirements to archive info related to community work and code provenance.
None of the three available mechanisms provides workflow requirements like Gerrit. Rather, they rely on committers' judgment to ensure project criteria are satisfied and merge the code in. What we need, however, is the basic code review and test automation capabilities.
In my estimation, GitHub provides the best support for code review and test automation. I'm currently working on changing our Jenkins testing automation to plug into GitHub, rather than Gerrit/Zuul. Of course, it may take a little while to get that working smoothly, but it is certainly do-able.
>From the development point of view, everyone would need a github account, rather than gerrit account. The workflow would be to do work on a branch, push the branch to their fork on github, then make a pull-request on github. I'll provide some detailed instructions on the wiki. In order to facilitate working with github, I recommend folks use the git wrapper tool call "hub": https://hub.github.com/
Details to come, but I wanted to start the discussion whether this is the right direction.
-Steve
Re: Proposal: Trafodion Code Development Process in Apache
Posted by Stack <st...@duboce.net>.
On Wed, Jun 3, 2015 at 11:25 AM, Sundaresan, Sandhya <
sandhya.sundaresan@hp.com> wrote:
> Hi,
> Automating the tests is very important. We have in the past managed with
> the review process not being automated but it has been good to have that
> automated as well recently.
>
Yeah, lets make this work. Hadoop has been doing some nice work of late
generalizing their CI scripts. T8 might be able to make use of these. Some
projects trigger CI by attaching patch to JIRA and clicking 'submit' AND
they have CI run tests post commit.
> Having only the committers be able to write to repository does make some
> kind of a bottle neck. I would prefer a weekly or biweekly rotation of
> committers rather than have each traf developer pick their committer. I am
> afraid some committers will get picked more than others and be over
> burndened. If we take turns, on a regular basis it may work better.
>
You could anoint more committers!
St.Ack
> I agree with Suresh that we need to get more input. A tech talk and/or
> email describing proposed workflow maybe ?
> Thanks
> Sandhya
>
> -----Original Message-----
> From: Subbiah, Suresh
> Sent: Wednesday, June 03, 2015 10:39 AM
> To: dev@trafodion.incubator.apache.org
> Subject: RE: Proposal: Trafodion Code Development Process in Apache
>
> Hi Steve,
>
> Thank you for this nice proposal. I too would prefer to use github pull
> requests rather than ReviewBoard or patch files in Jira.
>
> In the new approach, to replace workflow requirements provided by Gerrit
> this is one possibility
> 1) Each contributor picks out one or more committers to deliver/merge
> their change
> 2) The committer runs required regressions after merge (preferably using
> some automated setup)
> 3) Ensures code is reviewed by a minimum number of folks
> 4) delivers the merged change.
>
> In effect the committer does what gerrit does for us today. If there is a
> regression failure or review issue, contributor provides a new patchset to
> commiter.
>
> Is this procedure too manual?
> Should we have this discussion on one of the current traf dlists where
> more people can weigh in? This is always a topic with multiple points of
> view. We could copy this dlist to ensure that apache has the discussion
> archived?
>
> Thanks
> Suresh
>
>
> -----Original Message-----
> From: Varnau, Steve (Trafodion)
> Sent: Wednesday, June 03, 2015 11:35 AM
> To: dev@trafodion.incubator.apache.org
> Subject: Proposal: Trafodion Code Development Process in Apache
>
> Trafodion Dev,
>
> I'm working on the process for how we'll manage code changes once our code
> migrates to Apache git server. Our current process won't work, so changes
> will be needed. I have a proposal for how it will work, but am looking for
> feedback.
>
> We currently submit code changes to Gerrit site (review.trafodion.org)
> for code review and automated testing. It enforces certain workflow
> criteria of reviews and tests and merges the code automatically. While
> Gerrit is highly configurable, it does require that it run against the
> canonical repo and then mirrors it elsewhere. That basic assumption is
> incompatible with the canonical repo being on the Apache server and
> committers having the only write access.
>
> Also, one of the most confusing things about our current usage of git is
> the way Gerrit requires changes to be packaged in single commit and with
> separate change-IDs in the comments. Changing our process will have the
> side benefit of avoiding those issues.
>
> Other Apache projects tend to submit code contributions in one three ways:
> Jira (attaching a patch file), ReviewBoard, GitHub pull-request Some
> projects use two of the three methods. In general, each project has
> specific instructions on which method you should use.
>
> While github is not running on the Apache infrastructure, Apache infra
> team provides integrations, so that github activity is mirrored over to
> Apache Jira and/or mail lists. That satisfies the Apache requirements to
> archive info related to community work and code provenance.
>
> None of the three available mechanisms provides workflow requirements like
> Gerrit. Rather, they rely on committers' judgment to ensure project
> criteria are satisfied and merge the code in. What we need, however, is the
> basic code review and test automation capabilities.
>
> In my estimation, GitHub provides the best support for code review and
> test automation. I'm currently working on changing our Jenkins testing
> automation to plug into GitHub, rather than Gerrit/Zuul. Of course, it may
> take a little while to get that working smoothly, but it is certainly
> do-able.
>
> From the development point of view, everyone would need a github account,
> rather than gerrit account. The workflow would be to do work on a branch,
> push the branch to their fork on github, then make a pull-request on
> github. I'll provide some detailed instructions on the wiki. In order to
> facilitate working with github, I recommend folks use the git wrapper tool
> call "hub": https://hub.github.com/
>
> Details to come, but I wanted to start the discussion whether this is the
> right direction.
>
> -Steve
>
>
>
>
Re: Proposal: Trafodion Code Development Process in Apache
Posted by Stack <st...@duboce.net>.
On Wed, Jun 3, 2015 at 1:03 PM, Birdsall, Dave <da...@hp.com> wrote:
...
> I had the impression that the HBase folks have the practice of subject
> matter experts in various parts of HBase. I saw a web page to this effect
> but am having trouble finding it now.
>
We tried to formalize a system where you had to get a +1 from someone who
had volunteered as 'expert' in the subcomponent before commit [1.] but the
formalization didn't work out so well; people come and go or get
busy/distracted, etc. In practise, this is what tends to happen though;
folks review in the areas they know. If a non-expert repeatedly +1s code
they don't understand, over time, their +1 loses value...
St.Ack
1. http://hbase.apache.org/book.html#community.roles
RE: Proposal: Trafodion Code Development Process in Apache
Posted by "Birdsall, Dave" <da...@hp.com>.
Hi,
I think a rotation model could work. But it might assume fairly broad expertise across the product. I usually read all code reviews but I know much more about the Executor than I do the Windows ODBC driver, so I can offer much better advice on the former.
If committers are active as shepherds while code changes are developed (that is, involved before the code is ready to submit) that would help. Those folks would be subject matter experts in the area(s) being changed. Then multiple committers would be in the loop.
I had the impression that the HBase folks have the practice of subject matter experts in various parts of HBase. I saw a web page to this effect but am having trouble finding it now.
Dave
-----Original Message-----
From: Varnau, Steve (Trafodion)
Sent: Wednesday, June 03, 2015 11:42 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Proposal: Trafodion Code Development Process in Apache
I have every intention of having automated tests of the pull-requests from the start, but it is not in place yet. And I imagine there may be some rough edges that will improve after we get going.
We don't have core reviewers assigned today, so I'm not sure why we'd need a committer assigned. But the practices can certainly evolve if needed (pull-requests are not reviewed in timely manner).
-Steve
> -----Original Message-----
> From: Sundaresan, Sandhya
> Sent: Wednesday, June 03, 2015 11:26
> To: dev@trafodion.incubator.apache.org
> Subject: RE: Proposal: Trafodion Code Development Process in Apache
>
> Hi,
> Automating the tests is very important. We have in the past managed
> with the review process not being automated but it has been good to
> have that automated as well recently.
> Having only the committers be able to write to repository does make
> some kind of a bottle neck. I would prefer a weekly or biweekly
> rotation of committers rather than have each traf developer pick their
> committer. I am afraid some committers will get picked more than
> others and be over burndened. If we take turns, on a regular basis it may work better.
> I agree with Suresh that we need to get more input. A tech talk and/or
> email describing proposed workflow maybe ?
> Thanks
> Sandhya
>
> -----Original Message-----
> From: Subbiah, Suresh
> Sent: Wednesday, June 03, 2015 10:39 AM
> To: dev@trafodion.incubator.apache.org
> Subject: RE: Proposal: Trafodion Code Development Process in Apache
>
> Hi Steve,
>
> Thank you for this nice proposal. I too would prefer to use github
> pull requests rather than ReviewBoard or patch files in Jira.
>
> In the new approach, to replace workflow requirements provided by
> Gerrit this is one possibility
> 1) Each contributor picks out one or more committers to deliver/merge
> their change
> 2) The committer runs required regressions after merge (preferably
> using some automated setup)
> 3) Ensures code is reviewed by a minimum number of folks
> 4) delivers the merged change.
>
> In effect the committer does what gerrit does for us today. If there
> is a regression failure or review issue, contributor provides a new
> patchset to commiter.
>
> Is this procedure too manual?
> Should we have this discussion on one of the current traf dlists where
> more people can weigh in? This is always a topic with multiple points
> of view. We could copy this dlist to ensure that apache has the
> discussion archived?
>
> Thanks
> Suresh
>
>
> -----Original Message-----
> From: Varnau, Steve (Trafodion)
> Sent: Wednesday, June 03, 2015 11:35 AM
> To: dev@trafodion.incubator.apache.org
> Subject: Proposal: Trafodion Code Development Process in Apache
>
> Trafodion Dev,
>
> I'm working on the process for how we'll manage code changes once our
> code migrates to Apache git server. Our current process won't work, so
> changes will be needed. I have a proposal for how it will work, but am
> looking for feedback.
>
> We currently submit code changes to Gerrit site (review.trafodion.org)
> for code review and automated testing. It enforces certain workflow
> criteria of reviews and tests and merges the code automatically.
> While Gerrit is highly configurable, it does require that it run
> against the canonical repo and then mirrors it elsewhere. That basic
> assumption is incompatible with the canonical repo being on the Apache
> server and committers having the only write access.
>
> Also, one of the most confusing things about our current usage of git
> is the way Gerrit requires changes to be packaged in single commit and
> with separate change-IDs in the comments. Changing our process will
> have the side benefit of avoiding those issues.
>
> Other Apache projects tend to submit code contributions in one three
> ways: Jira (attaching a patch file), ReviewBoard, GitHub pull-request
> Some projects use two of the three methods. In general, each project
> has specific instructions on which method you should use.
>
> While github is not running on the Apache infrastructure, Apache infra
> team provides integrations, so that github activity is mirrored over
> to Apache Jira and/or mail lists. That satisfies the Apache
> requirements to archive info related to community work and code provenance.
>
> None of the three available mechanisms provides workflow requirements
> like Gerrit. Rather, they rely on committers' judgment to ensure
> project criteria are satisfied and merge the code in. What we need,
> however, is the basic code review and test automation capabilities.
>
> In my estimation, GitHub provides the best support for code review and
> test automation. I'm currently working on changing our Jenkins testing
> automation to plug into GitHub, rather than Gerrit/Zuul. Of course, it
> may take a little while to get that working smoothly, but it is
> certainly do-able.
>
> From the development point of view, everyone would need a github
> account, rather than gerrit account. The workflow would be to do work
> on a branch, push the branch to their fork on github, then make a
> pull- request on github. I'll provide some detailed instructions on the wiki.
> In order to facilitate working with github, I recommend folks use the
> git wrapper tool call "hub": https://hub.github.com/
>
> Details to come, but I wanted to start the discussion whether this is
> the right direction.
>
> -Steve
>
>
RE: Proposal: Trafodion Code Development Process in Apache
Posted by "Varnau, Steve (Trafodion)" <st...@hp.com>.
I have every intention of having automated tests of the pull-requests from the start, but it is not in place yet. And I imagine there may be some rough edges that will improve after we get going.
We don't have core reviewers assigned today, so I'm not sure why we'd need a committer assigned. But the practices can certainly evolve if needed (pull-requests are not reviewed in timely manner).
-Steve
> -----Original Message-----
> From: Sundaresan, Sandhya
> Sent: Wednesday, June 03, 2015 11:26
> To: dev@trafodion.incubator.apache.org
> Subject: RE: Proposal: Trafodion Code Development Process in Apache
>
> Hi,
> Automating the tests is very important. We have in the past managed
> with the review process not being automated but it has been good to have
> that automated as well recently.
> Having only the committers be able to write to repository does make some
> kind of a bottle neck. I would prefer a weekly or biweekly rotation of
> committers rather than have each traf developer pick their committer. I
> am afraid some committers will get picked more than others and be over
> burndened. If we take turns, on a regular basis it may work better.
> I agree with Suresh that we need to get more input. A tech talk and/or
> email describing proposed workflow maybe ?
> Thanks
> Sandhya
>
> -----Original Message-----
> From: Subbiah, Suresh
> Sent: Wednesday, June 03, 2015 10:39 AM
> To: dev@trafodion.incubator.apache.org
> Subject: RE: Proposal: Trafodion Code Development Process in Apache
>
> Hi Steve,
>
> Thank you for this nice proposal. I too would prefer to use github pull
> requests rather than ReviewBoard or patch files in Jira.
>
> In the new approach, to replace workflow requirements provided by Gerrit
> this is one possibility
> 1) Each contributor picks out one or more committers to deliver/merge
> their change
> 2) The committer runs required regressions after merge (preferably using
> some automated setup)
> 3) Ensures code is reviewed by a minimum number of folks
> 4) delivers the merged change.
>
> In effect the committer does what gerrit does for us today. If there is
> a regression failure or review issue, contributor provides a new
> patchset to commiter.
>
> Is this procedure too manual?
> Should we have this discussion on one of the current traf dlists where
> more people can weigh in? This is always a topic with multiple points of
> view. We could copy this dlist to ensure that apache has the discussion
> archived?
>
> Thanks
> Suresh
>
>
> -----Original Message-----
> From: Varnau, Steve (Trafodion)
> Sent: Wednesday, June 03, 2015 11:35 AM
> To: dev@trafodion.incubator.apache.org
> Subject: Proposal: Trafodion Code Development Process in Apache
>
> Trafodion Dev,
>
> I'm working on the process for how we'll manage code changes once our
> code migrates to Apache git server. Our current process won't work, so
> changes will be needed. I have a proposal for how it will work, but am
> looking for feedback.
>
> We currently submit code changes to Gerrit site (review.trafodion.org)
> for code review and automated testing. It enforces certain workflow
> criteria of reviews and tests and merges the code automatically. While
> Gerrit is highly configurable, it does require that it run against the
> canonical repo and then mirrors it elsewhere. That basic assumption is
> incompatible with the canonical repo being on the Apache server and
> committers having the only write access.
>
> Also, one of the most confusing things about our current usage of git is
> the way Gerrit requires changes to be packaged in single commit and with
> separate change-IDs in the comments. Changing our process will have the
> side benefit of avoiding those issues.
>
> Other Apache projects tend to submit code contributions in one three
> ways: Jira (attaching a patch file), ReviewBoard, GitHub pull-request
> Some projects use two of the three methods. In general, each project has
> specific instructions on which method you should use.
>
> While github is not running on the Apache infrastructure, Apache infra
> team provides integrations, so that github activity is mirrored over to
> Apache Jira and/or mail lists. That satisfies the Apache requirements to
> archive info related to community work and code provenance.
>
> None of the three available mechanisms provides workflow requirements
> like Gerrit. Rather, they rely on committers' judgment to ensure project
> criteria are satisfied and merge the code in. What we need, however, is
> the basic code review and test automation capabilities.
>
> In my estimation, GitHub provides the best support for code review and
> test automation. I'm currently working on changing our Jenkins testing
> automation to plug into GitHub, rather than Gerrit/Zuul. Of course, it
> may take a little while to get that working smoothly, but it is
> certainly do-able.
>
> From the development point of view, everyone would need a github
> account, rather than gerrit account. The workflow would be to do work on
> a branch, push the branch to their fork on github, then make a pull-
> request on github. I'll provide some detailed instructions on the wiki.
> In order to facilitate working with github, I recommend folks use the
> git wrapper tool call "hub": https://hub.github.com/
>
> Details to come, but I wanted to start the discussion whether this is
> the right direction.
>
> -Steve
>
>
RE: Proposal: Trafodion Code Development Process in Apache
Posted by "Sundaresan, Sandhya" <sa...@hp.com>.
Hi,
Automating the tests is very important. We have in the past managed with the review process not being automated but it has been good to have that automated as well recently.
Having only the committers be able to write to repository does make some kind of a bottle neck. I would prefer a weekly or biweekly rotation of committers rather than have each traf developer pick their committer. I am afraid some committers will get picked more than others and be over burndened. If we take turns, on a regular basis it may work better.
I agree with Suresh that we need to get more input. A tech talk and/or email describing proposed workflow maybe ?
Thanks
Sandhya
-----Original Message-----
From: Subbiah, Suresh
Sent: Wednesday, June 03, 2015 10:39 AM
To: dev@trafodion.incubator.apache.org
Subject: RE: Proposal: Trafodion Code Development Process in Apache
Hi Steve,
Thank you for this nice proposal. I too would prefer to use github pull requests rather than ReviewBoard or patch files in Jira.
In the new approach, to replace workflow requirements provided by Gerrit this is one possibility
1) Each contributor picks out one or more committers to deliver/merge their change
2) The committer runs required regressions after merge (preferably using some automated setup)
3) Ensures code is reviewed by a minimum number of folks
4) delivers the merged change.
In effect the committer does what gerrit does for us today. If there is a regression failure or review issue, contributor provides a new patchset to commiter.
Is this procedure too manual?
Should we have this discussion on one of the current traf dlists where more people can weigh in? This is always a topic with multiple points of view. We could copy this dlist to ensure that apache has the discussion archived?
Thanks
Suresh
-----Original Message-----
From: Varnau, Steve (Trafodion)
Sent: Wednesday, June 03, 2015 11:35 AM
To: dev@trafodion.incubator.apache.org
Subject: Proposal: Trafodion Code Development Process in Apache
Trafodion Dev,
I'm working on the process for how we'll manage code changes once our code migrates to Apache git server. Our current process won't work, so changes will be needed. I have a proposal for how it will work, but am looking for feedback.
We currently submit code changes to Gerrit site (review.trafodion.org) for code review and automated testing. It enforces certain workflow criteria of reviews and tests and merges the code automatically. While Gerrit is highly configurable, it does require that it run against the canonical repo and then mirrors it elsewhere. That basic assumption is incompatible with the canonical repo being on the Apache server and committers having the only write access.
Also, one of the most confusing things about our current usage of git is the way Gerrit requires changes to be packaged in single commit and with separate change-IDs in the comments. Changing our process will have the side benefit of avoiding those issues.
Other Apache projects tend to submit code contributions in one three ways: Jira (attaching a patch file), ReviewBoard, GitHub pull-request Some projects use two of the three methods. In general, each project has specific instructions on which method you should use.
While github is not running on the Apache infrastructure, Apache infra team provides integrations, so that github activity is mirrored over to Apache Jira and/or mail lists. That satisfies the Apache requirements to archive info related to community work and code provenance.
None of the three available mechanisms provides workflow requirements like Gerrit. Rather, they rely on committers' judgment to ensure project criteria are satisfied and merge the code in. What we need, however, is the basic code review and test automation capabilities.
In my estimation, GitHub provides the best support for code review and test automation. I'm currently working on changing our Jenkins testing automation to plug into GitHub, rather than Gerrit/Zuul. Of course, it may take a little while to get that working smoothly, but it is certainly do-able.
>From the development point of view, everyone would need a github account, rather than gerrit account. The workflow would be to do work on a branch, push the branch to their fork on github, then make a pull-request on github. I'll provide some detailed instructions on the wiki. In order to facilitate working with github, I recommend folks use the git wrapper tool call "hub": https://hub.github.com/
Details to come, but I wanted to start the discussion whether this is the right direction.
-Steve
RE: Proposal: Trafodion Code Development Process in Apache
Posted by "Subbiah, Suresh" <Su...@hp.com>.
Hi Steve,
Thank you for this nice proposal. I too would prefer to use github pull requests rather than ReviewBoard or patch files in Jira.
In the new approach, to replace workflow requirements provided by Gerrit this is one possibility
1) Each contributor picks out one or more committers to deliver/merge their change
2) The committer runs required regressions after merge (preferably using some automated setup)
3) Ensures code is reviewed by a minimum number of folks
4) delivers the merged change.
In effect the committer does what gerrit does for us today. If there is a regression failure or review issue, contributor provides a new patchset to commiter.
Is this procedure too manual?
Should we have this discussion on one of the current traf dlists where more people can weigh in? This is always a topic with multiple points of view. We could copy this dlist to ensure that apache has the discussion archived?
Thanks
Suresh
-----Original Message-----
From: Varnau, Steve (Trafodion)
Sent: Wednesday, June 03, 2015 11:35 AM
To: dev@trafodion.incubator.apache.org
Subject: Proposal: Trafodion Code Development Process in Apache
Trafodion Dev,
I'm working on the process for how we'll manage code changes once our code migrates to Apache git server. Our current process won't work, so changes will be needed. I have a proposal for how it will work, but am looking for feedback.
We currently submit code changes to Gerrit site (review.trafodion.org) for code review and automated testing. It enforces certain workflow criteria of reviews and tests and merges the code automatically. While Gerrit is highly configurable, it does require that it run against the canonical repo and then mirrors it elsewhere. That basic assumption is incompatible with the canonical repo being on the Apache server and committers having the only write access.
Also, one of the most confusing things about our current usage of git is the way Gerrit requires changes to be packaged in single commit and with separate change-IDs in the comments. Changing our process will have the side benefit of avoiding those issues.
Other Apache projects tend to submit code contributions in one three ways: Jira (attaching a patch file), ReviewBoard, GitHub pull-request Some projects use two of the three methods. In general, each project has specific instructions on which method you should use.
While github is not running on the Apache infrastructure, Apache infra team provides integrations, so that github activity is mirrored over to Apache Jira and/or mail lists. That satisfies the Apache requirements to archive info related to community work and code provenance.
None of the three available mechanisms provides workflow requirements like Gerrit. Rather, they rely on committers' judgment to ensure project criteria are satisfied and merge the code in. What we need, however, is the basic code review and test automation capabilities.
In my estimation, GitHub provides the best support for code review and test automation. I'm currently working on changing our Jenkins testing automation to plug into GitHub, rather than Gerrit/Zuul. Of course, it may take a little while to get that working smoothly, but it is certainly do-able.
>From the development point of view, everyone would need a github account, rather than gerrit account. The workflow would be to do work on a branch, push the branch to their fork on github, then make a pull-request on github. I'll provide some detailed instructions on the wiki. In order to facilitate working with github, I recommend folks use the git wrapper tool call "hub": https://hub.github.com/
Details to come, but I wanted to start the discussion whether this is the right direction.
-Steve
Re: Proposal: Trafodion Code Development Process in Apache
Posted by Stack <st...@duboce.net>.
On Sat, Jun 6, 2015 at 9:07 PM, Stack <st...@duboce.net> wrote:
...
> The pull request would come into the apache dev list and then a committer
> would apply it with back and forth between contributor and committer
> happening in github -- rather than in Apache INFRA JIRA or review board?
>
> Would this process have a "dark side" if looking in from an Apache view
> only? Patches would show up in dev as pull requests and be committed
> without comment because all discussion had happened already elsewhere.
>
>
This thread that I was witness too might have some useful bearing on this
discussion [1.] (The thread is "[DISCUSS] Github integration"). In
particular, an objection is made that it is " ...frustrating to search through
github pull requests. There is no interface (like there is on JIRA) to
search using any kind of structured query language, and we don't have the
tools to track things by release, contributor, etc." Would be interested in
T8 folks take.
St.Ack
1.
http://mail-archives.apache.org/mod_mbox/incubator-htrace-dev/201504.mbox/%3CCANZa%3DGsoauJ4LJUWfHb5-XedBUdiU0GA_faH_HKcy3FCaGTu6Q%40mail.gmail.com%3E
RE: Proposal: Trafodion Code Development Process in Apache
Posted by "Varnau, Steve (Trafodion)" <st...@hp.com>.
The github integration offers option to also mirror all of the project activity to an apache mail list, which would make sure *all* activity of the project makes it to Apache archive. The complaints I've seen are that such messages make the dev lists way too noisy. I suppose there could be a separate mail list just for those messages and those that do want to follow it in email rather than in github. The other way is that committers just make sure all pull-requests have JIRA ID added to the title, which allows it all be sucked into JIRA.
I also like the idea to move to docker. We'll definitely have to explore that. We currently have quite a few project VM's managed with puppet, including build and test machines.
-Steve
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> Stack
> Sent: Sunday, June 07, 2015 21:14
> To: dev@trafodion.incubator.apache.org
> Subject: Re: Proposal: Trafodion Code Development Process in Apache
>
> On Sat, Jun 6, 2015 at 10:23 PM, Varnau, Steve (Trafodion) <
> steve.varnau@hp.com> wrote:
>
> > Comment below...
>
> ....
>
> > >
> > > This will split the comment history across two systems.
> >
> > Apache Infra provides integration with github, such that all the
> > github pull-request activity is mirrored back to JIRA. Jira is the
> > system of record. Github provides a better code-review interface than
> > Jira, so it is convenient to make code review comments there.
> > Apache/Spark and other projects use similar process.
> >
> >
> The thread I cite was explicitly about difficulty being able to search
> pulls in Spark.
>
> If JIRA "is the system of record" and pull-request back and forth shows
> in JIRA, then that sounds good; my concern is addressed.
>
>
>
> > The Apache CI system does not have any RedHat/Centos systems as far as
> > we can tell, so we'd need to continue running them elsewhere. Again, I
> > believe Spark and other projects do as well.
> >
> >
> I like the Roman suggestion in a subsequent message, that T8 do docker;
> it would seem to make sense particularly in your case given the list of
> dependencies required.
>
>
> > One key is that each pull-request in github should have a JIRA-ID in
> > the title. Committers should not discuss/merge something that doesn't.
> >
> >
> Sounds good Steve,
> St.Ack
>
>
>
>
> > -Steve
> >
> > >
> > >
> > >
> > > > From the development point of view, everyone would need a github
> > > > account, rather than gerrit account. The workflow would be to do
> > > > work on a branch, push the branch to their fork on github, then
> > > > make a pull-request on github. I'll provide some detailed
> > > > instructions on the wiki. In order to facilitate working with
> > > > github, I recommend folks use the git wrapper tool call "hub":
> > > > https://hub.github.com/
> > > >
> > > > Details to come, but I wanted to start the discussion whether this
> > > > is the right direction.
> > > >
> > > >
> > > The pull request would come into the apache dev list and then a
> > > committer would apply it with back and forth between contributor and
> > > committer happening in github -- rather than in Apache INFRA JIRA or
> > > review board?
> > >
> > > Would this process have a "dark side" if looking in from an Apache
> > > view only? Patches would show up in dev as pull requests and be
> > > committed without comment because all discussion had happened
> already elsewhere.
> > >
> > > St.Ack
> > >
> > >
> > >
> > > > -Steve
> > > >
> > > >
> > > >
> > > >
> >
Re: Proposal: Trafodion Code Development Process in Apache
Posted by Stack <st...@duboce.net>.
On Sat, Jun 6, 2015 at 10:23 PM, Varnau, Steve (Trafodion) <
steve.varnau@hp.com> wrote:
> Comment below...
....
> >
> > This will split the comment history across two systems.
>
> Apache Infra provides integration with github, such that all the github
> pull-request activity is mirrored back to JIRA. Jira is the system of
> record. Github provides a better code-review interface than Jira, so it is
> convenient to make code review comments there. Apache/Spark and other
> projects use similar process.
>
>
The thread I cite was explicitly about difficulty being able to search
pulls in Spark.
If JIRA "is the system of record" and pull-request back and forth shows in
JIRA, then that sounds good; my concern is addressed.
> The Apache CI system does not have any RedHat/Centos systems as far as we
> can tell, so we'd need to continue running them elsewhere. Again, I believe
> Spark and other projects do as well.
>
>
I like the Roman suggestion in a subsequent message, that T8 do docker; it
would seem to make sense particularly in your case given the list of
dependencies required.
> One key is that each pull-request in github should have a JIRA-ID in the
> title. Committers should not discuss/merge something that doesn't.
>
>
Sounds good Steve,
St.Ack
> -Steve
>
> >
> >
> >
> > > From the development point of view, everyone would need a github
> > > account, rather than gerrit account. The workflow would be to do work
> > > on a branch, push the branch to their fork on github, then make a
> > > pull-request on github. I'll provide some detailed instructions on the
> > > wiki. In order to facilitate working with github, I recommend folks
> > > use the git wrapper tool call "hub": https://hub.github.com/
> > >
> > > Details to come, but I wanted to start the discussion whether this is
> > > the right direction.
> > >
> > >
> > The pull request would come into the apache dev list and then a
> > committer would apply it with back and forth between contributor and
> > committer happening in github -- rather than in Apache INFRA JIRA or
> > review board?
> >
> > Would this process have a "dark side" if looking in from an Apache view
> > only? Patches would show up in dev as pull requests and be committed
> > without comment because all discussion had happened already elsewhere.
> >
> > St.Ack
> >
> >
> >
> > > -Steve
> > >
> > >
> > >
> > >
>
Re: Proposal: Trafodion Code Development Process in Apache
Posted by Stack <st...@duboce.net>.
On Sun, Jun 7, 2015 at 4:47 PM, Roman Shaposhnik <ro...@shaposhnik.org>
wrote:
...
> To Stack's point -- Spark offers a good example of how tooling can help
> solve
> 99% of these integrations issues. Here's the dashboard (available in Spark
> GH)
> that Spark folks use to pull it all together:
> https://spark-prs.appspot.com/
> it strikes me as a rather nice tool.
>
It looks nice, yeah, but strikes me as odd going off-apache to figure state
of the project.
St.Ack
Re: Proposal: Trafodion Code Development Process in Apache
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Sat, Jun 6, 2015 at 10:23 PM, Varnau, Steve (Trafodion)
<st...@hp.com> wrote:
> Comment below...
A few comments of my own:
>> So, folks would comment on code up in github and CI running on Apache
>> servers would pull from github? JIRA would also carry comments on
>> patches and design?
>>
>> This will split the comment history across two systems.
>
> Apache Infra provides integration with github, such that all the github pull-request
> activity is mirrored back to JIRA. Jira is the system of record.
The workflow is pretty nice. Really the only thing you can NOT do is to
use 'merge PR' button on the GH UI.
> Github provides a
> better code-review interface than Jira, so it is convenient to make code
> review comments there. Apache/Spark and other projects use similar process.
Some folks prefer Reviewboard which is available for ASF projects at:
http://reviews.apache.org
Some developers swear by its side-by-side diff UI for reviews.
> The Apache CI system does not have any RedHat/Centos systems as far as we can tell,
I highly recommend using Docker containers. There's absolutely nothing
that a Docker
container can't do for your build AND it doesn't require you to setup
slaves in any
special way whatsoever.
> One key is that each pull-request in github should have a JIRA-ID in the title.
You don't actually need it. But it helps. Basically, try to make
receiving PRs into
your project as simple as possible -- that's your gateway to community growth.
To Stack's point -- Spark offers a good example of how tooling can help solve
99% of these integrations issues. Here's the dashboard (available in Spark GH)
that Spark folks use to pull it all together:
https://spark-prs.appspot.com/
it strikes me as a rather nice tool.
Thanks,
Roman.
RE: Proposal: Trafodion Code Development Process in Apache
Posted by "Varnau, Steve (Trafodion)" <st...@hp.com>.
Comment below...
> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> Stack
> Sent: Saturday, June 06, 2015 21:07
> To: dev@trafodion.incubator.apache.org
> Subject: Re: Proposal: Trafodion Code Development Process in Apache
>
> On Wed, Jun 3, 2015 at 9:34 AM, Varnau, Steve (Trafodion) <
> steve.varnau@hp.com> wrote:
>
> > Trafodion Dev,
> >
> > I'm working on the process for how we'll manage code changes once our
> > code migrates to Apache git server. Our current process won't work, so
> > changes will be needed. I have a proposal for how it will work, but am
> > looking for feedback.
> >
> > We currently submit code changes to Gerrit site (review.trafodion.org)
> > for code review and automated testing. It enforces certain workflow
> > criteria of reviews and tests and merges the code automatically.
> > While Gerrit is highly configurable, it does require that it run
> > against the canonical repo and then mirrors it elsewhere. That basic
> > assumption is incompatible with the canonical repo being on the Apache
> > server and committers having the only write access.
> >
> >
> Unfortunately, gerrit is not an offering up in Apache (I would like it
> if it was). There is talk of it showing up but it is not here yet.
>
>
>
> > Also, one of the most confusing things about our current usage of git
> > is the way Gerrit requires changes to be packaged in single commit and
> > with separate change-IDs in the comments. Changing our process will
> > have the side benefit of avoiding those issues.
> >
> > Other Apache projects tend to submit code contributions in one three
> ways:
> > Jira (attaching a patch file), ReviewBoard, GitHub pull-request Some
> > projects use two of the three methods. In general, each project has
> > specific instructions on which method you should use.
> >
> > While github is not running on the Apache infrastructure, Apache infra
> > team provides integrations, so that github activity is mirrored over
> > to Apache Jira and/or mail lists. That satisfies the Apache
> > requirements to archive info related to community work and code
> provenance.
> >
> > None of the three available mechanisms provides workflow requirements
> > like Gerrit. Rather, they rely on committers' judgment to ensure
> > project criteria are satisfied and merge the code in. What we need,
> > however, is the basic code review and test automation capabilities.
> >
> > In my estimation, GitHub provides the best support for code review and
> > test automation. I'm currently working on changing our Jenkins testing
> > automation to plug into GitHub, rather than Gerrit/Zuul. Of course, it
> > may take a little while to get that working smoothly, but it is
> > certainly do-able.
> >
> >
> So, folks would comment on code up in github and CI running on Apache
> servers would pull from github? JIRA would also carry comments on
> patches and design?
>
> This will split the comment history across two systems.
Apache Infra provides integration with github, such that all the github pull-request activity is mirrored back to JIRA. Jira is the system of record. Github provides a better code-review interface than Jira, so it is convenient to make code review comments there. Apache/Spark and other projects use similar process.
The Apache CI system does not have any RedHat/Centos systems as far as we can tell, so we'd need to continue running them elsewhere. Again, I believe Spark and other projects do as well.
One key is that each pull-request in github should have a JIRA-ID in the title. Committers should not discuss/merge something that doesn't.
-Steve
>
>
>
> > From the development point of view, everyone would need a github
> > account, rather than gerrit account. The workflow would be to do work
> > on a branch, push the branch to their fork on github, then make a
> > pull-request on github. I'll provide some detailed instructions on the
> > wiki. In order to facilitate working with github, I recommend folks
> > use the git wrapper tool call "hub": https://hub.github.com/
> >
> > Details to come, but I wanted to start the discussion whether this is
> > the right direction.
> >
> >
> The pull request would come into the apache dev list and then a
> committer would apply it with back and forth between contributor and
> committer happening in github -- rather than in Apache INFRA JIRA or
> review board?
>
> Would this process have a "dark side" if looking in from an Apache view
> only? Patches would show up in dev as pull requests and be committed
> without comment because all discussion had happened already elsewhere.
>
> St.Ack
>
>
>
> > -Steve
> >
> >
> >
> >
Re: Proposal: Trafodion Code Development Process in Apache
Posted by Stack <st...@duboce.net>.
On Wed, Jun 3, 2015 at 9:34 AM, Varnau, Steve (Trafodion) <
steve.varnau@hp.com> wrote:
> Trafodion Dev,
>
> I'm working on the process for how we'll manage code changes once our code
> migrates to Apache git server. Our current process won't work, so changes
> will be needed. I have a proposal for how it will work, but am looking for
> feedback.
>
> We currently submit code changes to Gerrit site (review.trafodion.org)
> for code review and automated testing. It enforces certain workflow
> criteria of reviews and tests and merges the code automatically. While
> Gerrit is highly configurable, it does require that it run against the
> canonical repo and then mirrors it elsewhere. That basic assumption is
> incompatible with the canonical repo being on the Apache server and
> committers having the only write access.
>
>
Unfortunately, gerrit is not an offering up in Apache (I would like it if
it was). There is talk of it showing up but it is not here yet.
> Also, one of the most confusing things about our current usage of git is
> the way Gerrit requires changes to be packaged in single commit and with
> separate change-IDs in the comments. Changing our process will have the
> side benefit of avoiding those issues.
>
> Other Apache projects tend to submit code contributions in one three ways:
> Jira (attaching a patch file), ReviewBoard, GitHub pull-request
> Some projects use two of the three methods. In general, each project has
> specific instructions on which method you should use.
>
> While github is not running on the Apache infrastructure, Apache infra
> team provides integrations, so that github activity is mirrored over to
> Apache Jira and/or mail lists. That satisfies the Apache requirements to
> archive info related to community work and code provenance.
>
> None of the three available mechanisms provides workflow requirements like
> Gerrit. Rather, they rely on committers' judgment to ensure project
> criteria are satisfied and merge the code in. What we need, however, is the
> basic code review and test automation capabilities.
>
> In my estimation, GitHub provides the best support for code review and
> test automation. I'm currently working on changing our Jenkins testing
> automation to plug into GitHub, rather than Gerrit/Zuul. Of course, it may
> take a little while to get that working smoothly, but it is certainly
> do-able.
>
>
So, folks would comment on code up in github and CI running on Apache
servers would pull from github? JIRA would also carry comments on patches
and design?
This will split the comment history across two systems.
> From the development point of view, everyone would need a github account,
> rather than gerrit account. The workflow would be to do work on a branch,
> push the branch to their fork on github, then make a pull-request on
> github. I'll provide some detailed instructions on the wiki. In order to
> facilitate working with github, I recommend folks use the git wrapper tool
> call "hub": https://hub.github.com/
>
> Details to come, but I wanted to start the discussion whether this is the
> right direction.
>
>
The pull request would come into the apache dev list and then a committer
would apply it with back and forth between contributor and committer
happening in github -- rather than in Apache INFRA JIRA or review board?
Would this process have a "dark side" if looking in from an Apache view
only? Patches would show up in dev as pull requests and be committed
without comment because all discussion had happened already elsewhere.
St.Ack
> -Steve
>
>
>
>