You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Russel Winder <ru...@winder.org.uk> on 2015/10/23 07:18:28 UTC
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
On Thu, 2015-10-22 at 18:02 +0200, Cédric Champeau wrote:
> Agreed. The only downside of this is that it is very painful to merge
> PRs
> on the GitHub Apache mirror, and even more painful to backport
> changes.
OK, so Apache still doesn't really get Git and GitHub oriented
workflows, it is still fundamentally a CVS/Subversion-based
organization. What is the fundamental workflow problem that you feel is
causing pain? Whatever it is, we must fix it, even if we have to
subvert the current official "Apache Way" workflow. Having a workflow
that works and is easy so that unpaid volunteers can actually achieve
progress on the project, but nonetheless ends up with the Apache
repository as mainline, is far, far, far more important than adherence
to some particular quasi-political philosophy.
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Russel Winder <ru...@winder.org.uk>.
On Sat, 2015-10-24 at 11:56 +0200, Thibault Kruse wrote:
>
[…]
> I believe the alternatives right now are between a single centralized
> repo at
> apache vs. a single centralized repo at github, so I see no need to
> always mention
> "centralized single".
The big point is that Apache has a bare repository whilst GitHub (and
BitBucket) surround this with extra technology for handling changesets
that is now seen as integral to good community. One of the reasons
behind the decline of Codehaus was exactly that in shifting from
Subversion to Git it assumed it need only support the repository and
failed to realize that in the shift from CVCS to DVCS you have to
change your view on supporting the community. With CVCS the repository
is all there is. With DVCS the repository is just a fraction of the
technological support the community associated with that project needs.
GitHub provides a lot of that support and it revolves around public
pull request management and review.
> Apache could set up something like gitlab for their repos,
> (https://about.gitlab.com/),
> without too much manpower involved.
I have no experience of GitLab, but in other places people have said
they are using it in preference to a private GitHub instance, because
it is free to use. They report that it isn't quite as nice as a private
GitHub instance, but it is nonetheless very usable.
As I mentioned in my reply to Jim, an alternative is to set up Gerrit
around the Apache repositories and to not use GitHub for pull requests,
just for repository distribution.
> Or Apache could decree that it is acceptable for projects to use
> github to merge PRs
> before syncing them to the "primary" repository at apache, and set up
> syncing support
> with commit hooks.
As an interim measure, till Apache puts some more DVCS support
technology in place, I think this would be a good thing, as I already
said in my reponse to Jim.
> Since the word "primary" is open for interpretation, it might be
> possible to convince
> Apache that what makes a repo primary for a project is that releases
> are made from
> that repo, not that all merges are done there first.
No need for this worry about primary: the Apache repository is the
primary mainline because it defines the state of the project. GitHub is
a secondary mainline since it is where the work happens, but it is not
part of the definition of the state until it is reflected in the Apache
repository. There is no doubt here, just a change in workflow that
allows GitHub pull request management to be used to its full.
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Thibault Kruse <ti...@googlemail.com>.
Here is the most recent thread on that topic from March this year that
I could find:
https://mail-archives.apache.org/mod_mbox/community-dev/201503.mbox/%3CCADmm+Kff9qR_zQstYxbW=sHMfV7=zZrFvuND_Mn7-8Ljp819hQ@mail.gmail.com%3E
Since Apache's effort depends on how many projects state their
interests, it might be useful to at least give a +1 somewhere there.
On Sat, Oct 24, 2015 at 4:12 PM, Maarten Boekhold <bo...@gmx.com> wrote:
> Maybe apache can ask atlassian if they can get an installation of bit bucket
> for free?
Just to clarify, you mean "Bitbucket Server", formewrly known as
Atlassian Stash Not sure how it compares to gitlab, but JIRA
integration is probably easier.
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Maarten Boekhold <bo...@gmx.com>.
Maybe apache can ask atlassian if they can get an installation of bit
bucket for free?
Maarten
On 24 October 2015 13:56:20 Thibault Kruse <ti...@googlemail.com> wrote:
> On Sat, Oct 24, 2015 at 11:28 AM, Jochen Theodorou <bl...@gmx.org> wrote:
>> On 23.10.2015 13:47, Jim Jagielski wrote:
>>> You do know, of course, that this workflow that has been used for
>>> decades at the ASF was started by and built-around and designed-for
>>> unpaid volunteers to do just that, right?
>>
>>
>> Oh,I would be careful with "used for decades". Just because people don't
>> know it better and got used it, doesn't mean it is good. I would for example
>> not want to change from Git (or Subversion) to CVS, just because CVS is in
>> use for decades. Back then, there was no real alternative. Today times are
>> different and there are different established workflows. And you know that
>> most people don't go to the extra effort of changing their established
>> workflow, just for a small commit. That means less contributions, which
>> means in the end less committers.
>>
>> Times change and so do the methods.
>>
>> The only reason I see to keep the centralized singled repo way is for legal
>> reasons. And since that is a very strong reason there is nothing really to
>> discuss with github for the moment. Unless there is some kind of cooperation
>> between apache and github, or apache offers something like github (I doubt
>> there is enough manpower for that).
>
> I believe the alternatives right now are between a single centralized repo at
> apache vs. a single centralized repo at github, so I see no need to
> always mention
> "centralized single".
>
> Apache could set up something like gitlab for their repos,
> (https://about.gitlab.com/),
> without too much manpower involved.
>
> Or Apache could decree that it is acceptable for projects to use
> github to merge PRs
> before syncing them to the "primary" repository at apache, and set up
> syncing support
> with commit hooks.
>
> Since the word "primary" is open for interpretation, it might be
> possible to convince
> Apache that what makes a repo primary for a project is that releases
> are made from
> that repo, not that all merges are done there first.
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Thibault Kruse <ti...@googlemail.com>.
On Sat, Oct 24, 2015 at 11:28 AM, Jochen Theodorou <bl...@gmx.org> wrote:
> On 23.10.2015 13:47, Jim Jagielski wrote:
>> You do know, of course, that this workflow that has been used for
>> decades at the ASF was started by and built-around and designed-for
>> unpaid volunteers to do just that, right?
>
>
> Oh,I would be careful with "used for decades". Just because people don't
> know it better and got used it, doesn't mean it is good. I would for example
> not want to change from Git (or Subversion) to CVS, just because CVS is in
> use for decades. Back then, there was no real alternative. Today times are
> different and there are different established workflows. And you know that
> most people don't go to the extra effort of changing their established
> workflow, just for a small commit. That means less contributions, which
> means in the end less committers.
>
> Times change and so do the methods.
>
> The only reason I see to keep the centralized singled repo way is for legal
> reasons. And since that is a very strong reason there is nothing really to
> discuss with github for the moment. Unless there is some kind of cooperation
> between apache and github, or apache offers something like github (I doubt
> there is enough manpower for that).
I believe the alternatives right now are between a single centralized repo at
apache vs. a single centralized repo at github, so I see no need to
always mention
"centralized single".
Apache could set up something like gitlab for their repos,
(https://about.gitlab.com/),
without too much manpower involved.
Or Apache could decree that it is acceptable for projects to use
github to merge PRs
before syncing them to the "primary" repository at apache, and set up
syncing support
with commit hooks.
Since the word "primary" is open for interpretation, it might be
possible to convince
Apache that what makes a repo primary for a project is that releases
are made from
that repo, not that all merges are done there first.
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Jochen Theodorou <bl...@gmx.org>.
On 23.10.2015 13:47, Jim Jagielski wrote:
>
>> On Oct 23, 2015, at 1:18 AM, Russel Winder <ru...@winder.org.uk> wrote:
>>
>> OK, so Apache still doesn't really get Git and GitHub oriented
>> workflows,
>
> Sure we do.
and because of the "one true repository at Apache" policy, it is largely
ignored. Now some people say it is ignored because they don't get it,
while others say Apache cannot do it like that for legal reasons. The
result is the same. Git and GitHub oriented workflows are more or less
cumbersome or impossible.
[...]
>> What is the fundamental workflow problem that you feel is
>> causing pain? Whatever it is, we must fix it, even if we have to
>> subvert the current official "Apache Way" workflow.
>
> Really? I assume that one reason Groovy joined the ASF is
> so it could partake of the numerous successes enjoyed by
> ASF projects which use and depend on the 'current official
> "Apache Way" workflow', whatever that is.
I really don't like "partake" here. Groovy joined Apache mainly because
Codehaus did shut down and Apache was the best alternative. Considering
for example how smoothly we got the JIRA over, this was imho a good
choice too.
>> Having a workflow
>> that works and is easy so that unpaid volunteers can actually achieve
>> progress on the project, but nonetheless ends up with the Apache
>> repository as mainline, is far, far, far more important than adherence
>> to some particular quasi-political philosophy.
>
> You do know, of course, that this workflow that has been used for
> decades at the ASF was started by and built-around and designed-for
> unpaid volunteers to do just that, right?
Oh,I would be careful with "used for decades". Just because people don't
know it better and got used it, doesn't mean it is good. I would for
example not want to change from Git (or Subversion) to CVS, just because
CVS is in use for decades. Back then, there was no real alternative.
Today times are different and there are different established workflows.
And you know that most people don't go to the extra effort of changing
their established workflow, just for a small commit. That means less
contributions, which means in the end less committers.
Times change and so do the methods.
The only reason I see to keep the centralized singled repo way is for
legal reasons. And since that is a very strong reason there is nothing
really to discuss with github for the moment. Unless there is some kind
of cooperation between apache and github, or apache offers something
like github (I doubt there is enough manpower for that).
bye blackdrag
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Russel Winder <ru...@winder.org.uk>.
On Fri, 2015-10-23 at 07:47 -0400, Jim Jagielski wrote:
> > […]
>
> Yep. We are built around a single canonical repo that
> you build a community and project around.
Clearly a project is defined by the primary mainline repository. The
point at issue here is the surrounding technology that is integral to
the community.
> […]
>
> You do know, of course, that this workflow that has been used for
> decades at the ASF was started by and built-around and designed-for
> unpaid volunteers to do just that, right?
The genesis though was in the days of CVCS, Apache Groovy and it's
community live in a DVCS context. Via GitHub and BitBucket, (and others
but these two are the survivors), pull requests are at the core of
being a community. Codehaus and GoogleCode died because the
organization were either not willing or not able to amend their CVCS
workflows to incorporate the technological changes that DVCS brings. It
is not clear that SourceForge is ale to make the change. Apache Git
repositories are not currently embedded in the technology a Git
community expects the Git repositories to be embedded in.
Handling pull requests needs either a tool such as Gerrit or Rietveld
or using the built in pull request system, in Apache Groovy case at
GitHub. The Go team choose to have their primary mainline on Google
servers with a mirror at GitHub. No pull request is processed on
GitHub, they must all go via the Gerrit instance run by Google. This
model would suit Apache well given the current insistance (which is
entirely reasonable for legal reasons) on the repository on Apache
servers being the primary mainline. Currently though this is not
available. So the Apache Groovy committer commiters either have to use
the GitHub pull request system and fib about making the commits to the
Apache server, or they have to not use the GitHub pull request
technology.
Unless or until Apache has a Gerrit instance so that the Go style
workflow can be put in place, I think it should be entirely acceptable
to Apache to have the GitHub repository seen, not as a mirror of the
Apache one, but as a cache for it. All the workflow happens on GitHub
using the GitHub pull requests as they are meant to be used, and the
committer committers then just push the result to the Apache servers.
To do this would be to work using Git in harmony with the technology
the community is now used to and is willing to use.
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Jim Jagielski <ji...@jaguNET.com>.
> On Oct 23, 2015, at 1:18 AM, Russel Winder <ru...@winder.org.uk> wrote:
>
> OK, so Apache still doesn't really get Git and GitHub oriented
> workflows,
Sure we do.
> it is still fundamentally a CVS/Subversion-based
> organization.
Yep. We are built around a single canonical repo that
you build a community and project around.
> What is the fundamental workflow problem that you feel is
> causing pain? Whatever it is, we must fix it, even if we have to
> subvert the current official "Apache Way" workflow.
Really? I assume that one reason Groovy joined the ASF is
so it could partake of the numerous successes enjoyed by
ASF projects which use and depend on the 'current official
"Apache Way" workflow', whatever that is.
> Having a workflow
> that works and is easy so that unpaid volunteers can actually achieve
> progress on the project, but nonetheless ends up with the Apache
> repository as mainline, is far, far, far more important than adherence
> to some particular quasi-political philosophy.
>
You do know, of course, that this workflow that has been used for
decades at the ASF was started by and built-around and designed-for
unpaid volunteers to do just that, right?
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Danny Hyun <hy...@gmail.com>.
Would be nice if we could maybe utilize git webhooks to cascade merges.
On Fri, Oct 23, 2015 at 11:52 AM, Thibault Kruse <ti...@googlemail.com>
wrote:
> There is an Apache policy that states that the "primary" repository of
> any apache project must be an apache hosted one (not github,
> bitbucket, etc).
>
> https://www.apache.org/dev/project-requirements:
> > The primary source control repository MUST be administered by the ASF
> Infra team on ASF hardware (currently, either using SVN or git). (citation
> needed)
>
> Now, the word "primary" seems ambiguous to me in what is allowed and
> what is not allowed, but apparently most apache project have decided
> to be on the safe side that new commits should be added to the apache
> repo first, and then synced to github, and never the other way round.
> This prevents the github repo to smell like a "primary" repository,
> even if most of the time, technically there is no difference in the
> end result (same commits end up both at apache and github).
> This means for merging native git commands have to be used by
> committers instead of using the github UI to merge (you pull from
> github, push to apache).
> Since this is a bit of a pain it seems several projects provide
> detailed instructions or custom scripts to help with that.
>
> Examples for custom scripts for merging:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Merging+Github+Pull+Requests
>
> https://github.com/apache/cordova-coho/blob/master/docs/processing-pull-requests.md
> https://github.com/apache/parquet-mr/blob/master/dev/README.md
>
> I would also be curious if that is all that Cedric labelled as
> painful, or whether there are more issues.
>
> On Fri, Oct 23, 2015 at 7:18 AM, Russel Winder <ru...@winder.org.uk>
> wrote:
> > On Thu, 2015-10-22 at 18:02 +0200, Cédric Champeau wrote:
> >> Agreed. The only downside of this is that it is very painful to merge
> >> PRs
> >> on the GitHub Apache mirror, and even more painful to backport
> >> changes.
> >
> > OK, so Apache still doesn't really get Git and GitHub oriented
> > workflows, it is still fundamentally a CVS/Subversion-based
> > organization. What is the fundamental workflow problem that you feel is
> > causing pain? Whatever it is, we must fix it, even if we have to
> > subvert the current official "Apache Way" workflow. Having a workflow
> > that works and is easy so that unpaid volunteers can actually achieve
> > progress on the project, but nonetheless ends up with the Apache
> > repository as mainline, is far, far, far more important than adherence
> > to some particular quasi-political philosophy.
> >
> > --
> > Russel.
> >
> =============================================================================
> > Dr Russel Winder t: +44 20 7585 2200 voip:
> sip:russel.winder@ekiga.net
> > 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
> > London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
> >
>
--
Thanks,
Daniel Hyun
blog: http://hyunlabs.com/
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Thibault Kruse <ti...@googlemail.com>.
There is an Apache policy that states that the "primary" repository of
any apache project must be an apache hosted one (not github,
bitbucket, etc).
https://www.apache.org/dev/project-requirements:
> The primary source control repository MUST be administered by the ASF Infra team on ASF hardware (currently, either using SVN or git). (citation needed)
Now, the word "primary" seems ambiguous to me in what is allowed and
what is not allowed, but apparently most apache project have decided
to be on the safe side that new commits should be added to the apache
repo first, and then synced to github, and never the other way round.
This prevents the github repo to smell like a "primary" repository,
even if most of the time, technically there is no difference in the
end result (same commits end up both at apache and github).
This means for merging native git commands have to be used by
committers instead of using the github UI to merge (you pull from
github, push to apache).
Since this is a bit of a pain it seems several projects provide
detailed instructions or custom scripts to help with that.
Examples for custom scripts for merging:
https://cwiki.apache.org/confluence/display/KAFKA/Merging+Github+Pull+Requests
https://github.com/apache/cordova-coho/blob/master/docs/processing-pull-requests.md
https://github.com/apache/parquet-mr/blob/master/dev/README.md
I would also be curious if that is all that Cedric labelled as
painful, or whether there are more issues.
On Fri, Oct 23, 2015 at 7:18 AM, Russel Winder <ru...@winder.org.uk> wrote:
> On Thu, 2015-10-22 at 18:02 +0200, Cédric Champeau wrote:
>> Agreed. The only downside of this is that it is very painful to merge
>> PRs
>> on the GitHub Apache mirror, and even more painful to backport
>> changes.
>
> OK, so Apache still doesn't really get Git and GitHub oriented
> workflows, it is still fundamentally a CVS/Subversion-based
> organization. What is the fundamental workflow problem that you feel is
> causing pain? Whatever it is, we must fix it, even if we have to
> subvert the current official "Apache Way" workflow. Having a workflow
> that works and is easy so that unpaid volunteers can actually achieve
> progress on the project, but nonetheless ends up with the Apache
> repository as mainline, is far, far, far more important than adherence
> to some particular quasi-political philosophy.
>
> --
> Russel.
> =============================================================================
> Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
> 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
> London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
>
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Thibault Kruse <ti...@googlemail.com>.
Looking at the incubator reports:
https://wiki.apache.org/incubator/June2015
https://wiki.apache.org/incubator/October2015
it seems that github integration is also debated in certain other
apache projects, so probably some solution will evolve or has already
evolved.
On Fri, Oct 23, 2015 at 9:58 AM, Russel Winder <ru...@winder.org.uk> wrote:
> On Fri, 2015-10-23 at 09:13 +0200, Bertrand Delacretaz wrote:
>> Hi,
>>
>> On Fri, Oct 23, 2015 at 7:18 AM, Russel Winder <ru...@winder.org.uk>
>> wrote:
>> > ...Whatever it is, we must fix it, even if we have to
>> > subvert the current official "Apache Way" workflow....
>>
>> I don't think there's an official Apache workflow related to GitHub
>> usage, but if you want to dig into that it's probably worth looking
>> how other projects do it. CouchDB and Cordova come to mind as early
>> users of Git at the ASF.
>
> As I had been led to believe, Apache rules demand that pull requests
> from the GitHub-verse were not allowed to be merged on the GitHub
> mainline repository and then that repository synced with the Apache
> master mainline one. Instead committers have to faff around with
> personal Git repositories and push changesets separately to the Apache
> mainline and the GitHub mainline thereby losing all the support,
> simplicity, and benefit of the whole GitHub pull request
> infrastructure, and moreover making the whole process annoyingly
> awkward.
>
>> The best place for such cross-project discussions might be the
>> dev@community.apache.org list, see http://community.apache.org/
>
> I'll let Cédric, Paul, Jochen, and Pascal take the lead on whether to
> take this issue there. They are the people this impacts.
>
> --
> Russel.
> =============================================================================
> Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
> 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
> London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
>
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Russel Winder <ru...@winder.org.uk>.
On Fri, 2015-10-23 at 09:13 +0200, Bertrand Delacretaz wrote:
> Hi,
>
> On Fri, Oct 23, 2015 at 7:18 AM, Russel Winder <ru...@winder.org.uk>
> wrote:
> > ...Whatever it is, we must fix it, even if we have to
> > subvert the current official "Apache Way" workflow....
>
> I don't think there's an official Apache workflow related to GitHub
> usage, but if you want to dig into that it's probably worth looking
> how other projects do it. CouchDB and Cordova come to mind as early
> users of Git at the ASF.
As I had been led to believe, Apache rules demand that pull requests
from the GitHub-verse were not allowed to be merged on the GitHub
mainline repository and then that repository synced with the Apache
master mainline one. Instead committers have to faff around with
personal Git repositories and push changesets separately to the Apache
mainline and the GitHub mainline thereby losing all the support,
simplicity, and benefit of the whole GitHub pull request
infrastructure, and moreover making the whole process annoyingly
awkward.
> The best place for such cross-project discussions might be the
> dev@community.apache.org list, see http://community.apache.org/
I'll let Cédric, Paul, Jochen, and Pascal take the lead on whether to
take this issue there. They are the people this impacts.
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: PR and workflow [was [ANN] New committer: Shil Sinha]
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Fri, Oct 23, 2015 at 7:18 AM, Russel Winder <ru...@winder.org.uk> wrote:
> ...Whatever it is, we must fix it, even if we have to
> subvert the current official "Apache Way" workflow....
I don't think there's an official Apache workflow related to GitHub
usage, but if you want to dig into that it's probably worth looking
how other projects do it. CouchDB and Cordova come to mind as early
users of Git at the ASF.
The best place for such cross-project discussions might be the
dev@community.apache.org list, see http://community.apache.org/
-Bertrand