You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by "Oliver Lietz (JIRA)" <ji...@apache.org> on 2014/10/01 15:55:35 UTC
[jira] [Created] (SLING-3987) move from Subversion to Git
Oliver Lietz created SLING-3987:
-----------------------------------
Summary: move from Subversion to Git
Key: SLING-3987
URL: https://issues.apache.org/jira/browse/SLING-3987
Project: Sling
Issue Type: Task
Components: Best practices
Reporter: Oliver Lietz
track work for moving to Git
https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to+Git
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Justin Edelson <ju...@justinedelson.com>.
On Wed, Oct 1, 2014 at 12:29 PM, Oliver Lietz <ap...@oliverlietz.de> wrote:
> On Wednesday 01 October 2014 17:25:50 Carsten Ziegeler wrote:
>> I'm not against moving to git, but the number one criteria would be to have
>> a git repository per module.
>
> To be honest, that is not a criteria.
I'll rephase Carsten's assertion slightly differently - we need a
repository per release unit. So for our purposes, this is mostly one
per module, but not necessarily always (Launchpad, for example).
> I also don't like one big repo (yet),
> but what problem(s) should one repo per module solve (we would than have
> between 200 and 300 repos)?
>
> I don't like to have the complete project source under a module's tag but the
> way it is now on GitHub where a tag for a module only contains that module.
> Is it that what you want to get from one repo per module?
I want to be able to diff a tag against master and get something
coherent. That can IMHO only happen if you have a repository per
release unit.
>
> Checking out/clone a complete big repo shouldn't be expensive - or several
> like bundles, contrib, installer, launchpad, parent, performance, samples,
> site, testing, tooling (which makes it harder to move modules between bundles
> and contrib but this is a different story).
There are a variety of solutions to this. Git Slave seems to be
working well enough.
Regards,
Justin
>
> O.
>
>> Carsten
>>
>> 2014-10-01 16:00 GMT+02:00 Oliver Lietz <ap...@oliverlietz.de>:
>> > Please discuss on list and add your findings to the wiki:
>> >
>> >
>> > https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to
>> > +Git
>> >
>> > Thanks,
>> > O.
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Wed, Oct 1, 2014 at 10:20 PM, Justin Edelson
<ju...@justinedelson.com> wrote:
> One other thing to consider is that *assuming* that we use a repo per
> module/release unit then we will be totally dependent upon infra to
> create new modules. This doesn't happen too frequently, but it is a
> significant workflow change...
New modules happen relatively frequently IMO. We might use the
whiteboard module for them while waiting for repository creation. Or
work with infra to allow PMCs to create new modules.
-Bertrand
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Justin Edelson <ju...@justinedelson.com>.
One other thing to consider is that *assuming* that we use a repo per
module/release unit then we will be totally dependent upon infra to
create new modules. This doesn't happen too frequently, but it is a
significant workflow change.
Justin
On Wed, Oct 1, 2014 at 12:44 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> I have worked with large single git repositories containing hundreds of
> modules and it was a pain. It gets expensive and slow as it grows pretty
> fast. We might not be in the range where it is noticeable now with Sling.
> And we always ended up in splitting them into single repositories.
>
> All operations (tagging, releasing, cloning/checking out, versioning,
> forking etc) happen on a single module - so a repository for a module
> exactly fits this model. Why should I clone a large repository, just to be
> able to work on commons scheduler for example?
> Also moving stuff out off / into contrib is totally easy with separate
> modules.
>
> We have a modular setup, so we should account for it.
>
> And if we stick with a single repository, why do we need to move at all -
> isn't there the git mirror already?
>
> Carsten
>
> 2014-10-01 18:29 GMT+02:00 Oliver Lietz <ap...@oliverlietz.de>:
>
>> On Wednesday 01 October 2014 17:25:50 Carsten Ziegeler wrote:
>> > I'm not against moving to git, but the number one criteria would be to
>> have
>> > a git repository per module.
>>
>> To be honest, that is not a criteria. I also don't like one big repo (yet),
>> but what problem(s) should one repo per module solve (we would than have
>> between 200 and 300 repos)?
>>
>> I don't like to have the complete project source under a module's tag but
>> the
>> way it is now on GitHub where a tag for a module only contains that module.
>> Is it that what you want to get from one repo per module?
>>
>> Checking out/clone a complete big repo shouldn't be expensive - or several
>> like bundles, contrib, installer, launchpad, parent, performance, samples,
>> site, testing, tooling (which makes it harder to move modules between
>> bundles
>> and contrib but this is a different story).
>>
>> O.
>>
>> > Carsten
>> >
>> > 2014-10-01 16:00 GMT+02:00 Oliver Lietz <ap...@oliverlietz.de>:
>> > > Please discuss on list and add your findings to the wiki:
>> > >
>> > >
>> > >
>> https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to
>> > > +Git
>> > >
>> > > Thanks,
>> > > O.
>>
>
>
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Carsten Ziegeler <cz...@apache.org>.
I have worked with large single git repositories containing hundreds of
modules and it was a pain. It gets expensive and slow as it grows pretty
fast. We might not be in the range where it is noticeable now with Sling.
And we always ended up in splitting them into single repositories.
All operations (tagging, releasing, cloning/checking out, versioning,
forking etc) happen on a single module - so a repository for a module
exactly fits this model. Why should I clone a large repository, just to be
able to work on commons scheduler for example?
Also moving stuff out off / into contrib is totally easy with separate
modules.
We have a modular setup, so we should account for it.
And if we stick with a single repository, why do we need to move at all -
isn't there the git mirror already?
Carsten
2014-10-01 18:29 GMT+02:00 Oliver Lietz <ap...@oliverlietz.de>:
> On Wednesday 01 October 2014 17:25:50 Carsten Ziegeler wrote:
> > I'm not against moving to git, but the number one criteria would be to
> have
> > a git repository per module.
>
> To be honest, that is not a criteria. I also don't like one big repo (yet),
> but what problem(s) should one repo per module solve (we would than have
> between 200 and 300 repos)?
>
> I don't like to have the complete project source under a module's tag but
> the
> way it is now on GitHub where a tag for a module only contains that module.
> Is it that what you want to get from one repo per module?
>
> Checking out/clone a complete big repo shouldn't be expensive - or several
> like bundles, contrib, installer, launchpad, parent, performance, samples,
> site, testing, tooling (which makes it harder to move modules between
> bundles
> and contrib but this is a different story).
>
> O.
>
> > Carsten
> >
> > 2014-10-01 16:00 GMT+02:00 Oliver Lietz <ap...@oliverlietz.de>:
> > > Please discuss on list and add your findings to the wiki:
> > >
> > >
> > >
> https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to
> > > +Git
> > >
> > > Thanks,
> > > O.
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Thu, Oct 2, 2014 at 3:14 PM, Robert Munteanu <ro...@apache.org> wrote:
> ...How will this
> story look for a non-committer trying to get the Sling source in order
> to inspect it / build it / contribute to it? Do we ask them upfront to
> install an external tool ( gitslave )...
Requiring that wouldn't be good, but we can suggest it and put
.gitslave files in our code repository to help.
> ...Or do we we guarantee that the
> Launchpad is always buildable outside the reactor by banning SNAPSHOT
> dependencies or by always deploying bundle SNAPSHOTS to the apache
> maven repo?...
Deploying snapshots is a good idea anyway, and we should be able to
enable that in our Jenkins builds.
-Bertrand
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Justin Edelson <ju...@justinedelson.com>.
And, FWIW, I think the input of non-PMC members on this question is
*very* important. So thanks Radu for speaking up!
On Thu, Oct 2, 2014 at 8:43 PM, Justin Edelson <ju...@justinedelson.com> wrote:
> Hi Radu,
> I agree. But this is an orthogonal problem to the question of what SCM
> system we use.
>
> I actually thought that we did deploy snapshots via CI. I see some
> artifacts under
> https://repository.apache.org/content/repositories/snapshots/org/apache/sling/
> but it is inconsistent.
>
> Anyone know what is going on with the snapshot deployment? And what we
> can do to re-enable it?
>
> FWIW, I don't think we should (or really could) following a
> launchpad-only-contains-releases rule. But I'm willing to be convinced
> otherwise if the snapshot thing is really impossible.
>
> Justin
>
> On Thu, Oct 2, 2014 at 3:46 PM, Radu Cotescu <ra...@apache.org> wrote:
>> Hi,
>> I know that I'm just an occasional contributor to Sling and I noticed that
>> this talk is mostly between PMC members. However, here's my big +1 for what
>> Robert proposed.
>>
>> Currently if somebody wants to contribute a new module to Sling they have
>> to build the full project to get a running launchpad, which (assuming that
>> they didn't use -DskipTests) takes a while.
>>
>> Just my €0.02.
>>
>> Cheers,
>> Radu
>>
>> On Thu, Oct 2, 2014 at 4:14 PM, Robert Munteanu <ro...@apache.org> wrote:
>>
>>> Or do we we guarantee that the
>>> Launchpad is always buildable outside the reactor by banning SNAPSHOT
>>> dependencies
>>>
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Justin Edelson <ju...@justinedelson.com>.
Hi Radu,
I agree. But this is an orthogonal problem to the question of what SCM
system we use.
I actually thought that we did deploy snapshots via CI. I see some
artifacts under
https://repository.apache.org/content/repositories/snapshots/org/apache/sling/
but it is inconsistent.
Anyone know what is going on with the snapshot deployment? And what we
can do to re-enable it?
FWIW, I don't think we should (or really could) following a
launchpad-only-contains-releases rule. But I'm willing to be convinced
otherwise if the snapshot thing is really impossible.
Justin
On Thu, Oct 2, 2014 at 3:46 PM, Radu Cotescu <ra...@apache.org> wrote:
> Hi,
> I know that I'm just an occasional contributor to Sling and I noticed that
> this talk is mostly between PMC members. However, here's my big +1 for what
> Robert proposed.
>
> Currently if somebody wants to contribute a new module to Sling they have
> to build the full project to get a running launchpad, which (assuming that
> they didn't use -DskipTests) takes a while.
>
> Just my €0.02.
>
> Cheers,
> Radu
>
> On Thu, Oct 2, 2014 at 4:14 PM, Robert Munteanu <ro...@apache.org> wrote:
>
>> Or do we we guarantee that the
>> Launchpad is always buildable outside the reactor by banning SNAPSHOT
>> dependencies
>>
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Radu Cotescu <ra...@apache.org>.
Hi,
I know that I'm just an occasional contributor to Sling and I noticed that
this talk is mostly between PMC members. However, here's my big +1 for what
Robert proposed.
Currently if somebody wants to contribute a new module to Sling they have
to build the full project to get a running launchpad, which (assuming that
they didn't use -DskipTests) takes a while.
Just my €0.02.
Cheers,
Radu
On Thu, Oct 2, 2014 at 4:14 PM, Robert Munteanu <ro...@apache.org> wrote:
> Or do we we guarantee that the
> Launchpad is always buildable outside the reactor by banning SNAPSHOT
> dependencies
>
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Robert Munteanu <ro...@apache.org>.
On Wed, Oct 1, 2014 at 7:29 PM, Oliver Lietz <ap...@oliverlietz.de> wrote:
>
> Checking out/clone a complete big repo shouldn't be expensive - or several
> like bundles, contrib, installer, launchpad, parent, performance, samples,
> site, testing, tooling (which makes it harder to move modules between bundles
> and contrib but this is a different story).
Well, tooling can easily solve transplanting histories ( been there,
done that ) so let's say we have the following scenarios covered
- migrating our big SVN repo to multiple git repositories, preserving history
- migrating from a whiteboard git repository to a separate git
repository, preserving history
I think, as Bertrand mentioned, that we can live with a delay in
creating repos, given that we have tooling to migrate from
sling-whiteboard to sling-$MODULE.
There are already projects which have a largeish number of git repos ,
see cordova at [1], but we might want to talk to infra first - we'll
definitely have a lot of repos.I counted 266 Maven modules in the
current checkout. We'll likely have fewer git repos, but even 100
sounds like a lot.
Now for that part that I'm not so certain about ... How will this
story look for a non-committer trying to get the Sling source in order
to inspect it / build it / contribute to it? Do we ask them upfront to
install an external tool ( gitslave ) ? Or do we we guarantee that the
Launchpad is always buildable outside the reactor by banning SNAPSHOT
dependencies or by always deploying bundle SNAPSHOTS to the apache
maven repo? Should we build additional scripts to easy checkout?
Robert
[1]: http://git.apache.org/
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Oliver Lietz <ap...@oliverlietz.de>.
On Wednesday 01 October 2014 17:25:50 Carsten Ziegeler wrote:
> I'm not against moving to git, but the number one criteria would be to have
> a git repository per module.
To be honest, that is not a criteria. I also don't like one big repo (yet),
but what problem(s) should one repo per module solve (we would than have
between 200 and 300 repos)?
I don't like to have the complete project source under a module's tag but the
way it is now on GitHub where a tag for a module only contains that module.
Is it that what you want to get from one repo per module?
Checking out/clone a complete big repo shouldn't be expensive - or several
like bundles, contrib, installer, launchpad, parent, performance, samples,
site, testing, tooling (which makes it harder to move modules between bundles
and contrib but this is a different story).
O.
> Carsten
>
> 2014-10-01 16:00 GMT+02:00 Oliver Lietz <ap...@oliverlietz.de>:
> > Please discuss on list and add your findings to the wiki:
> >
> >
> > https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to
> > +Git
> >
> > Thanks,
> > O.
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Carsten Ziegeler <cz...@apache.org>.
I'm not against moving to git, but the number one criteria would be to have
a git repository per module.
Carsten
2014-10-01 16:00 GMT+02:00 Oliver Lietz <ap...@oliverlietz.de>:
> Please discuss on list and add your findings to the wiki:
>
>
> https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to+Git
>
> Thanks,
> O.
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Carsten Ziegeler <cz...@apache.org>.
2014-10-02 15:37 GMT+02:00 Felix Meschberger <fm...@adobe.com>:
> Hi Oliver
>
> Am 01.10.2014 um 16:00 schrieb Oliver Lietz <ap...@oliverlietz.de>:
>
> > Please discuss on list and add your findings to the wiki:
> >
> >
> https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to+Git
>
> First and foremost: Thanks for structuring a discussion which pops up
> every now and then here.
>
> Second: I really like git. It has very nice features, offline-full
> repository is not the least of which. Simple branching is really a plus.
>
> But then: Git is targeted at single monolithic codebases. It can handle
> them perfectly with all the Git goodies. I have yet to see easy to use
> support for modular codebases.
>
> Why is this important ? It is important because Sling is a modular code
> base and a single Sling deployment will contain various versions of the
> Sling modules. Sling modules evolve independently. For each module version
> we create a tag. If in git the tag applies globally. But a single Git
> repository has a single HEAD revision. So you are only able to checkout one
> tag per repository. Testing multiple bundle versions at different stages
> becomes hairy (there are solutions, but not simple/easy ones).
>
And the same goes for branches - with a single repo you could only work at
a single branch for a single module at a time - or use the same branch for
work on multiple ones
Carsten
>
> So for the time being, I am rather on the cautious side with respect to
> moving Sling to Git.
>
> Regards
> Felix
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Felix Meschberger <fm...@adobe.com>.
Hi Oliver
Am 01.10.2014 um 16:00 schrieb Oliver Lietz <ap...@oliverlietz.de>:
> Please discuss on list and add your findings to the wiki:
>
> https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to+Git
First and foremost: Thanks for structuring a discussion which pops up every now and then here.
Second: I really like git. It has very nice features, offline-full repository is not the least of which. Simple branching is really a plus.
But then: Git is targeted at single monolithic codebases. It can handle them perfectly with all the Git goodies. I have yet to see easy to use support for modular codebases.
Why is this important ? It is important because Sling is a modular code base and a single Sling deployment will contain various versions of the Sling modules. Sling modules evolve independently. For each module version we create a tag. If in git the tag applies globally. But a single Git repository has a single HEAD revision. So you are only able to checkout one tag per repository. Testing multiple bundle versions at different stages becomes hairy (there are solutions, but not simple/easy ones).
So for the time being, I am rather on the cautious side with respect to moving Sling to Git.
Regards
Felix
Re: [jira] [Created] (SLING-3987) move from Subversion to Git
Posted by Oliver Lietz <ap...@oliverlietz.de>.
Please discuss on list and add your findings to the wiki:
https://cwiki.apache.org/confluence/display/SLING/Move+from+Subversion+to+Git
Thanks,
O.