You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@turbine.apache.org by Georg Kallidis <ge...@cedis.fu-berlin.de> on 2021/08/12 08:23:13 UTC
[VOTE] Migrate SVN Repos to Git, How to Migrate Site Building + Publishing
Hi Turbine Devs,
I propose this as a vote, which allows us to get an overview of what we
want to do or have to do, the workload, and how to communicate with INFRA.
SVN->Git
(1.1) Move Turbine-core and Turbine-site from SVN to Git as detailed
below. Turbine-Core is currently mirrored only in GitHub/Gitbox.
(1.2) Move Fulcrum components from SVN to Git. Fulcrum is currently
mirrored only in GitHub.
a) try to keep it in one git repo or if not possible split up. b) split it
up in any case. Find more information below!
(1.3) Move Turbine Maven from SVN to Git as detailed below.
Turbine-Archetypes is already a GIT repo.
(2) Performing a Turbine-core release v5.1: As this might take some time
depending on workload, should we release before the migration?
We probably want still use the maven scm plugin to publish releases, but
for building and publishing web content we have to change more (as it
seems):
(3) For each Git repo with a maven-site-configuration to build and publish
web content, we should define either
a) a Jenkins Build file(s) and/or b) a build directive within the
.asf.yaml-file (supports jekyll and pelican) c) an instruction, how to do
it locally, similar how it is explained here [2], but with additional
restrictions (see below .asf.yml file, web root folder). The last
alternative might be too much effort. It might be possible to do this
even for mirrored git-repos (we have to ask INFRA).
We might vote like this:
[ ] +1 support (optional with order): Example: 3,2,1 > 1,2,3 > 2,1,3 / 1a
> 1b
[ ] +0 go ahead I don't care
[ ] -1 no, do not ... , because
Finally to achieve all of this correctly, we have to inform INFRA about
our mappings and other required changes.
- I expect Turbine-core and Turbine-site be migrated without much hassle
and map easily to git from the current svn (the svn structure for
turbine/core and turbine/site is the same).
- Fulcrum components could be kept as currently in a single git repo or
split up each into a separate git repo. As consistent the latter approach
would be I would prefer keeping them together (if possible, see below!).
Reasons are: The master module pom will be "lost". This could be solved
with git modules, but requires some additional effort and advanced git
knowledge + documentation/testing IMO. Who will help? - Maintenance effort
increases (e.g. an update of a shared dependency has to be done in x (x=
number of git repos) commits versus 1 commit, if we keep them as a
collection also in version control. On the other side publishing has to be
discussed with INFRA and is not done out-of-the-box, see below. What do
you prefer?
- Turbine-maven-parent and Parent-assembly might be split up (without
archetypes) as well..
The policy of INFRA on how a project should publish their web site seems
to be now, that by default HTML will be fetched from an branch "asf-site"
inside a git repository. Additionally an .asf.yml file must be present in
this branch [1]. As Turbine is structured in such a way, that the website
is build modularily, we would need then by default a git repo for each
module. We may use "subdir" command in .asf.yaml and default root URL
setting for turbine-site and fulcrum-site, if the latter will not destroy
the tree contents below ;-). The content to be published (in the branch
asf-site) has to be in the root or in a folder "content" or "output"[1].
Building may take place using a Jenkins build file updating the asf-site
branch or a buildbot-definition in .asf.yaml using Jekyll or Pelican hooks
[1]. To keep Fulcrum components together, we might do it like Apache Jena
achieved it to do it for their javadoc: Sync a named branch with the help
of INFRA. Gavin McDonald pointed us to this:
https://cwiki.apache.org/confluence/display/INFRA/How+Apache+Jena+migrated+from+the+CMS
. In step 8 (optional) Jena created a branch javadoc, which maps to a
subfolder in the website, https://github.com/apache/jena-site/tree/javadoc
. We may proceed in a similar way for Fulcrum having only one git repo
with multiple site branches?
Nevertheless, be aware, that currently publishing a site could not be done
and all steps require some effort! Any support is welcome, of course ;-)
Thanks!
Best regards, Georg
[1]
https://cwiki.apache.org/confluence/display/INFRA/git+-+.asf.yaml+features
[2]
https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tips.html
[RESULT][VOTE] Migrate SVN Repos to Git, How to Migrate Site Building + Publishing
Posted by Georg Kallidis <ge...@cedis.fu-berlin.de>.
Thanks, Jeffery, Thomas!
I would summarize the result as follows:
We
1) move our SVN repos to Git. Turbine SVNs as is (almost), Fulcrum as well
using different repos (I follow Thomas' argument, git is not svn), but
remove some components to the attic / archive or declare it dormant (as
Apache Commons components currently inactive): Check list below *. I would
suggest to clean up this as much as possible, before moving. Currently I
count 20 fulcrum (including site, and excluding all in the list below)
2) release Turbine Core later after this is done ASAP.
3) use whatever is appropriate and supported by INFRA to publish our
website, may it be Github tooling, Jenkins or what else. I am not sure, if
we could follow Github tooling to the end ..
I'll start with the process in INFRA-22176, see below as soon as
possible..
@Thomas, thanks for helping out later with git modules, which may be not
required, if we actually move/set up a separate repo "master build" (eg. a
repo fulcrum-build from trunk/pom.xml +xdocs), as you suggested.
Fulcrum-parent may be also an archive candidate, as I can't find any
usages anywhere (?).
If you think anything's missing here/wrong or have another idea, please
don't hesitate ..
Thanks so far!
Best regards, Georg
* Searching
https://lists.apache.org/list.html?dev@turbine.apache.org:lte=3y:attic I
found this list of candidates to archive:
(
https://lists.apache.org/thread.html/50d1e77fb93dd290803347a5bfff00ccc1709ef31d3e59fb64d361cd%40%3Cdev.turbine.apache.org%3E
:
commonsemail
configuration/impl
hsqldb
pbe
naming
resourcemanager
groovy
script
spring
xmlrpc
xslt
+ jetty has a comment in master pom "needs upgrade to eclipse jetty",
but may be as well archived?
+ bsf, osworkflow, dvsl, parent -
Is this appropriate or should be more/less removed? To archive means IMO:
Keep it in svn and do not migrate it to git; declare it like this.
Von: Thomas Vandahl <tv...@apache.org>
An: Turbine Developers List <de...@turbine.apache.org>
Datum: 15.08.2021 11:52
Betreff: Re: [VOTE] Migrate SVN Repos to Git, How to Migrate Site
Building + Publishing
+1
See comments below.
> Am 12.08.2021 um 10:23 schrieb Georg Kallidis
<ge...@cedis.fu-berlin.de>:
>
> Hi Turbine Devs,
>
> I propose this as a vote, which allows us to get an overview of what we
> want to do or have to do, the workload, and how to communicate with
INFRA.
>
> SVN->Git
>
> (1.1) Move Turbine-core and Turbine-site from SVN to Git as detailed
> below. Turbine-Core is currently mirrored only in GitHub/Gitbox.
>
> (1.2) Move Fulcrum components from SVN to Git. Fulcrum is currently
> mirrored only in GitHub.
> a) try to keep it in one git repo or if not possible split up. b) split
it
> up in any case. Find more information below!
I'm in favour of 1.2b because releases from GIT use signed tags and in the
future it would be difficult to know, which component release is meant by
which tag. My understanding of GIT is, that, if in doubt, use more repos.
> (1.3) Move Turbine Maven from SVN to Git as detailed below.
> Turbine-Archetypes is already a GIT repo.
>
> (2) Performing a Turbine-core release v5.1: As this might take some time
> depending on workload, should we release before the migration?
If the release is not needed urgently by anyone (which I assume), a
release after the migration might help to prove that everything works
again.
> We probably want still use the maven scm plugin to publish releases, but
> for building and publishing web content we have to change more (as it
> seems):
>
> (3) For each Git repo with a maven-site-configuration to build and
publish
> web content, we should define either
> a) a Jenkins Build file(s) and/or b) a build directive within the
> .asf.yaml-file (supports jekyll and pelican) c) an instruction, how to
do
> it locally, similar how it is explained here [2], but with additional
> restrictions (see below .asf.yml file, web root folder). The last
> alternative might be too much effort. It might be possible to do this
> even for mirrored git-repos (we have to ask INFRA).
Whether I like it or not: I see a lot of movement into the direction of
Github tooling. Probably less maintenance, dunno. Shall we follow that
road?
> - Fulcrum components could be kept as currently in a single git repo or
> split up each into a separate git repo. As consistent the latter
approach
> would be I would prefer keeping them together (if possible, see below!).
> Reasons are: The master module pom will be "lost". This could be solved
> with git modules, but requires some additional effort and advanced git
> knowledge + documentation/testing IMO. Who will help? - Maintenance
effort
> increases (e.g. an update of a shared dependency has to be done in x (x=
> number of git repos) commits versus 1 commit, if we keep them as a
> collection also in version control. On the other side publishing has to
be
> discussed with INFRA and is not done out-of-the-box, see below. What do
> you prefer?
As stated above, I'm in favour for the one-component/one-repo solution.
The parent POM is just another module - similar to the Turbine parent POM.
If your concern is about local builds, it's just a matter of building the
parent first.
GIT modules are easy, I can help with that. I propose to create a separate
"master build" repo with a layout suitable for building everything at
once.
I also support Jeffs idea of an attic of Fulcrum components.
Bye, Thomas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@turbine.apache.org
For additional commands, e-mail: dev-help@turbine.apache.org
Re: [VOTE] Migrate SVN Repos to Git, How to Migrate Site Building +
Publishing
Posted by Thomas Vandahl <tv...@apache.org>.
+1
See comments below.
> Am 12.08.2021 um 10:23 schrieb Georg Kallidis <ge...@cedis.fu-berlin.de>:
>
> Hi Turbine Devs,
>
> I propose this as a vote, which allows us to get an overview of what we
> want to do or have to do, the workload, and how to communicate with INFRA.
>
> SVN->Git
>
> (1.1) Move Turbine-core and Turbine-site from SVN to Git as detailed
> below. Turbine-Core is currently mirrored only in GitHub/Gitbox.
>
> (1.2) Move Fulcrum components from SVN to Git. Fulcrum is currently
> mirrored only in GitHub.
> a) try to keep it in one git repo or if not possible split up. b) split it
> up in any case. Find more information below!
I'm in favour of 1.2b because releases from GIT use signed tags and in the future it would be difficult to know, which component release is meant by which tag. My understanding of GIT is, that, if in doubt, use more repos.
> (1.3) Move Turbine Maven from SVN to Git as detailed below.
> Turbine-Archetypes is already a GIT repo.
>
> (2) Performing a Turbine-core release v5.1: As this might take some time
> depending on workload, should we release before the migration?
If the release is not needed urgently by anyone (which I assume), a release after the migration might help to prove that everything works again.
> We probably want still use the maven scm plugin to publish releases, but
> for building and publishing web content we have to change more (as it
> seems):
>
> (3) For each Git repo with a maven-site-configuration to build and publish
> web content, we should define either
> a) a Jenkins Build file(s) and/or b) a build directive within the
> .asf.yaml-file (supports jekyll and pelican) c) an instruction, how to do
> it locally, similar how it is explained here [2], but with additional
> restrictions (see below .asf.yml file, web root folder). The last
> alternative might be too much effort. It might be possible to do this
> even for mirrored git-repos (we have to ask INFRA).
Whether I like it or not: I see a lot of movement into the direction of Github tooling. Probably less maintenance, dunno. Shall we follow that road?
> - Fulcrum components could be kept as currently in a single git repo or
> split up each into a separate git repo. As consistent the latter approach
> would be I would prefer keeping them together (if possible, see below!).
> Reasons are: The master module pom will be "lost". This could be solved
> with git modules, but requires some additional effort and advanced git
> knowledge + documentation/testing IMO. Who will help? - Maintenance effort
> increases (e.g. an update of a shared dependency has to be done in x (x=
> number of git repos) commits versus 1 commit, if we keep them as a
> collection also in version control. On the other side publishing has to be
> discussed with INFRA and is not done out-of-the-box, see below. What do
> you prefer?
As stated above, I'm in favour for the one-component/one-repo solution. The parent POM is just another module - similar to the Turbine parent POM.
If your concern is about local builds, it's just a matter of building the parent first.
GIT modules are easy, I can help with that. I propose to create a separate "master build" repo with a layout suitable for building everything at once.
I also support Jeffs idea of an attic of Fulcrum components.
Bye, Thomas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@turbine.apache.org
For additional commands, e-mail: dev-help@turbine.apache.org
Re: [VOTE] Migrate SVN Repos to Git, How to Migrate Site Building +
Publishing
Posted by Jeffery Painter <je...@jivecast.com>.
Hi Georg,
I will defer to you, so I will also endorse your recommendation.
[x] +1 support 1,2,3 > . , 1a > 1b
My only additional comment is can we move some of the fulcrum components that are no longer used to the attic at the same time and avoid moving those over to git? Might help avoid confusion down the road.
-
Jeff
On 8/12/21 4:23 AM, Georg Kallidis wrote:
> Hi Turbine Devs,
>
> I propose this as a vote, which allows us to get an overview of what we
> want to do or have to do, the workload, and how to communicate with INFRA.
>
> SVN->Git
>
> (1.1) Move Turbine-core and Turbine-site from SVN to Git as detailed
> below. Turbine-Core is currently mirrored only in GitHub/Gitbox.
>
> (1.2) Move Fulcrum components from SVN to Git. Fulcrum is currently
> mirrored only in GitHub.
> a) try to keep it in one git repo or if not possible split up. b) split it
> up in any case. Find more information below!
>
> (1.3) Move Turbine Maven from SVN to Git as detailed below.
> Turbine-Archetypes is already a GIT repo.
>
> (2) Performing a Turbine-core release v5.1: As this might take some time
> depending on workload, should we release before the migration?
>
> We probably want still use the maven scm plugin to publish releases, but
> for building and publishing web content we have to change more (as it
> seems):
>
> (3) For each Git repo with a maven-site-configuration to build and publish
> web content, we should define either
> a) a Jenkins Build file(s) and/or b) a build directive within the
> .asf.yaml-file (supports jekyll and pelican) c) an instruction, how to do
> it locally, similar how it is explained here [2], but with additional
> restrictions (see below .asf.yml file, web root folder). The last
> alternative might be too much effort. It might be possible to do this
> even for mirrored git-repos (we have to ask INFRA).
>
> We might vote like this:
>
> [ ] +1 support (optional with order): Example: 3,2,1 > 1,2,3 > 2,1,3 / 1a
>> 1b
> [ ] +0 go ahead I don't care
> [ ] -1 no, do not ... , because
>
> Finally to achieve all of this correctly, we have to inform INFRA about
> our mappings and other required changes.
>
> - I expect Turbine-core and Turbine-site be migrated without much hassle
> and map easily to git from the current svn (the svn structure for
> turbine/core and turbine/site is the same).
> - Fulcrum components could be kept as currently in a single git repo or
> split up each into a separate git repo. As consistent the latter approach
> would be I would prefer keeping them together (if possible, see below!).
> Reasons are: The master module pom will be "lost". This could be solved
> with git modules, but requires some additional effort and advanced git
> knowledge + documentation/testing IMO. Who will help? - Maintenance effort
> increases (e.g. an update of a shared dependency has to be done in x (x=
> number of git repos) commits versus 1 commit, if we keep them as a
> collection also in version control. On the other side publishing has to be
> discussed with INFRA and is not done out-of-the-box, see below. What do
> you prefer?
> - Turbine-maven-parent and Parent-assembly might be split up (without
> archetypes) as well..
> The policy of INFRA on how a project should publish their web site seems
> to be now, that by default HTML will be fetched from an branch "asf-site"
> inside a git repository. Additionally an .asf.yml file must be present in
> this branch [1]. As Turbine is structured in such a way, that the website
> is build modularily, we would need then by default a git repo for each
> module. We may use "subdir" command in .asf.yaml and default root URL
> setting for turbine-site and fulcrum-site, if the latter will not destroy
> the tree contents below ;-). The content to be published (in the branch
> asf-site) has to be in the root or in a folder "content" or "output"[1].
> Building may take place using a Jenkins build file updating the asf-site
> branch or a buildbot-definition in .asf.yaml using Jekyll or Pelican hooks
> [1]. To keep Fulcrum components together, we might do it like Apache Jena
> achieved it to do it for their javadoc: Sync a named branch with the help
> of INFRA. Gavin McDonald pointed us to this:
> https://cwiki.apache.org/confluence/display/INFRA/How+Apache+Jena+migrated+from+the+CMS
> . In step 8 (optional) Jena created a branch javadoc, which maps to a
> subfolder in the website, https://github.com/apache/jena-site/tree/javadoc
> . We may proceed in a similar way for Fulcrum having only one git repo
> with multiple site branches?
>
> Nevertheless, be aware, that currently publishing a site could not be done
> and all steps require some effort! Any support is welcome, of course ;-)
>
> Thanks!
>
> Best regards, Georg
>
> [1]
> https://cwiki.apache.org/confluence/display/INFRA/git+-+.asf.yaml+features
> [2]
> https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tips.html
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@turbine.apache.org
For additional commands, e-mail: dev-help@turbine.apache.org