svnpubsub: migration plan

[Nicolas LE BAS <ma...@nlebas.net>]

Hi everyone,

I am late on this subject for a number of personal reasons, however I'd
like to present the PMC with this plan to migrate tiles web site to
svnpubsub.

We must take into account the deadline of Dec 31st, all necessary
migration requests for be filed at infra before that date. Then we must
publish using svnpubsub. We should also automate that publication by
using the maven svnpubsub plugin, although the automation aspect is not
actually a blocker at Dec 31st.

========
Analysis
========

There are actually two websites that we should migrate to svnpubsub:
1. the distribution website at http://www.apache.org/dist/tiles/.
2. the maven website at http://tiles.apache.org/ (which includes 4
parts, tiles-site, tiles-autotag, tiles-request and tiles-parent);

Concerning 1., I've already committed the currently distributed files to
https://dist.apache.org/repos/dist/release/tiles on Dec 3. We just need
to file the request for infra to take it into account and get the mailer
config set up, as Daniel Shahaf pointed out.

Concerning 2., we need to host the website itself in one or several
parts inside our svn tree at http://svn.apache.org/repos/asf/tiles/.
Then we file a request for infra to map the selected subdirectories to
the web.

So we need to decide on the layout of the website in SVN. I've been
reviewing where other projects have hosted their site. The most common
location seems to be site/publish (myfaces, avro, cassandra), other
choices include site(couchdb), site/site(cocoon),
site/trunk/docs(tomcat), infra/websites/production/camel/content(camel)...

Of special interest to us is the umbrella TLP Ant, which is hosting ant
at site/ant/production, ivy at site/ivy/production, ivyde at
site/ivyde/production.

So the most popular choice is to host the production website somewhere
in the site subdirectory. Otherwise everyone is doing it differently.
This is a complication to us because the site directory is already
hosting our "main" website.

Then we also have to consider updating our release process
(http://tiles.apache.org/framework/dev/release.html). We're currently
using the following spaces:
- for staging distribution artefacts: /www/people.apache.org/builds/tiles/
- for distributing artefacts: /www/www.apache.org/dist/tiles/
- for publishing the website: /www/tiles.apache.org (including
subdirectories tiles-request, tiles-autotag, framework and
subdirectories for past versions)

I'd also like to take the opportunity to introduce a staging location
for the website before voting.

===============
Proposed layout
===============
My personal choice would go for the following layout, similar to
cassandra's:
/site/
  /doap_Tiles.rdf
  /forms/
  /KEYS
  /pom.xml
  /publish/ (*)
  /src/
  /staging/ (*)

(*) The new directory site/publish would contain the publishable version
of the website. The new directory site/staging would be branch (as in
svn copy) of site/publish, and would be the eventual target of the maven
svnpubsub plugin in the various projects (site, framework, request,
autotag).

The release process would be modified as follows:
- updating the index and the download pages: xdoc/index.xml and
apt/download.apt, would happen in the staging directory before the vote.
- after the vote, we would just have 2 operations to perform: committing
the new artefacts to dist.apache.org and svn merge staging into publish.

Please share your thoughts on this proposition.
If nobody commented before Dec 25, I will then proceed with the above
plan in order to meet the deadline of Dec 31.

Thank you all for your attention.

Nick.

Re: svnpubsub: migration plan

[Mick Semb Wever <mc...@apache.org>]

> > In other cases, like "wildcards documentation", documentation
> > enhancements come in that can be pushed immediately to staging. my
> > question what is best practice here?
> 
> If this is indeed the need, then I would gather all the documentation
> into a single maven project and separate it from the code itself.

I was also trying to highlight that there isn't just one need. That
there are two separate needs for documentation updates. Incompatible
changes related to new releases, and ongoing documentation improvements
based off the last release.

> > The way i see it must be done seems rather clumsy…
> > 1) checkout ^/framework/tags/tiles-parent-3.0.1/
> > 2) svn merge -rXXX ^/framework/trunk/src/site/apt/tutorial/advancedwildcard.apt
> > 3) mvn site:site
> > 4) cp target/site/tutorial/advanced/wildcard.html ../site/staging/framework/tutorial/advanced/wildcard.html
> > 4a) hack the snapshot header back to the release version
> > 5) svn ci ../site/staging/framework/tutorial/advanced/wildcard.html
> > 6) cast vote
> 
> This is basically the same process as the release of a project. 

Yes it is. And it seems a little superfluous to me. What highlights my
point is how we deal with http://svn.apache.org/repos/asf/tiles/site/

We make no releases of this code. eg there is no version number at the
top of these webpages. Any improvement there follows a simpler approach…

1) svn ci src/site/...
2) mvn site:site
3) cp target/site staging/  (maven-scm-publish-plugin removes this?)
4) svn ci staging
5) cast vote


I'm questioning the underlying reason of making releases and whether it
applies to documentation. Releases best solve the need for distributed
artifacts. Artifacts whom consumers take and use. The website, be it
static content or a dynamic application like a war file, is used by
others but is not "taken and used". We don't need to make releases of
such things and continuous deployment (CD) works. CD typically deals
with two branches: trunk and stable; where stable is pushed to
production.

I can see it to be easier if documentation follows the CD approach. This
permits for both use cases. Those incompatible doc changes don't get
push prematurely to the website because they are sitting in trunk. Quick
and compatible doc improvement can be pushed quickly out by "deploying"
them directly from our "stable branch" without having to make releases.
Our stable branch corresponds to the last release, eg
^/framework/branches/TILES_3_0_X/. And deploying documentation for us
means mvn site:site, maven-scm-publish-plugin, and casting a vote to
push staging to publish. 

If we are to go down this path then we should remove the version from
all webpages. The stable version is stated clear enough on the
frontpage.

~mck


-- 
“People only see what they're prepared to see.” - Ralph Waldo Emerson 

| http://github.com/finn-no | http://tech.finn.no |

Re: svnpubsub: migration plan

[Nicolas LE BAS <ma...@nlebas.net>]

On 12-12-27 06:06 AM, Mick Semb Wever wrote:
> 
> In some cases documentation must wait until a release is made, ie that
> documentation depends on new features yet to be released. Here
> everything just falls into the release process.
> 
> In other cases, like "wildcards documentation", documentation
> enhancements come in that can be pushed immediately to staging. my
> question what is best practice here?

If this is indeed the need, then I would gather all the documentation
into a single maven project and separate it from the code itself.

Then we can use "wildcard" version numbers to track the documentation
(3.0.x, 3.1.x...) and update it in a different timeframe from the code
releases. Or we can see how to setup the apache CMS for the doc.

> The way i see it must be done seems rather clumsy…
> 1) checkout ^/framework/tags/tiles-parent-3.0.1/
> 2) svn merge -rXXX ^/framework/trunk/src/site/apt/tutorial/advancedwildcard.apt
> 3) mvn site:site
> 4) cp target/site/tutorial/advanced/wildcard.html ../site/staging/framework/tutorial/advanced/wildcard.html
> 4a) hack the snapshot header back to the release version
> 5) svn ci ../site/staging/framework/tutorial/advanced/wildcard.html
> 6) cast vote

This is basically the same process as the release of a project.

Re: svnpubsub: migration plan

[Mick Semb Wever <mc...@apache.org>]

> > Will it be and at what address would the staging website be visible at? 
> 
> None, unless you checkout site/staging to some web server at localhost.
> Is that a problem?

No :-) Although it is safest to cast a vote against one published
staging url.

> > Will this mean that updates to tiles.apache.org can only be done via
> > releases?
> [snip]
> maven-scm-publish-plugin works by checking out the current site into a
> local directory, copying the new files into that location, and
> committing again. It doesn't remove anything as far as I can tell. 

I can see using maven-scm-publish-plugin outside of the release cycle
often useful. 

For example the patch from Michael re "wildcards documentation" i plan
to commit (it makes sense to wait until svnpubsub is in place). I don't
see the need for a release of tiles framework just to see it go to
staging and then publish? But the maven generated sites list the version
number at the top of the page, eg
> Last Published: 2012-07-11  | Version: 3.0.1 
and manually generating the site and pushing wildcard.html into
site/staging/ means this header becomes "3.1-SNAPSHOT". Ugly.

In some cases documentation must wait until a release is made, ie that
documentation depends on new features yet to be released. Here
everything just falls into the release process.

In other cases, like "wildcards documentation", documentation
enhancements come in that can be pushed immediately to staging. my
question what is best practice here?

The way i see it must be done seems rather clumsy…
1) checkout ^/framework/tags/tiles-parent-3.0.1/
2) svn merge -rXXX ^/framework/trunk/src/site/apt/tutorial/advancedwildcard.apt
3) mvn site:site
4) cp target/site/tutorial/advanced/wildcard.html ../site/staging/framework/tutorial/advanced/wildcard.html
4a) hack the snapshot header back to the release version
5) svn ci ../site/staging/framework/tutorial/advanced/wildcard.html
6) cast vote

~mck

-- 
"Although the Buddhists will tell you that desire is the root of
suffering, my personal experience leads me to point the finger at system
administration." Philip Greenspun 

| http://github.com/finn-no | http://tech.finn.no |

Re: svnpubsub: migration plan

[Nicolas LE BAS <ma...@nlebas.net>]

On 12-12-26 04:50 AM, Mick Semb Wever wrote:
> 
>> Please share your thoughts on this proposition.
>> If nobody commented before Dec 25, I will then proceed with the above
>> plan in order to meet the deadline of Dec 31.
>>
>> Thank you all for your attention. 
> 
> In a the middle of xmas session i was a little slow on this.
> Dec 25 has past and everything you're written looks excellent!
> 
> Some questions…
> 
> Will it be and at what address would the staging website be visible at? 

None, unless you checkout site/staging to some web server at localhost.
Is that a problem?

> Will site/publish/ and site/staging/ always include all 4
> parts (tiles-site, tiles-autotag, tiles-request and tiles-parent)? 

Site/publish and site/staging would contain what is currently published
at tiles.apache.org.

> The idea is that any release of any of the projects will commit to
> site/staging/ ?

mvn site-deploy would do exactly that.

> Will this mean that updates to tiles.apache.org can only be done via
> releases?

Manual commits are also possible (for archiving old documentation during
a major release for instance, svn mv framework 2.2/framework).

maven-scm-publish-plugin works by checking out the current site into a
local directory, copying the new files into that location, and
committing again. It doesn't remove anything as far as I can tell.

Re: svnpubsub: migration plan

[Mick Semb Wever <mc...@apache.org>]

> Please share your thoughts on this proposition.
> If nobody commented before Dec 25, I will then proceed with the above
> plan in order to meet the deadline of Dec 31.
> 
> Thank you all for your attention. 

In a the middle of xmas session i was a little slow on this.
Dec 25 has past and everything you're written looks excellent!

Some questions…

Will it be and at what address would the staging website be visible at? 
 
Will site/publish/ and site/staging/ always include all 4
parts (tiles-site, tiles-autotag, tiles-request and tiles-parent)? 
The idea is that any release of any of the projects will commit to
site/staging/ ?

Will this mean that updates to tiles.apache.org can only be done via
releases? Might there not be documentation changes we want to push to
staging without the need of making a release? (this applies most to
site/src/site/).


~mck

-- 
“An invasion of armies can be resisted, but not an idea whose time has
come.” - Victor Hugo 

| http://github.com/finn-no | http://tech.finn.no |