[RT] Managing our website in SVN, for improved staging and faster updates

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

ASF Infra is experimenting with a (much) faster way [1] of replicating
website content to the ASF's live websites, provided said content is
committed to SVN. They're looking for volunteer projects to test that
(aka guinea pigs ;-)

Currently, the part of our website that's generated from Confluence
[2] is not committed to SVN, but rather goes directly to the website,
as described in appendix below.

I suggest changing the process slightly, to update content coming from
SLINGxSITE the scenario would be:

1. Edit in Confluence
2. Wait for auto export, or trigger it
3. cd <my sling svn sandbox>/site/site (new folder at  [3])
3. rsync  -rcv me@people.apache.org:/www/confluence-exports/SLINGxSITE .
4. review and commit
5. SvnWcSub/SvnPubSub picks up changes syncs to live site (typically
within a minute)

At step 5 we could also setup a staging site easily, using an SVN
branch, if we want.

The advantages would be better overview of what's in the site and how
things are generated, along with much faster updates of the live site
- whoever edits the docs can immediately check that they're online.

This would also allow us to play more with generated content, like for
example creating tag-based navigation pages by running a script or app
on the content before step 4.

And maybe also easily include individual bundle's documentations
(generated with mvn site/APT?) under paths like
/bundles/somebundle/V1.2.34 to have per-bundle versioned docs.

Last but not least, we'd have all the history of our website content in SVN.

The only downside is a slightly more "manual" update process, as
described above, but that's not too painful IMHO, and we can create a
script for that if needed.

What do people think?

-Bertrand

[1] http://mail-archives.apache.org/mod_mbox/www-infrastructure-dev/200909.mbox/%3C4239a4320909030318s4fe3b877md4b094a6bbbb49a7@mail.gmail.com%3E

[2] http://cwiki.apache.org/confluence/display/SLINGxSITE/Apache+Sling

[3] http://svn.apache.org/repos/asf/sling/site/

Current sling.apache.org update process:
1. Confluence auto-export writes to /www/confluence-exports/SLINGxSITE
on people.apache.org. This happens quickly, within 1-2 minutes
usually.

2. A command from user fmeschbe's crontab copies that to
/www/sling.apache.org/site

3. An ASF script rsyncs that (every few hours IIRC) to the live website

Re: [RT] Managing our website in SVN, for improved staging and faster updates

[Felix Meschberger <fm...@gmail.com>]

Hi,

Bertrand Delacretaz schrieb:
> ASF Infra is experimenting with a (much) faster way [1] of replicating
> website content to the ASF's live websites, provided said content is
> committed to SVN. They're looking for volunteer projects to test that
> (aka guinea pigs ;-)

Thanks for bringing it up here !

I am all for Sling to play a guinea pig role here. Especially for the
parts of the site prepared from SVN (everything except site/, which is
svn:ignored in http://svn.apache.org/repos/asf/sling/site).

> Currently, the part of our website that's generated from Confluence
> [2] is not committed to SVN, but rather goes directly to the website,
> as described in appendix below.
> 
> I suggest changing the process slightly, to update content coming from
> SLINGxSITE the scenario would be:
> 
> 1. Edit in Confluence
> 2. Wait for auto export, or trigger it
> 3. cd <my sling svn sandbox>/site/site (new folder at  [3])
> 3. rsync  -rcv me@people.apache.org:/www/confluence-exports/SLINGxSITE .
> 4. review and commit

This is what worries me a lot since this involves a manual step, which
we do not currently have: The current setup is fully automatic, albeit
with a two hour delay (which also has its advantages)

The main problem we have today is not proof reading or so (you already
can do this in Confluence or you can look at the export exactly as the
page will be rendered from sling.a.o/site). The problem is the delay and
the script running from my crontab.

In essence we have the following situation

  * today's setup
      + fully automatic
      - some delay (up to 2-3 hours)
      - requirs script run from my crontab

  * proposed setup
      - fully manual
      + less delay

At the end of the day, I valuate the disadvantage of manual update much
higher than the delay. Just consider having to switch context for each
and every small update you make to the confluence site.

Granted: the mvn/apt approach has a similar problem, which is why we
started with the Confluence based site in the first place ... it is
really much simpler to edit/publish content.


So I would like to modify your proposal as follows:

  * Use the new SvnWcSub/SvnPubSub mechanism for the svn based part
    of the site (currently JavaDoc and OBR)

  * Look for a better solution for the Confluence based part of
    the site --- granted not to require manual intervention.

> 5. SvnWcSub/SvnPubSub picks up changes syncs to live site (typically
> within a minute)
> 
> At step 5 we could also setup a staging site easily, using an SVN
> branch, if we want.
> 
> The advantages would be better overview of what's in the site and how
> things are generated, along with much faster updates of the live site
> - whoever edits the docs can immediately check that they're online.

Agreed, that this should be the ultimate goal, but doing the manual SVN
commit step sounds more like brute force.

> 
> This would also allow us to play more with generated content, like for
> example creating tag-based navigation pages by running a script or app
> on the content before step 4.

We already can do that today: the "staging" area on people.a.o is setup
with content from SVN where the site folder is svn:ignored. So whatever
we put into svn (http://svn.apache.org/repos/asf/sling/site) will be
pushed to sling.a.o.

> 
> And maybe also easily include individual bundle's documentations
> (generated with mvn site/APT?) under paths like
> /bundles/somebundle/V1.2.34 to have per-bundle versioned docs.

As I said, this is already possible (and to some extend foreseen).

> 
> Last but not least, we'd have all the history of our website content in SVN.

Confluence also has a history of its content.

Of course having everything in SVN would be a single point of history,
but IMHO the price to pay for this is too high (manual step required).

> 
> The only downside is a slightly more "manual" update process, as
> described above, but that's not too painful IMHO, and we can create a
> script for that if needed.

This is the killer for me: currently there is no manual intervention
required (except for rather seldom SVN update).

The problem is not, that the manual step is technically painful. It is
painful in that it may be forgotten about and that it requires a context
switch.


Regards
Felix

> 
> What do people think?
> 
> -Bertrand
> 
> [1] http://mail-archives.apache.org/mod_mbox/www-infrastructure-dev/200909.mbox/%3C4239a4320909030318s4fe3b877md4b094a6bbbb49a7@mail.gmail.com%3E
> 
> [2] http://cwiki.apache.org/confluence/display/SLINGxSITE/Apache+Sling
> 
> [3] http://svn.apache.org/repos/asf/sling/site/
> 
> Current sling.apache.org update process:
> 1. Confluence auto-export writes to /www/confluence-exports/SLINGxSITE
> on people.apache.org. This happens quickly, within 1-2 minutes
> usually.
> 
> 2. A command from user fmeschbe's crontab copies that to
> /www/sling.apache.org/site
> 
> 3. An ASF script rsyncs that (every few hours IIRC) to the live website
>