You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@attic.apache.org by sebb <se...@gmail.com> on 2018/04/29 05:50:21 UTC
Separate branch for site (was: Move attic to git,)
On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
> On 28 April 2018 at 12:37, Hervé BOUTEMY <he...@free.fr> wrote:
...
>> In Git, this would naturally be in a separate branch named "asf-site"
How would that work for Attic?
Where would the source files used to generate the site be held?
Re: Separate branch for site (was: Move attic to git,)
Posted by Hervé BOUTEMY <he...@free.fr>.
Le lundi 30 avril 2018, 20:47:47 CEST sebb a écrit :
> On 30 April 2018 at 19:24, Hervé BOUTEMY <he...@free.fr> wrote:
> > Le lundi 30 avril 2018, 12:46:09 CEST sebb a écrit :
> >> On 30 April 2018 at 06:56, Hervé BOUTEMY <he...@free.fr> wrote:
> >> > Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit :
> >> >> On 29 April 2018 at 22:11, Hervé BOUTEMY <he...@free.fr>
wrote:
> >> >> > better than a discussion: a demo
> >> >> >
> >> >> > here is the website:
> >> >> > https://asf-attic.github.io/
> >> >> >
> >> >> > its source:
> >> >> > https://github.com/asf-attic/asf-attic.github.io/tree/source
> >> >> > its output:
> >> >> > https://github.com/asf-attic/asf-attic.github.io/tree/master
> >> >> >
> >> >> > the only thing that is not included here is the code for the CI to
> >> >> > checkout
> >> >> > the output branch and update content after rebuild
> >> >>
> >> >> All I see is two disjoint branches.
> >> >
> >> > yes, that's what it's about
> >> >
> >> >> It's not obvious how a user is supposed to update the master from the
> >> >> source.
> >> >
> >> > people who know GitHub pages know: that's now part of common culture
> >> > for
> >> > many people
> >>
> >> So how do people update the site?
> >>
> >> Is it enough to edit the source branch and commit the change?
> >
> > yes, once a CI job is configured (like you do with Buildbot, and I do
> > usually with Jenkins + Maven scm-publish)
> >
> >> > and from an operational point of view, that's why in general there is a
> >> > CI
> >> > server doing the official build from source branch then committing
> >> > output
> >> > to output branch
> >>
> >> Is that part of Github's offering?
> >
> > if you use the GitHub Jekyll, yes, the build is magic
> > if you want your own build engine, it's up to you to do the CI setup
> >
> >> >> The build.sh script can be used to create a docs/ tree, but then what?
> >> >
> >> > when you build locally, it's just for yourself, because you want to
> >> > check
> >> > a
> >> > change that is not as trivial as usual
> >> > then the change in output branch is usually done by a CI server
> >> >
> >> > When used with Maven, there is scm-pulbish plugin for that: it's a
> >> > common
> >> > convention, and that's what is used by Apache Cayenne for example.
> >> >
> >> > When used with Jekyll or anything else as rendering engine, I don't
> >> > know
> >> > how people script the output commit: Apache Freemarker just tells in
> >> > their documentation "To publish the built site, commit the output into
> >> > the "asf- site" branch". And Apache Accumulo writes a serie of shell
> >> > commands that they tell are launched by a Git hook (and not a CI
> >> > server).
> >>
> >> AFAICT that only commits the change to the asf-site branch; it does
> >> not push the updated branch.
> >
> > perhaps instructions are not fully written: in general, a CI job does the
> > work>
> >> > And again, in general, there is a central official setup does the job
> >> > in a
> >> > central and official way, to avoid subtle differences when rendering
> >> > from
> >> > multiple personal configurations
> >> >
> >> >> > But what is done here with GitHub GitPubSub equivalent can be done
> >> >> > exactly
> >> >> > the same way at Apache Software Foundation
> >>
> >> I'm not disputing that, but if I am to amend the Buildbot to mimic the
> >> GitHub behaviour I need to know what it does.
> >
> > sure :)
> > it's about doing a git commit + push to the output branch, like it did
> > previously the svn commit to svnpubsub location
> >
> > the demo on GitHub is just here to see it in a very concrete way, since
> > GitHub provides the gitwcsub to publish from the branch to the web server
> Ah, OK.
>
> I don't seem to be able to edit the source, so I cannot try it out.
>
> But if I did, I assume I would see my changes applied to the source
> and then master branches, is that correct?
I did not setup a CI job, then the build won't happen
but if there was a CI job somewhere to build and commit the result, yes, this
would be the process = what we expect to have at Attic once migrated to Git
IIUC
>
> >> >> > Regards,
> >> >> >
> >> >> > Hervé
> >> >> >
> >> >> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit :
> >> >> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit :
> >> >> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr>
> >
> > wrote:
> >> >> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
> >> >> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY
> >> >> >> > >> <he...@free.fr>
> >> >
> >> > wrote:
> >> >> >> > >> > first, I want to reassure everybody: this is a discussion,
> >> >> >> > >> > to
> >> >> >> > >> > get
> >> >> >> > >> > common
> >> >> >> > >> > knowledge of how things work in other projects then may work
> >> >> >> > >> > in
> >> >> >> > >> > the
> >> >> >> > >> > future for Attic if we decide to do an equivalent setup
> >> >> >> > >>
> >> >> >> > >> +1
> >> >> >> > >>
> >> >> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
> >> >> >> > >> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
> >> >> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY
> >> >> >> > >> >> > <he...@free.fr>
> >> >> >>
> >> >> >> wrote:
> >> >> >> > >> >> ...
> >> >> >> > >> >>
> >> >> >> > >> >> >> In Git, this would naturally be in a separate branch
> >> >> >> > >> >> >> named
> >> >> >> > >> >> >> "asf-site"
> >> >> >> > >> >>
> >> >> >> > >> >> How would that work for Attic?
> >> >> >> > >> >>
> >> >> >> > >> >> Where would the source files used to generate the site be
> >> >> >> > >> >> held?
> >> >> >> > >> >
> >> >> >> > >> > There are multiple ways of doing, and GitHub documented it
> >> >> >> > >> > as
> >> >> >> > >> > clearly
> >> >> >> > >> > as
> >> >> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is
> >> >> >> > >> > exactly
> >> >> >> > >> > what
> >> >> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on
> >> >> >> > >> > and
> >> >> >> > >> > technical
> >> >> >> > >> > details on the build solution turned off)
> >> >> >> > >> >
> >> >> >> > >> > The 2 common ways are:
> >> >> >> > >> > 1. publish html from separate branch (which would be by
> >> >> >> > >> > default
> >> >> >> > >> > "asf-site"
> >> >> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a
> >> >> >> > >> > subdirectory
> >> >> >> > >> > on master branch (you see Attic current pattern?)
> >> >> >> > >> >
> >> >> >> > >> > I find the first option a lot more clear from a build+scm
> >> >> >> > >> > perspective
> >> >> >> > >> > than
> >> >> >> > >> > the second one. This will avoid the exact same discussion we
> >> >> >> > >> > have
> >> >> >> > >> > currently at Attic with svn to know who commits the
> >> >> >> > >> > generated
> >> >> >> > >> > content
> >> >> >> > >> > (&
> >> >> >> > >> > when as a consequence): - CI after source-only commit?
> >> >> >> > >> > - or user who builds on his machine then commits
> >> >> >> > >> > simultaneously
> >> >> >> > >> > source
> >> >> >> > >> > and
> >> >> >> > >> > generated content?
> >> >> >> > >> >
> >> >> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look
> >> >> >> > >> > at
> >> >> >> > >> > many
> >> >> >> > >> > ASF
> >> >> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to
> >> >> >> > >> > show a
> >> >> >> > >> > case
> >> >> >> > >> > where:
> >> >> >> > >> > - master branch is a source branch, with markup and a build
> >> >> >> > >> > script
> >> >> >> > >> > - asf-site branch is a completely separate branch that
> >> >> >> > >> > contains
> >> >> >> > >> > generated
> >> >> >> > >> > html It uses Maven scm-publish plugin to update asf-git
> >> >> >> > >> > branch
> >> >> >> > >> > with
> >> >> >> > >> > generated html [6]
> >> >> >> > >> >
> >> >> >> > >> > But there is also Freemarker [4], that has a simple README
> >> >> >> > >> > telling
> >> >> >> > >> > "To
> >> >> >> > >> > publish the built site, commit the output into the
> >> >> >> > >> > "asf-site"
> >> >> >> > >> > branch".
> >> >> >> > >> >
> >> >> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions
> >> >> >> > >> > to
> >> >> >> > >> > publish
> >> >> >> > >> > generated output to asf-site branch with a git-hook that I
> >> >> >> > >> > don't
> >> >> >> > >> > fully
> >> >> >> > >> > understand, but that maybe Attic members will prefer since
> >> >> >> > >> > it
> >> >> >> > >> > seems
> >> >> >> > >> > it's
> >> >> >> > >> > more the common culture here
> >> >> >> > >> >
> >> >> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish
> >> >> >> > >> > plugin
> >> >> >> > >> > used
> >> >> >> > >> > by
> >> >> >> > >> > Cayenne, and I use it in many projects, initially with svn
> >> >> >> > >> > as
> >> >> >> > >> > target
> >> >> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then
> >> >> >> > >> > with
> >> >> >> > >> > git
> >> >> >> > >> > also,
> >> >> >> > >> > when GitHub pages became popular. But I see that it's not
> >> >> >> > >> > the
> >> >> >> > >> > right
> >> >> >> > >> > choice at Attic because it's not the most common Attic
> >> >> >> > >> > culture.
> >> >> >> > >>
> >> >> >> > >> Attic does not produce source code.
> >> >> >> > >> The only output from its SCM is the website.
> >> >> >> > >
> >> >> >> > > yes, like any other website that I showed: source code here is
> >> >> >> > > a
> >> >> >> > > markup
> >> >> >> > > language (be it Markdown, xdoc, static content, or anything
> >> >> >> > > else)
> >> >> >> > > Attic is really exactly the same
> >> >> >> > >
> >> >> >> > > What makes Attic different is that Attic does not have any
> >> >> >> > > other
> >> >> >> > > repo
> >> >> >> > > for
> >> >> >> > > "programming" code: that's true, but does not change anything
> >> >> >> > > regarding
> >> >> >> > > site>
> >> >> >> > >
> >> >> >> > >> AFAICT, all the above examples have a branch which contains
> >> >> >> > >> the
> >> >> >> > >> source
> >> >> >> > >> for building the website.
> >> >> >> > >
> >> >> >> > > yes, I explained I chose them exactly for that reason
> >> >> >> > >
> >> >> >> > > I can show you Airavata site, which is quite simple and did the
> >> >> >> > > other
> >> >> >> > > choice: https://github.com/apache/airavata-site
> >> >> >> > >
> >> >> >> > > source is in source, output is in content in the same branch
> >> >> >> >
> >> >> >> > This is equivalent to Attic.
> >> >> >> >
> >> >> >> > > it could have been: source in master branch, content in
> >> >> >> > > asf-site
> >> >> >> > > branch
> >> >> >> > > which is the most common setup in GitHub pages (= what people
> >> >> >> > > nowadays
> >> >> >> > > know a lot)
> >> >> >> > >
> >> >> >> > >> 1) The source is edited.
> >> >> >> > >> 2) Run the build script to create the output in a clean
> >> >> >> > >> subdirectory
> >> >> >> > >> 3) Copy the subdirectory tree to the asf-site branch
> >> >> >> > >> 4) commit the asf-site branch
> >> >> >> > >> 5) The entire asf-site branch is then published via pubsub.
> >> >> >> > >>
> >> >> >> > >> What Attic does currently is:
> >> >> >> > >> 1) & 2) as above
> >> >> >> > >> 3) commit the changes
> >> >> >> > >> 5) as above
> >> >> >> > >>
> >> >> >> > >> i.e. there is no need to copy the generated output anywhere
> >> >> >> > >> because
> >> >> >> > >> it
> >> >> >> > >> is part of the same repo.
> >> >> >> > >>
> >> >> >> > >> This works because svnpubsub is set up to get its source from
> >> >> >> > >> the
> >> >> >> > >> docs/ subdirectory
> >> >> >> > >
> >> >> >> > > yes, the setup with source and output in the same svn repo or
> >> >> >> > > Git
> >> >> >> > > branch
> >> >> >> > > makes it simple to checkout, but it mixes 2 types of files
> >> >> >> > > (source
> >> >> >> > > and
> >> >> >> > > generated)
> >> >> >> > >
> >> >> >> > > separating source and generated in 2 separate locations
> >> >> >> > > (separate
> >> >> >> > > svn
> >> >> >> > > root
> >> >> >> > > or different branches in the same git repo) makes things more
> >> >> >> > > clear,
> >> >> >> > > at
> >> >> >> > > the cost of an extra step to check out the generated content
> >> >> >> > > then
> >> >> >> > > update
> >> >> >> > > with the updated content
> >> >> >> >
> >> >> >> > The workspace still contains both source and generated output in
> >> >> >> > the
> >> >> >> > examples I have seen.
> >> >> >> >
> >> >> >> > I assume it is ignored by SVN/Git so does not get committed or
> >> >> >> > show
> >> >> >> > up
> >> >> >> > as a local change.
> >> >> >>
> >> >> >> I showed you Cayenne, Freemarker and Accumulo that are not like
> >> >> >> this.
> >> >> >> Here we go back to Freemarker:
> >> >> >> - source: https://github.com/apache/freemarker-site
> >> >> >> - output: https://github.com/apache/freemarker-site/tree/asf-site
> >> >> >>
> >> >> >> > > yes: choose your issue
> >> >> >> > > personally, I prefer the second setup (clear but a little
> >> >> >> > > harder
> >> >> >> > > to
> >> >> >> > > setup)
> >> >> >> > > I don't like having mixed content in one repo (source and
> >> >> >> > > generated
> >> >> >> > > output)
> >> >> >> > >
> >> >> >> > > If everybody understands that these 2 setups a completely
> >> >> >> > > equivalent
> >> >> >> > > but
> >> >> >> > > really prefer the mixed one (just to avoid a second checkout),
> >> >> >> > > I'll
> >> >> >> > > let
> >> >> >> > > you
> >> >> >> > > go: I don't have any problem myself, I make a strong difference
> >> >> >> > > between
> >> >> >> > > source directory and output directory
> >> >> >> >
> >> >> >> > There's still mixed content in local workspaces unless you
> >> >> >> > generate
> >> >> >> > the output in a separate tree.
> >> >> >> >
> >> >> >> > > But if people start to edit output directory instead of source
> >> >> >> > > (like
> >> >> >> > > it
> >> >> >> > > is
> >> >> >> > > so easy to do in the mixed content setup), you're at risk
> >> >> >> >
> >> >> >> > It's also possible to checkin the generated output if it's not
> >> >> >> > properly ignored in a 2 branch version.
> >> >> >> > And then wonder why the site does not get updated.
> >> >> >>
> >> >> >> that's why in general there is a .gitignore or svn:ignore that is
> >> >> >> properly
> >> >> >> configured
> >> >> >>
> >> >> >> > >> I don't know if gitpubsub can take its input from a
> >> >> >> > >> subdirectory
> >> >> >> > >> of
> >> >> >> > >> a
> >> >> >> > >> branch.
> >> >> >> > >
> >> >> >> > > it can: see Airavata site
> >> >> >> >
> >> >> >> > Ah - I see now.
> >> >> >> >
> >> >> >> > The webserver defines the site to be under content/, so the
> >> >> >> > branch
> >> >> >> > can
> >> >> >> > contain other files in parallel directories.
> >> >> >> >
> >> >> >> > >> If not, then we will have to change strategy in order to use
> >> >> >> > >> Git.
> >> >> >> > >> Otherwise, we have a choice.
> >> >> >> > >
> >> >> >> > > we have a choice
> >> >> >> >
> >> >> >> > Yes.
> >> >> >> >
> >> >> >> > I prefer the status quo, not least because it involves fewer
> >> >> >> > changes
> >> >> >> > (I think only renaming docs/ to content/ if we move to Git).
> >> >> >> > Using multiple repos would involve updating instructions as well.
> >> >> >>
> >> >> >> no, it's not multiple repos but multiple branches of the same repo
> >> >> >>
> >> >> >> > But if the majority want to change I won't object.
> >> >> >>
> >> >> >> since you are the guy who does the buidbot script, that does the
> >> >> >> commit,
> >> >> >> you'll have to be confident that you can code the multi-branch
> >> >> >> option
> >> >> >>
> >> >> >> > > Regards,
> >> >> >> > >
> >> >> >> > > Hervé
> >> >> >> > >
> >> >> >> > >> > Regards,
> >> >> >> > >> >
> >> >> >> > >> > Hervé
> >> >> >> > >> >
> >> >> >> > >> >
> >> >> >> > >> > [1]
> >> >> >> > >> > https://help.github.com/articles/configuring-a-publishing-so
> >> >> >> > >> > urc
> >> >> >> > >> > e-f
> >> >> >> > >> > or-> > >> > gi th
> >> >> >> > >> > ub-pages/
> >> >> >> > >> >
> >> >> >> > >> > [2]
> >> >> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deploym
> >> >> >> > >> > ent
> >> >> >> > >> > /mo
> >> >> >> > >> > dul
> >> >> >> > >> > es
> >> >> >> > >> > /g
> >> >> >> > >> > itwcsub/files/config/gitwcsub.cfg
> >> >> >> > >> >
> >> >> >> > >> > [3] https://github.com/apache/cayenne-website/
> >> >> >> > >> >
> >> >> >> > >> > [4] https://github.com/apache/freemarker-site
> >> >> >> > >> >
> >> >> >> > >> > [5] https://github.com/apache/accumulo-website
> >> >> >> > >> >
> >> >> >> > >> > [6]
> >> >> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/va
> >> >> >> > >> > rio
> >> >> >> > >> > us-> >> > >> > tip s.
> >> >> >> > >> > ht
> >> >> >> > >> > ml#Git_branch
Re: Separate branch for site (was: Move attic to git,)
Posted by sebb <se...@gmail.com>.
On 30 April 2018 at 19:24, Hervé BOUTEMY <he...@free.fr> wrote:
> Le lundi 30 avril 2018, 12:46:09 CEST sebb a écrit :
>> On 30 April 2018 at 06:56, Hervé BOUTEMY <he...@free.fr> wrote:
>> > Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit :
>> >> On 29 April 2018 at 22:11, Hervé BOUTEMY <he...@free.fr> wrote:
>> >> > better than a discussion: a demo
>> >> >
>> >> > here is the website:
>> >> > https://asf-attic.github.io/
>> >> >
>> >> > its source:
>> >> > https://github.com/asf-attic/asf-attic.github.io/tree/source
>> >> > its output:
>> >> > https://github.com/asf-attic/asf-attic.github.io/tree/master
>> >> >
>> >> > the only thing that is not included here is the code for the CI to
>> >> > checkout
>> >> > the output branch and update content after rebuild
>> >>
>> >> All I see is two disjoint branches.
>> >
>> > yes, that's what it's about
>> >
>> >> It's not obvious how a user is supposed to update the master from the
>> >> source.
>> >
>> > people who know GitHub pages know: that's now part of common culture for
>> > many people
>>
>> So how do people update the site?
>>
>> Is it enough to edit the source branch and commit the change?
> yes, once a CI job is configured (like you do with Buildbot, and I do usually
> with Jenkins + Maven scm-publish)
>
>>
>> > and from an operational point of view, that's why in general there is a CI
>> > server doing the official build from source branch then committing output
>> > to output branch
>>
>> Is that part of Github's offering?
> if you use the GitHub Jekyll, yes, the build is magic
> if you want your own build engine, it's up to you to do the CI setup
>
>>
>> >> The build.sh script can be used to create a docs/ tree, but then what?
>> >
>> > when you build locally, it's just for yourself, because you want to check
>> > a
>> > change that is not as trivial as usual
>> > then the change in output branch is usually done by a CI server
>> >
>> > When used with Maven, there is scm-pulbish plugin for that: it's a common
>> > convention, and that's what is used by Apache Cayenne for example.
>> >
>> > When used with Jekyll or anything else as rendering engine, I don't know
>> > how people script the output commit: Apache Freemarker just tells in
>> > their documentation "To publish the built site, commit the output into
>> > the "asf- site" branch". And Apache Accumulo writes a serie of shell
>> > commands that they tell are launched by a Git hook (and not a CI server).
>>
>> AFAICT that only commits the change to the asf-site branch; it does
>> not push the updated branch.
> perhaps instructions are not fully written: in general, a CI job does the work
>
>>
>> > And again, in general, there is a central official setup does the job in a
>> > central and official way, to avoid subtle differences when rendering from
>> > multiple personal configurations
>> >
>> >> > But what is done here with GitHub GitPubSub equivalent can be done
>> >> > exactly
>> >> > the same way at Apache Software Foundation
>>
>> I'm not disputing that, but if I am to amend the Buildbot to mimic the
>> GitHub behaviour I need to know what it does.
> sure :)
> it's about doing a git commit + push to the output branch, like it did
> previously the svn commit to svnpubsub location
>
> the demo on GitHub is just here to see it in a very concrete way, since GitHub
> provides the gitwcsub to publish from the branch to the web server
Ah, OK.
I don't seem to be able to edit the source, so I cannot try it out.
But if I did, I assume I would see my changes applied to the source
and then master branches, is that correct?
>>
>> >> > Regards,
>> >> >
>> >> > Hervé
>> >> >
>> >> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit :
>> >> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit :
>> >> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr>
> wrote:
>> >> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
>> >> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr>
>> >
>> > wrote:
>> >> >> > >> > first, I want to reassure everybody: this is a discussion, to
>> >> >> > >> > get
>> >> >> > >> > common
>> >> >> > >> > knowledge of how things work in other projects then may work in
>> >> >> > >> > the
>> >> >> > >> > future for Attic if we decide to do an equivalent setup
>> >> >> > >>
>> >> >> > >> +1
>> >> >> > >>
>> >> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
>> >> >> > >> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
>> >> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY
>> >> >> > >> >> > <he...@free.fr>
>> >> >>
>> >> >> wrote:
>> >> >> > >> >> ...
>> >> >> > >> >>
>> >> >> > >> >> >> In Git, this would naturally be in a separate branch named
>> >> >> > >> >> >> "asf-site"
>> >> >> > >> >>
>> >> >> > >> >> How would that work for Attic?
>> >> >> > >> >>
>> >> >> > >> >> Where would the source files used to generate the site be
>> >> >> > >> >> held?
>> >> >> > >> >
>> >> >> > >> > There are multiple ways of doing, and GitHub documented it as
>> >> >> > >> > clearly
>> >> >> > >> > as
>> >> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly
>> >> >> > >> > what
>> >> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and
>> >> >> > >> > technical
>> >> >> > >> > details on the build solution turned off)
>> >> >> > >> >
>> >> >> > >> > The 2 common ways are:
>> >> >> > >> > 1. publish html from separate branch (which would be by default
>> >> >> > >> > "asf-site"
>> >> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a
>> >> >> > >> > subdirectory
>> >> >> > >> > on master branch (you see Attic current pattern?)
>> >> >> > >> >
>> >> >> > >> > I find the first option a lot more clear from a build+scm
>> >> >> > >> > perspective
>> >> >> > >> > than
>> >> >> > >> > the second one. This will avoid the exact same discussion we
>> >> >> > >> > have
>> >> >> > >> > currently at Attic with svn to know who commits the generated
>> >> >> > >> > content
>> >> >> > >> > (&
>> >> >> > >> > when as a consequence): - CI after source-only commit?
>> >> >> > >> > - or user who builds on his machine then commits simultaneously
>> >> >> > >> > source
>> >> >> > >> > and
>> >> >> > >> > generated content?
>> >> >> > >> >
>> >> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at
>> >> >> > >> > many
>> >> >> > >> > ASF
>> >> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to
>> >> >> > >> > show a
>> >> >> > >> > case
>> >> >> > >> > where:
>> >> >> > >> > - master branch is a source branch, with markup and a build
>> >> >> > >> > script
>> >> >> > >> > - asf-site branch is a completely separate branch that contains
>> >> >> > >> > generated
>> >> >> > >> > html It uses Maven scm-publish plugin to update asf-git branch
>> >> >> > >> > with
>> >> >> > >> > generated html [6]
>> >> >> > >> >
>> >> >> > >> > But there is also Freemarker [4], that has a simple README
>> >> >> > >> > telling
>> >> >> > >> > "To
>> >> >> > >> > publish the built site, commit the output into the "asf-site"
>> >> >> > >> > branch".
>> >> >> > >> >
>> >> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to
>> >> >> > >> > publish
>> >> >> > >> > generated output to asf-site branch with a git-hook that I
>> >> >> > >> > don't
>> >> >> > >> > fully
>> >> >> > >> > understand, but that maybe Attic members will prefer since it
>> >> >> > >> > seems
>> >> >> > >> > it's
>> >> >> > >> > more the common culture here
>> >> >> > >> >
>> >> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish
>> >> >> > >> > plugin
>> >> >> > >> > used
>> >> >> > >> > by
>> >> >> > >> > Cayenne, and I use it in many projects, initially with svn as
>> >> >> > >> > target
>> >> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with
>> >> >> > >> > git
>> >> >> > >> > also,
>> >> >> > >> > when GitHub pages became popular. But I see that it's not the
>> >> >> > >> > right
>> >> >> > >> > choice at Attic because it's not the most common Attic culture.
>> >> >> > >>
>> >> >> > >> Attic does not produce source code.
>> >> >> > >> The only output from its SCM is the website.
>> >> >> > >
>> >> >> > > yes, like any other website that I showed: source code here is a
>> >> >> > > markup
>> >> >> > > language (be it Markdown, xdoc, static content, or anything else)
>> >> >> > > Attic is really exactly the same
>> >> >> > >
>> >> >> > > What makes Attic different is that Attic does not have any other
>> >> >> > > repo
>> >> >> > > for
>> >> >> > > "programming" code: that's true, but does not change anything
>> >> >> > > regarding
>> >> >> > > site>
>> >> >> > >
>> >> >> > >> AFAICT, all the above examples have a branch which contains the
>> >> >> > >> source
>> >> >> > >> for building the website.
>> >> >> > >
>> >> >> > > yes, I explained I chose them exactly for that reason
>> >> >> > >
>> >> >> > > I can show you Airavata site, which is quite simple and did the
>> >> >> > > other
>> >> >> > > choice: https://github.com/apache/airavata-site
>> >> >> > >
>> >> >> > > source is in source, output is in content in the same branch
>> >> >> >
>> >> >> > This is equivalent to Attic.
>> >> >> >
>> >> >> > > it could have been: source in master branch, content in asf-site
>> >> >> > > branch
>> >> >> > > which is the most common setup in GitHub pages (= what people
>> >> >> > > nowadays
>> >> >> > > know a lot)
>> >> >> > >
>> >> >> > >> 1) The source is edited.
>> >> >> > >> 2) Run the build script to create the output in a clean
>> >> >> > >> subdirectory
>> >> >> > >> 3) Copy the subdirectory tree to the asf-site branch
>> >> >> > >> 4) commit the asf-site branch
>> >> >> > >> 5) The entire asf-site branch is then published via pubsub.
>> >> >> > >>
>> >> >> > >> What Attic does currently is:
>> >> >> > >> 1) & 2) as above
>> >> >> > >> 3) commit the changes
>> >> >> > >> 5) as above
>> >> >> > >>
>> >> >> > >> i.e. there is no need to copy the generated output anywhere
>> >> >> > >> because
>> >> >> > >> it
>> >> >> > >> is part of the same repo.
>> >> >> > >>
>> >> >> > >> This works because svnpubsub is set up to get its source from the
>> >> >> > >> docs/ subdirectory
>> >> >> > >
>> >> >> > > yes, the setup with source and output in the same svn repo or Git
>> >> >> > > branch
>> >> >> > > makes it simple to checkout, but it mixes 2 types of files (source
>> >> >> > > and
>> >> >> > > generated)
>> >> >> > >
>> >> >> > > separating source and generated in 2 separate locations (separate
>> >> >> > > svn
>> >> >> > > root
>> >> >> > > or different branches in the same git repo) makes things more
>> >> >> > > clear,
>> >> >> > > at
>> >> >> > > the cost of an extra step to check out the generated content then
>> >> >> > > update
>> >> >> > > with the updated content
>> >> >> >
>> >> >> > The workspace still contains both source and generated output in the
>> >> >> > examples I have seen.
>> >> >> >
>> >> >> > I assume it is ignored by SVN/Git so does not get committed or show
>> >> >> > up
>> >> >> > as a local change.
>> >> >>
>> >> >> I showed you Cayenne, Freemarker and Accumulo that are not like this.
>> >> >> Here we go back to Freemarker:
>> >> >> - source: https://github.com/apache/freemarker-site
>> >> >> - output: https://github.com/apache/freemarker-site/tree/asf-site
>> >> >>
>> >> >> > > yes: choose your issue
>> >> >> > > personally, I prefer the second setup (clear but a little harder
>> >> >> > > to
>> >> >> > > setup)
>> >> >> > > I don't like having mixed content in one repo (source and
>> >> >> > > generated
>> >> >> > > output)
>> >> >> > >
>> >> >> > > If everybody understands that these 2 setups a completely
>> >> >> > > equivalent
>> >> >> > > but
>> >> >> > > really prefer the mixed one (just to avoid a second checkout),
>> >> >> > > I'll
>> >> >> > > let
>> >> >> > > you
>> >> >> > > go: I don't have any problem myself, I make a strong difference
>> >> >> > > between
>> >> >> > > source directory and output directory
>> >> >> >
>> >> >> > There's still mixed content in local workspaces unless you generate
>> >> >> > the output in a separate tree.
>> >> >> >
>> >> >> > > But if people start to edit output directory instead of source
>> >> >> > > (like
>> >> >> > > it
>> >> >> > > is
>> >> >> > > so easy to do in the mixed content setup), you're at risk
>> >> >> >
>> >> >> > It's also possible to checkin the generated output if it's not
>> >> >> > properly ignored in a 2 branch version.
>> >> >> > And then wonder why the site does not get updated.
>> >> >>
>> >> >> that's why in general there is a .gitignore or svn:ignore that is
>> >> >> properly
>> >> >> configured
>> >> >>
>> >> >> > >> I don't know if gitpubsub can take its input from a subdirectory
>> >> >> > >> of
>> >> >> > >> a
>> >> >> > >> branch.
>> >> >> > >
>> >> >> > > it can: see Airavata site
>> >> >> >
>> >> >> > Ah - I see now.
>> >> >> >
>> >> >> > The webserver defines the site to be under content/, so the branch
>> >> >> > can
>> >> >> > contain other files in parallel directories.
>> >> >> >
>> >> >> > >> If not, then we will have to change strategy in order to use Git.
>> >> >> > >> Otherwise, we have a choice.
>> >> >> > >
>> >> >> > > we have a choice
>> >> >> >
>> >> >> > Yes.
>> >> >> >
>> >> >> > I prefer the status quo, not least because it involves fewer changes
>> >> >> > (I think only renaming docs/ to content/ if we move to Git).
>> >> >> > Using multiple repos would involve updating instructions as well.
>> >> >>
>> >> >> no, it's not multiple repos but multiple branches of the same repo
>> >> >>
>> >> >> > But if the majority want to change I won't object.
>> >> >>
>> >> >> since you are the guy who does the buidbot script, that does the
>> >> >> commit,
>> >> >> you'll have to be confident that you can code the multi-branch option
>> >> >>
>> >> >> > > Regards,
>> >> >> > >
>> >> >> > > Hervé
>> >> >> > >
>> >> >> > >> > Regards,
>> >> >> > >> >
>> >> >> > >> > Hervé
>> >> >> > >> >
>> >> >> > >> >
>> >> >> > >> > [1]
>> >> >> > >> > https://help.github.com/articles/configuring-a-publishing-sourc
>> >> >> > >> > e-f
>> >> >> > >> > or-> > >> > gi th
>> >> >> > >> > ub-pages/
>> >> >> > >> >
>> >> >> > >> > [2]
>> >> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment
>> >> >> > >> > /mo
>> >> >> > >> > dul
>> >> >> > >> > es
>> >> >> > >> > /g
>> >> >> > >> > itwcsub/files/config/gitwcsub.cfg
>> >> >> > >> >
>> >> >> > >> > [3] https://github.com/apache/cayenne-website/
>> >> >> > >> >
>> >> >> > >> > [4] https://github.com/apache/freemarker-site
>> >> >> > >> >
>> >> >> > >> > [5] https://github.com/apache/accumulo-website
>> >> >> > >> >
>> >> >> > >> > [6]
>> >> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/vario
>> >> >> > >> > us-> >> > >> > tip s.
>> >> >> > >> > ht
>> >> >> > >> > ml#Git_branch
>
>
Re: Separate branch for site (was: Move attic to git,)
Posted by Hervé BOUTEMY <he...@free.fr>.
Le lundi 30 avril 2018, 12:46:09 CEST sebb a écrit :
> On 30 April 2018 at 06:56, Hervé BOUTEMY <he...@free.fr> wrote:
> > Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit :
> >> On 29 April 2018 at 22:11, Hervé BOUTEMY <he...@free.fr> wrote:
> >> > better than a discussion: a demo
> >> >
> >> > here is the website:
> >> > https://asf-attic.github.io/
> >> >
> >> > its source:
> >> > https://github.com/asf-attic/asf-attic.github.io/tree/source
> >> > its output:
> >> > https://github.com/asf-attic/asf-attic.github.io/tree/master
> >> >
> >> > the only thing that is not included here is the code for the CI to
> >> > checkout
> >> > the output branch and update content after rebuild
> >>
> >> All I see is two disjoint branches.
> >
> > yes, that's what it's about
> >
> >> It's not obvious how a user is supposed to update the master from the
> >> source.
> >
> > people who know GitHub pages know: that's now part of common culture for
> > many people
>
> So how do people update the site?
>
> Is it enough to edit the source branch and commit the change?
yes, once a CI job is configured (like you do with Buildbot, and I do usually
with Jenkins + Maven scm-publish)
>
> > and from an operational point of view, that's why in general there is a CI
> > server doing the official build from source branch then committing output
> > to output branch
>
> Is that part of Github's offering?
if you use the GitHub Jekyll, yes, the build is magic
if you want your own build engine, it's up to you to do the CI setup
>
> >> The build.sh script can be used to create a docs/ tree, but then what?
> >
> > when you build locally, it's just for yourself, because you want to check
> > a
> > change that is not as trivial as usual
> > then the change in output branch is usually done by a CI server
> >
> > When used with Maven, there is scm-pulbish plugin for that: it's a common
> > convention, and that's what is used by Apache Cayenne for example.
> >
> > When used with Jekyll or anything else as rendering engine, I don't know
> > how people script the output commit: Apache Freemarker just tells in
> > their documentation "To publish the built site, commit the output into
> > the "asf- site" branch". And Apache Accumulo writes a serie of shell
> > commands that they tell are launched by a Git hook (and not a CI server).
>
> AFAICT that only commits the change to the asf-site branch; it does
> not push the updated branch.
perhaps instructions are not fully written: in general, a CI job does the work
>
> > And again, in general, there is a central official setup does the job in a
> > central and official way, to avoid subtle differences when rendering from
> > multiple personal configurations
> >
> >> > But what is done here with GitHub GitPubSub equivalent can be done
> >> > exactly
> >> > the same way at Apache Software Foundation
>
> I'm not disputing that, but if I am to amend the Buildbot to mimic the
> GitHub behaviour I need to know what it does.
sure :)
it's about doing a git commit + push to the output branch, like it did
previously the svn commit to svnpubsub location
the demo on GitHub is just here to see it in a very concrete way, since GitHub
provides the gitwcsub to publish from the branch to the web server
>
> >> > Regards,
> >> >
> >> > Hervé
> >> >
> >> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit :
> >> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit :
> >> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr>
wrote:
> >> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
> >> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr>
> >
> > wrote:
> >> >> > >> > first, I want to reassure everybody: this is a discussion, to
> >> >> > >> > get
> >> >> > >> > common
> >> >> > >> > knowledge of how things work in other projects then may work in
> >> >> > >> > the
> >> >> > >> > future for Attic if we decide to do an equivalent setup
> >> >> > >>
> >> >> > >> +1
> >> >> > >>
> >> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
> >> >> > >> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
> >> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY
> >> >> > >> >> > <he...@free.fr>
> >> >>
> >> >> wrote:
> >> >> > >> >> ...
> >> >> > >> >>
> >> >> > >> >> >> In Git, this would naturally be in a separate branch named
> >> >> > >> >> >> "asf-site"
> >> >> > >> >>
> >> >> > >> >> How would that work for Attic?
> >> >> > >> >>
> >> >> > >> >> Where would the source files used to generate the site be
> >> >> > >> >> held?
> >> >> > >> >
> >> >> > >> > There are multiple ways of doing, and GitHub documented it as
> >> >> > >> > clearly
> >> >> > >> > as
> >> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly
> >> >> > >> > what
> >> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and
> >> >> > >> > technical
> >> >> > >> > details on the build solution turned off)
> >> >> > >> >
> >> >> > >> > The 2 common ways are:
> >> >> > >> > 1. publish html from separate branch (which would be by default
> >> >> > >> > "asf-site"
> >> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a
> >> >> > >> > subdirectory
> >> >> > >> > on master branch (you see Attic current pattern?)
> >> >> > >> >
> >> >> > >> > I find the first option a lot more clear from a build+scm
> >> >> > >> > perspective
> >> >> > >> > than
> >> >> > >> > the second one. This will avoid the exact same discussion we
> >> >> > >> > have
> >> >> > >> > currently at Attic with svn to know who commits the generated
> >> >> > >> > content
> >> >> > >> > (&
> >> >> > >> > when as a consequence): - CI after source-only commit?
> >> >> > >> > - or user who builds on his machine then commits simultaneously
> >> >> > >> > source
> >> >> > >> > and
> >> >> > >> > generated content?
> >> >> > >> >
> >> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at
> >> >> > >> > many
> >> >> > >> > ASF
> >> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to
> >> >> > >> > show a
> >> >> > >> > case
> >> >> > >> > where:
> >> >> > >> > - master branch is a source branch, with markup and a build
> >> >> > >> > script
> >> >> > >> > - asf-site branch is a completely separate branch that contains
> >> >> > >> > generated
> >> >> > >> > html It uses Maven scm-publish plugin to update asf-git branch
> >> >> > >> > with
> >> >> > >> > generated html [6]
> >> >> > >> >
> >> >> > >> > But there is also Freemarker [4], that has a simple README
> >> >> > >> > telling
> >> >> > >> > "To
> >> >> > >> > publish the built site, commit the output into the "asf-site"
> >> >> > >> > branch".
> >> >> > >> >
> >> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to
> >> >> > >> > publish
> >> >> > >> > generated output to asf-site branch with a git-hook that I
> >> >> > >> > don't
> >> >> > >> > fully
> >> >> > >> > understand, but that maybe Attic members will prefer since it
> >> >> > >> > seems
> >> >> > >> > it's
> >> >> > >> > more the common culture here
> >> >> > >> >
> >> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish
> >> >> > >> > plugin
> >> >> > >> > used
> >> >> > >> > by
> >> >> > >> > Cayenne, and I use it in many projects, initially with svn as
> >> >> > >> > target
> >> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with
> >> >> > >> > git
> >> >> > >> > also,
> >> >> > >> > when GitHub pages became popular. But I see that it's not the
> >> >> > >> > right
> >> >> > >> > choice at Attic because it's not the most common Attic culture.
> >> >> > >>
> >> >> > >> Attic does not produce source code.
> >> >> > >> The only output from its SCM is the website.
> >> >> > >
> >> >> > > yes, like any other website that I showed: source code here is a
> >> >> > > markup
> >> >> > > language (be it Markdown, xdoc, static content, or anything else)
> >> >> > > Attic is really exactly the same
> >> >> > >
> >> >> > > What makes Attic different is that Attic does not have any other
> >> >> > > repo
> >> >> > > for
> >> >> > > "programming" code: that's true, but does not change anything
> >> >> > > regarding
> >> >> > > site>
> >> >> > >
> >> >> > >> AFAICT, all the above examples have a branch which contains the
> >> >> > >> source
> >> >> > >> for building the website.
> >> >> > >
> >> >> > > yes, I explained I chose them exactly for that reason
> >> >> > >
> >> >> > > I can show you Airavata site, which is quite simple and did the
> >> >> > > other
> >> >> > > choice: https://github.com/apache/airavata-site
> >> >> > >
> >> >> > > source is in source, output is in content in the same branch
> >> >> >
> >> >> > This is equivalent to Attic.
> >> >> >
> >> >> > > it could have been: source in master branch, content in asf-site
> >> >> > > branch
> >> >> > > which is the most common setup in GitHub pages (= what people
> >> >> > > nowadays
> >> >> > > know a lot)
> >> >> > >
> >> >> > >> 1) The source is edited.
> >> >> > >> 2) Run the build script to create the output in a clean
> >> >> > >> subdirectory
> >> >> > >> 3) Copy the subdirectory tree to the asf-site branch
> >> >> > >> 4) commit the asf-site branch
> >> >> > >> 5) The entire asf-site branch is then published via pubsub.
> >> >> > >>
> >> >> > >> What Attic does currently is:
> >> >> > >> 1) & 2) as above
> >> >> > >> 3) commit the changes
> >> >> > >> 5) as above
> >> >> > >>
> >> >> > >> i.e. there is no need to copy the generated output anywhere
> >> >> > >> because
> >> >> > >> it
> >> >> > >> is part of the same repo.
> >> >> > >>
> >> >> > >> This works because svnpubsub is set up to get its source from the
> >> >> > >> docs/ subdirectory
> >> >> > >
> >> >> > > yes, the setup with source and output in the same svn repo or Git
> >> >> > > branch
> >> >> > > makes it simple to checkout, but it mixes 2 types of files (source
> >> >> > > and
> >> >> > > generated)
> >> >> > >
> >> >> > > separating source and generated in 2 separate locations (separate
> >> >> > > svn
> >> >> > > root
> >> >> > > or different branches in the same git repo) makes things more
> >> >> > > clear,
> >> >> > > at
> >> >> > > the cost of an extra step to check out the generated content then
> >> >> > > update
> >> >> > > with the updated content
> >> >> >
> >> >> > The workspace still contains both source and generated output in the
> >> >> > examples I have seen.
> >> >> >
> >> >> > I assume it is ignored by SVN/Git so does not get committed or show
> >> >> > up
> >> >> > as a local change.
> >> >>
> >> >> I showed you Cayenne, Freemarker and Accumulo that are not like this.
> >> >> Here we go back to Freemarker:
> >> >> - source: https://github.com/apache/freemarker-site
> >> >> - output: https://github.com/apache/freemarker-site/tree/asf-site
> >> >>
> >> >> > > yes: choose your issue
> >> >> > > personally, I prefer the second setup (clear but a little harder
> >> >> > > to
> >> >> > > setup)
> >> >> > > I don't like having mixed content in one repo (source and
> >> >> > > generated
> >> >> > > output)
> >> >> > >
> >> >> > > If everybody understands that these 2 setups a completely
> >> >> > > equivalent
> >> >> > > but
> >> >> > > really prefer the mixed one (just to avoid a second checkout),
> >> >> > > I'll
> >> >> > > let
> >> >> > > you
> >> >> > > go: I don't have any problem myself, I make a strong difference
> >> >> > > between
> >> >> > > source directory and output directory
> >> >> >
> >> >> > There's still mixed content in local workspaces unless you generate
> >> >> > the output in a separate tree.
> >> >> >
> >> >> > > But if people start to edit output directory instead of source
> >> >> > > (like
> >> >> > > it
> >> >> > > is
> >> >> > > so easy to do in the mixed content setup), you're at risk
> >> >> >
> >> >> > It's also possible to checkin the generated output if it's not
> >> >> > properly ignored in a 2 branch version.
> >> >> > And then wonder why the site does not get updated.
> >> >>
> >> >> that's why in general there is a .gitignore or svn:ignore that is
> >> >> properly
> >> >> configured
> >> >>
> >> >> > >> I don't know if gitpubsub can take its input from a subdirectory
> >> >> > >> of
> >> >> > >> a
> >> >> > >> branch.
> >> >> > >
> >> >> > > it can: see Airavata site
> >> >> >
> >> >> > Ah - I see now.
> >> >> >
> >> >> > The webserver defines the site to be under content/, so the branch
> >> >> > can
> >> >> > contain other files in parallel directories.
> >> >> >
> >> >> > >> If not, then we will have to change strategy in order to use Git.
> >> >> > >> Otherwise, we have a choice.
> >> >> > >
> >> >> > > we have a choice
> >> >> >
> >> >> > Yes.
> >> >> >
> >> >> > I prefer the status quo, not least because it involves fewer changes
> >> >> > (I think only renaming docs/ to content/ if we move to Git).
> >> >> > Using multiple repos would involve updating instructions as well.
> >> >>
> >> >> no, it's not multiple repos but multiple branches of the same repo
> >> >>
> >> >> > But if the majority want to change I won't object.
> >> >>
> >> >> since you are the guy who does the buidbot script, that does the
> >> >> commit,
> >> >> you'll have to be confident that you can code the multi-branch option
> >> >>
> >> >> > > Regards,
> >> >> > >
> >> >> > > Hervé
> >> >> > >
> >> >> > >> > Regards,
> >> >> > >> >
> >> >> > >> > Hervé
> >> >> > >> >
> >> >> > >> >
> >> >> > >> > [1]
> >> >> > >> > https://help.github.com/articles/configuring-a-publishing-sourc
> >> >> > >> > e-f
> >> >> > >> > or-> > >> > gi th
> >> >> > >> > ub-pages/
> >> >> > >> >
> >> >> > >> > [2]
> >> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment
> >> >> > >> > /mo
> >> >> > >> > dul
> >> >> > >> > es
> >> >> > >> > /g
> >> >> > >> > itwcsub/files/config/gitwcsub.cfg
> >> >> > >> >
> >> >> > >> > [3] https://github.com/apache/cayenne-website/
> >> >> > >> >
> >> >> > >> > [4] https://github.com/apache/freemarker-site
> >> >> > >> >
> >> >> > >> > [5] https://github.com/apache/accumulo-website
> >> >> > >> >
> >> >> > >> > [6]
> >> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/vario
> >> >> > >> > us-> >> > >> > tip s.
> >> >> > >> > ht
> >> >> > >> > ml#Git_branch
Re: Separate branch for site (was: Move attic to git,)
Posted by sebb <se...@gmail.com>.
On 30 April 2018 at 06:56, Hervé BOUTEMY <he...@free.fr> wrote:
> Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit :
>> On 29 April 2018 at 22:11, Hervé BOUTEMY <he...@free.fr> wrote:
>> > better than a discussion: a demo
>> >
>> > here is the website:
>> > https://asf-attic.github.io/
>> >
>> > its source: https://github.com/asf-attic/asf-attic.github.io/tree/source
>> > its output: https://github.com/asf-attic/asf-attic.github.io/tree/master
>> >
>> > the only thing that is not included here is the code for the CI to
>> > checkout
>> > the output branch and update content after rebuild
>>
>> All I see is two disjoint branches.
> yes, that's what it's about
>
>> It's not obvious how a user is supposed to update the master from the
>> source.
> people who know GitHub pages know: that's now part of common culture for many
> people
So how do people update the site?
Is it enough to edit the source branch and commit the change?
> and from an operational point of view, that's why in general there is a CI
> server doing the official build from source branch then committing output to
> output branch
Is that part of Github's offering?
>>
>> The build.sh script can be used to create a docs/ tree, but then what?
> when you build locally, it's just for yourself, because you want to check a
> change that is not as trivial as usual
> then the change in output branch is usually done by a CI server
>
> When used with Maven, there is scm-pulbish plugin for that: it's a common
> convention, and that's what is used by Apache Cayenne for example.
>
> When used with Jekyll or anything else as rendering engine, I don't know how
> people script the output commit: Apache Freemarker just tells in their
> documentation "To publish the built site, commit the output into the "asf-
> site" branch". And Apache Accumulo writes a serie of shell commands that they
> tell are launched by a Git hook (and not a CI server).
AFAICT that only commits the change to the asf-site branch; it does
not push the updated branch.
> And again, in general, there is a central official setup does the job in a
> central and official way, to avoid subtle differences when rendering from
> multiple personal configurations
>
>>
>> > But what is done here with GitHub GitPubSub equivalent can be done exactly
>> > the same way at Apache Software Foundation
I'm not disputing that, but if I am to amend the Buildbot to mimic the
GitHub behaviour I need to know what it does.
>> >
>> > Regards,
>> >
>> > Hervé
>> >
>> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit :
>> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit :
>> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr> wrote:
>> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
>> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr>
> wrote:
>> >> > >> > first, I want to reassure everybody: this is a discussion, to get
>> >> > >> > common
>> >> > >> > knowledge of how things work in other projects then may work in
>> >> > >> > the
>> >> > >> > future for Attic if we decide to do an equivalent setup
>> >> > >>
>> >> > >> +1
>> >> > >>
>> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
>> >> > >> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
>> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY
>> >> > >> >> > <he...@free.fr>
>> >>
>> >> wrote:
>> >> > >> >> ...
>> >> > >> >>
>> >> > >> >> >> In Git, this would naturally be in a separate branch named
>> >> > >> >> >> "asf-site"
>> >> > >> >>
>> >> > >> >> How would that work for Attic?
>> >> > >> >>
>> >> > >> >> Where would the source files used to generate the site be held?
>> >> > >> >
>> >> > >> > There are multiple ways of doing, and GitHub documented it as
>> >> > >> > clearly
>> >> > >> > as
>> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly
>> >> > >> > what
>> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and
>> >> > >> > technical
>> >> > >> > details on the build solution turned off)
>> >> > >> >
>> >> > >> > The 2 common ways are:
>> >> > >> > 1. publish html from separate branch (which would be by default
>> >> > >> > "asf-site"
>> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a
>> >> > >> > subdirectory
>> >> > >> > on master branch (you see Attic current pattern?)
>> >> > >> >
>> >> > >> > I find the first option a lot more clear from a build+scm
>> >> > >> > perspective
>> >> > >> > than
>> >> > >> > the second one. This will avoid the exact same discussion we have
>> >> > >> > currently at Attic with svn to know who commits the generated
>> >> > >> > content
>> >> > >> > (&
>> >> > >> > when as a consequence): - CI after source-only commit?
>> >> > >> > - or user who builds on his machine then commits simultaneously
>> >> > >> > source
>> >> > >> > and
>> >> > >> > generated content?
>> >> > >> >
>> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at
>> >> > >> > many
>> >> > >> > ASF
>> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to show a
>> >> > >> > case
>> >> > >> > where:
>> >> > >> > - master branch is a source branch, with markup and a build script
>> >> > >> > - asf-site branch is a completely separate branch that contains
>> >> > >> > generated
>> >> > >> > html It uses Maven scm-publish plugin to update asf-git branch
>> >> > >> > with
>> >> > >> > generated html [6]
>> >> > >> >
>> >> > >> > But there is also Freemarker [4], that has a simple README telling
>> >> > >> > "To
>> >> > >> > publish the built site, commit the output into the "asf-site"
>> >> > >> > branch".
>> >> > >> >
>> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to
>> >> > >> > publish
>> >> > >> > generated output to asf-site branch with a git-hook that I don't
>> >> > >> > fully
>> >> > >> > understand, but that maybe Attic members will prefer since it
>> >> > >> > seems
>> >> > >> > it's
>> >> > >> > more the common culture here
>> >> > >> >
>> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin
>> >> > >> > used
>> >> > >> > by
>> >> > >> > Cayenne, and I use it in many projects, initially with svn as
>> >> > >> > target
>> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with git
>> >> > >> > also,
>> >> > >> > when GitHub pages became popular. But I see that it's not the
>> >> > >> > right
>> >> > >> > choice at Attic because it's not the most common Attic culture.
>> >> > >>
>> >> > >> Attic does not produce source code.
>> >> > >> The only output from its SCM is the website.
>> >> > >
>> >> > > yes, like any other website that I showed: source code here is a
>> >> > > markup
>> >> > > language (be it Markdown, xdoc, static content, or anything else)
>> >> > > Attic is really exactly the same
>> >> > >
>> >> > > What makes Attic different is that Attic does not have any other repo
>> >> > > for
>> >> > > "programming" code: that's true, but does not change anything
>> >> > > regarding
>> >> > > site>
>> >> > >
>> >> > >> AFAICT, all the above examples have a branch which contains the
>> >> > >> source
>> >> > >> for building the website.
>> >> > >
>> >> > > yes, I explained I chose them exactly for that reason
>> >> > >
>> >> > > I can show you Airavata site, which is quite simple and did the other
>> >> > > choice: https://github.com/apache/airavata-site
>> >> > >
>> >> > > source is in source, output is in content in the same branch
>> >> >
>> >> > This is equivalent to Attic.
>> >> >
>> >> > > it could have been: source in master branch, content in asf-site
>> >> > > branch
>> >> > > which is the most common setup in GitHub pages (= what people
>> >> > > nowadays
>> >> > > know a lot)
>> >> > >
>> >> > >> 1) The source is edited.
>> >> > >> 2) Run the build script to create the output in a clean subdirectory
>> >> > >> 3) Copy the subdirectory tree to the asf-site branch
>> >> > >> 4) commit the asf-site branch
>> >> > >> 5) The entire asf-site branch is then published via pubsub.
>> >> > >>
>> >> > >> What Attic does currently is:
>> >> > >> 1) & 2) as above
>> >> > >> 3) commit the changes
>> >> > >> 5) as above
>> >> > >>
>> >> > >> i.e. there is no need to copy the generated output anywhere because
>> >> > >> it
>> >> > >> is part of the same repo.
>> >> > >>
>> >> > >> This works because svnpubsub is set up to get its source from the
>> >> > >> docs/ subdirectory
>> >> > >
>> >> > > yes, the setup with source and output in the same svn repo or Git
>> >> > > branch
>> >> > > makes it simple to checkout, but it mixes 2 types of files (source
>> >> > > and
>> >> > > generated)
>> >> > >
>> >> > > separating source and generated in 2 separate locations (separate svn
>> >> > > root
>> >> > > or different branches in the same git repo) makes things more clear,
>> >> > > at
>> >> > > the cost of an extra step to check out the generated content then
>> >> > > update
>> >> > > with the updated content
>> >> >
>> >> > The workspace still contains both source and generated output in the
>> >> > examples I have seen.
>> >> >
>> >> > I assume it is ignored by SVN/Git so does not get committed or show up
>> >> > as a local change.
>> >>
>> >> I showed you Cayenne, Freemarker and Accumulo that are not like this.
>> >> Here we go back to Freemarker:
>> >> - source: https://github.com/apache/freemarker-site
>> >> - output: https://github.com/apache/freemarker-site/tree/asf-site
>> >>
>> >> > > yes: choose your issue
>> >> > > personally, I prefer the second setup (clear but a little harder to
>> >> > > setup)
>> >> > > I don't like having mixed content in one repo (source and generated
>> >> > > output)
>> >> > >
>> >> > > If everybody understands that these 2 setups a completely equivalent
>> >> > > but
>> >> > > really prefer the mixed one (just to avoid a second checkout), I'll
>> >> > > let
>> >> > > you
>> >> > > go: I don't have any problem myself, I make a strong difference
>> >> > > between
>> >> > > source directory and output directory
>> >> >
>> >> > There's still mixed content in local workspaces unless you generate
>> >> > the output in a separate tree.
>> >> >
>> >> > > But if people start to edit output directory instead of source (like
>> >> > > it
>> >> > > is
>> >> > > so easy to do in the mixed content setup), you're at risk
>> >> >
>> >> > It's also possible to checkin the generated output if it's not
>> >> > properly ignored in a 2 branch version.
>> >> > And then wonder why the site does not get updated.
>> >>
>> >> that's why in general there is a .gitignore or svn:ignore that is
>> >> properly
>> >> configured
>> >>
>> >> > >> I don't know if gitpubsub can take its input from a subdirectory of
>> >> > >> a
>> >> > >> branch.
>> >> > >
>> >> > > it can: see Airavata site
>> >> >
>> >> > Ah - I see now.
>> >> >
>> >> > The webserver defines the site to be under content/, so the branch can
>> >> > contain other files in parallel directories.
>> >> >
>> >> > >> If not, then we will have to change strategy in order to use Git.
>> >> > >> Otherwise, we have a choice.
>> >> > >
>> >> > > we have a choice
>> >> >
>> >> > Yes.
>> >> >
>> >> > I prefer the status quo, not least because it involves fewer changes
>> >> > (I think only renaming docs/ to content/ if we move to Git).
>> >> > Using multiple repos would involve updating instructions as well.
>> >>
>> >> no, it's not multiple repos but multiple branches of the same repo
>> >>
>> >> > But if the majority want to change I won't object.
>> >>
>> >> since you are the guy who does the buidbot script, that does the commit,
>> >> you'll have to be confident that you can code the multi-branch option
>> >>
>> >> > > Regards,
>> >> > >
>> >> > > Hervé
>> >> > >
>> >> > >> > Regards,
>> >> > >> >
>> >> > >> > Hervé
>> >> > >> >
>> >> > >> >
>> >> > >> > [1]
>> >> > >> > https://help.github.com/articles/configuring-a-publishing-source-f
>> >> > >> > or-> > >> > gi th
>> >> > >> > ub-pages/
>> >> > >> >
>> >> > >> > [2]
>> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment/mo
>> >> > >> > dul
>> >> > >> > es
>> >> > >> > /g
>> >> > >> > itwcsub/files/config/gitwcsub.cfg
>> >> > >> >
>> >> > >> > [3] https://github.com/apache/cayenne-website/
>> >> > >> >
>> >> > >> > [4] https://github.com/apache/freemarker-site
>> >> > >> >
>> >> > >> > [5] https://github.com/apache/accumulo-website
>> >> > >> >
>> >> > >> > [6]
>> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/various-> >> > >> > tip
>> >> > >> > s.
>> >> > >> > ht
>> >> > >> > ml#Git_branch
>
>
Re: Separate branch for site (was: Move attic to git,)
Posted by Hervé BOUTEMY <he...@free.fr>.
Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit :
> On 29 April 2018 at 22:11, Hervé BOUTEMY <he...@free.fr> wrote:
> > better than a discussion: a demo
> >
> > here is the website:
> > https://asf-attic.github.io/
> >
> > its source: https://github.com/asf-attic/asf-attic.github.io/tree/source
> > its output: https://github.com/asf-attic/asf-attic.github.io/tree/master
> >
> > the only thing that is not included here is the code for the CI to
> > checkout
> > the output branch and update content after rebuild
>
> All I see is two disjoint branches.
yes, that's what it's about
> It's not obvious how a user is supposed to update the master from the
> source.
people who know GitHub pages know: that's now part of common culture for many
people
and from an operational point of view, that's why in general there is a CI
server doing the official build from source branch then committing output to
output branch
>
> The build.sh script can be used to create a docs/ tree, but then what?
when you build locally, it's just for yourself, because you want to check a
change that is not as trivial as usual
then the change in output branch is usually done by a CI server
When used with Maven, there is scm-pulbish plugin for that: it's a common
convention, and that's what is used by Apache Cayenne for example.
When used with Jekyll or anything else as rendering engine, I don't know how
people script the output commit: Apache Freemarker just tells in their
documentation "To publish the built site, commit the output into the "asf-
site" branch". And Apache Accumulo writes a serie of shell commands that they
tell are launched by a Git hook (and not a CI server).
And again, in general, there is a central official setup does the job in a
central and official way, to avoid subtle differences when rendering from
multiple personal configurations
>
> > But what is done here with GitHub GitPubSub equivalent can be done exactly
> > the same way at Apache Software Foundation
> >
> > Regards,
> >
> > Hervé
> >
> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit :
> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit :
> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr> wrote:
> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr>
wrote:
> >> > >> > first, I want to reassure everybody: this is a discussion, to get
> >> > >> > common
> >> > >> > knowledge of how things work in other projects then may work in
> >> > >> > the
> >> > >> > future for Attic if we decide to do an equivalent setup
> >> > >>
> >> > >> +1
> >> > >>
> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
> >> > >> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY
> >> > >> >> > <he...@free.fr>
> >>
> >> wrote:
> >> > >> >> ...
> >> > >> >>
> >> > >> >> >> In Git, this would naturally be in a separate branch named
> >> > >> >> >> "asf-site"
> >> > >> >>
> >> > >> >> How would that work for Attic?
> >> > >> >>
> >> > >> >> Where would the source files used to generate the site be held?
> >> > >> >
> >> > >> > There are multiple ways of doing, and GitHub documented it as
> >> > >> > clearly
> >> > >> > as
> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly
> >> > >> > what
> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and
> >> > >> > technical
> >> > >> > details on the build solution turned off)
> >> > >> >
> >> > >> > The 2 common ways are:
> >> > >> > 1. publish html from separate branch (which would be by default
> >> > >> > "asf-site"
> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a
> >> > >> > subdirectory
> >> > >> > on master branch (you see Attic current pattern?)
> >> > >> >
> >> > >> > I find the first option a lot more clear from a build+scm
> >> > >> > perspective
> >> > >> > than
> >> > >> > the second one. This will avoid the exact same discussion we have
> >> > >> > currently at Attic with svn to know who commits the generated
> >> > >> > content
> >> > >> > (&
> >> > >> > when as a consequence): - CI after source-only commit?
> >> > >> > - or user who builds on his machine then commits simultaneously
> >> > >> > source
> >> > >> > and
> >> > >> > generated content?
> >> > >> >
> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at
> >> > >> > many
> >> > >> > ASF
> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to show a
> >> > >> > case
> >> > >> > where:
> >> > >> > - master branch is a source branch, with markup and a build script
> >> > >> > - asf-site branch is a completely separate branch that contains
> >> > >> > generated
> >> > >> > html It uses Maven scm-publish plugin to update asf-git branch
> >> > >> > with
> >> > >> > generated html [6]
> >> > >> >
> >> > >> > But there is also Freemarker [4], that has a simple README telling
> >> > >> > "To
> >> > >> > publish the built site, commit the output into the "asf-site"
> >> > >> > branch".
> >> > >> >
> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to
> >> > >> > publish
> >> > >> > generated output to asf-site branch with a git-hook that I don't
> >> > >> > fully
> >> > >> > understand, but that maybe Attic members will prefer since it
> >> > >> > seems
> >> > >> > it's
> >> > >> > more the common culture here
> >> > >> >
> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin
> >> > >> > used
> >> > >> > by
> >> > >> > Cayenne, and I use it in many projects, initially with svn as
> >> > >> > target
> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with git
> >> > >> > also,
> >> > >> > when GitHub pages became popular. But I see that it's not the
> >> > >> > right
> >> > >> > choice at Attic because it's not the most common Attic culture.
> >> > >>
> >> > >> Attic does not produce source code.
> >> > >> The only output from its SCM is the website.
> >> > >
> >> > > yes, like any other website that I showed: source code here is a
> >> > > markup
> >> > > language (be it Markdown, xdoc, static content, or anything else)
> >> > > Attic is really exactly the same
> >> > >
> >> > > What makes Attic different is that Attic does not have any other repo
> >> > > for
> >> > > "programming" code: that's true, but does not change anything
> >> > > regarding
> >> > > site>
> >> > >
> >> > >> AFAICT, all the above examples have a branch which contains the
> >> > >> source
> >> > >> for building the website.
> >> > >
> >> > > yes, I explained I chose them exactly for that reason
> >> > >
> >> > > I can show you Airavata site, which is quite simple and did the other
> >> > > choice: https://github.com/apache/airavata-site
> >> > >
> >> > > source is in source, output is in content in the same branch
> >> >
> >> > This is equivalent to Attic.
> >> >
> >> > > it could have been: source in master branch, content in asf-site
> >> > > branch
> >> > > which is the most common setup in GitHub pages (= what people
> >> > > nowadays
> >> > > know a lot)
> >> > >
> >> > >> 1) The source is edited.
> >> > >> 2) Run the build script to create the output in a clean subdirectory
> >> > >> 3) Copy the subdirectory tree to the asf-site branch
> >> > >> 4) commit the asf-site branch
> >> > >> 5) The entire asf-site branch is then published via pubsub.
> >> > >>
> >> > >> What Attic does currently is:
> >> > >> 1) & 2) as above
> >> > >> 3) commit the changes
> >> > >> 5) as above
> >> > >>
> >> > >> i.e. there is no need to copy the generated output anywhere because
> >> > >> it
> >> > >> is part of the same repo.
> >> > >>
> >> > >> This works because svnpubsub is set up to get its source from the
> >> > >> docs/ subdirectory
> >> > >
> >> > > yes, the setup with source and output in the same svn repo or Git
> >> > > branch
> >> > > makes it simple to checkout, but it mixes 2 types of files (source
> >> > > and
> >> > > generated)
> >> > >
> >> > > separating source and generated in 2 separate locations (separate svn
> >> > > root
> >> > > or different branches in the same git repo) makes things more clear,
> >> > > at
> >> > > the cost of an extra step to check out the generated content then
> >> > > update
> >> > > with the updated content
> >> >
> >> > The workspace still contains both source and generated output in the
> >> > examples I have seen.
> >> >
> >> > I assume it is ignored by SVN/Git so does not get committed or show up
> >> > as a local change.
> >>
> >> I showed you Cayenne, Freemarker and Accumulo that are not like this.
> >> Here we go back to Freemarker:
> >> - source: https://github.com/apache/freemarker-site
> >> - output: https://github.com/apache/freemarker-site/tree/asf-site
> >>
> >> > > yes: choose your issue
> >> > > personally, I prefer the second setup (clear but a little harder to
> >> > > setup)
> >> > > I don't like having mixed content in one repo (source and generated
> >> > > output)
> >> > >
> >> > > If everybody understands that these 2 setups a completely equivalent
> >> > > but
> >> > > really prefer the mixed one (just to avoid a second checkout), I'll
> >> > > let
> >> > > you
> >> > > go: I don't have any problem myself, I make a strong difference
> >> > > between
> >> > > source directory and output directory
> >> >
> >> > There's still mixed content in local workspaces unless you generate
> >> > the output in a separate tree.
> >> >
> >> > > But if people start to edit output directory instead of source (like
> >> > > it
> >> > > is
> >> > > so easy to do in the mixed content setup), you're at risk
> >> >
> >> > It's also possible to checkin the generated output if it's not
> >> > properly ignored in a 2 branch version.
> >> > And then wonder why the site does not get updated.
> >>
> >> that's why in general there is a .gitignore or svn:ignore that is
> >> properly
> >> configured
> >>
> >> > >> I don't know if gitpubsub can take its input from a subdirectory of
> >> > >> a
> >> > >> branch.
> >> > >
> >> > > it can: see Airavata site
> >> >
> >> > Ah - I see now.
> >> >
> >> > The webserver defines the site to be under content/, so the branch can
> >> > contain other files in parallel directories.
> >> >
> >> > >> If not, then we will have to change strategy in order to use Git.
> >> > >> Otherwise, we have a choice.
> >> > >
> >> > > we have a choice
> >> >
> >> > Yes.
> >> >
> >> > I prefer the status quo, not least because it involves fewer changes
> >> > (I think only renaming docs/ to content/ if we move to Git).
> >> > Using multiple repos would involve updating instructions as well.
> >>
> >> no, it's not multiple repos but multiple branches of the same repo
> >>
> >> > But if the majority want to change I won't object.
> >>
> >> since you are the guy who does the buidbot script, that does the commit,
> >> you'll have to be confident that you can code the multi-branch option
> >>
> >> > > Regards,
> >> > >
> >> > > Hervé
> >> > >
> >> > >> > Regards,
> >> > >> >
> >> > >> > Hervé
> >> > >> >
> >> > >> >
> >> > >> > [1]
> >> > >> > https://help.github.com/articles/configuring-a-publishing-source-f
> >> > >> > or-> > >> > gi th
> >> > >> > ub-pages/
> >> > >> >
> >> > >> > [2]
> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment/mo
> >> > >> > dul
> >> > >> > es
> >> > >> > /g
> >> > >> > itwcsub/files/config/gitwcsub.cfg
> >> > >> >
> >> > >> > [3] https://github.com/apache/cayenne-website/
> >> > >> >
> >> > >> > [4] https://github.com/apache/freemarker-site
> >> > >> >
> >> > >> > [5] https://github.com/apache/accumulo-website
> >> > >> >
> >> > >> > [6]
> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/various-> >> > >> > tip
> >> > >> > s.
> >> > >> > ht
> >> > >> > ml#Git_branch
Re: Separate branch for site (was: Move attic to git,)
Posted by sebb <se...@gmail.com>.
On 29 April 2018 at 22:11, Hervé BOUTEMY <he...@free.fr> wrote:
> better than a discussion: a demo
>
> here is the website:
> https://asf-attic.github.io/
>
> its source: https://github.com/asf-attic/asf-attic.github.io/tree/source
> its output: https://github.com/asf-attic/asf-attic.github.io/tree/master
>
> the only thing that is not included here is the code for the CI to checkout
> the output branch and update content after rebuild
>
All I see is two disjoint branches.
It's not obvious how a user is supposed to update the master from the source.
The build.sh script can be used to create a docs/ tree, but then what?
> But what is done here with GitHub GitPubSub equivalent can be done exactly the
> same way at Apache Software Foundation
>
> Regards,
>
> Hervé
>
> Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit :
>> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit :
>> > On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr> wrote:
>> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
>> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr> wrote:
>> > >> > first, I want to reassure everybody: this is a discussion, to get
>> > >> > common
>> > >> > knowledge of how things work in other projects then may work in the
>> > >> > future for Attic if we decide to do an equivalent setup
>> > >>
>> > >> +1
>> > >>
>> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
>> > >> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
>> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY <he...@free.fr>
>>
>> wrote:
>> > >> >> ...
>> > >> >>
>> > >> >> >> In Git, this would naturally be in a separate branch named
>> > >> >> >> "asf-site"
>> > >> >>
>> > >> >> How would that work for Attic?
>> > >> >>
>> > >> >> Where would the source files used to generate the site be held?
>> > >> >
>> > >> > There are multiple ways of doing, and GitHub documented it as clearly
>> > >> > as
>> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly what
>> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and
>> > >> > technical
>> > >> > details on the build solution turned off)
>> > >> >
>> > >> > The 2 common ways are:
>> > >> > 1. publish html from separate branch (which would be by default
>> > >> > "asf-site"
>> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a
>> > >> > subdirectory
>> > >> > on master branch (you see Attic current pattern?)
>> > >> >
>> > >> > I find the first option a lot more clear from a build+scm perspective
>> > >> > than
>> > >> > the second one. This will avoid the exact same discussion we have
>> > >> > currently at Attic with svn to know who commits the generated content
>> > >> > (&
>> > >> > when as a consequence): - CI after source-only commit?
>> > >> > - or user who builds on his machine then commits simultaneously
>> > >> > source
>> > >> > and
>> > >> > generated content?
>> > >> >
>> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at many
>> > >> > ASF
>> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to show a
>> > >> > case
>> > >> > where:
>> > >> > - master branch is a source branch, with markup and a build script
>> > >> > - asf-site branch is a completely separate branch that contains
>> > >> > generated
>> > >> > html It uses Maven scm-publish plugin to update asf-git branch with
>> > >> > generated html [6]
>> > >> >
>> > >> > But there is also Freemarker [4], that has a simple README telling
>> > >> > "To
>> > >> > publish the built site, commit the output into the "asf-site"
>> > >> > branch".
>> > >> >
>> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to
>> > >> > publish
>> > >> > generated output to asf-site branch with a git-hook that I don't
>> > >> > fully
>> > >> > understand, but that maybe Attic members will prefer since it seems
>> > >> > it's
>> > >> > more the common culture here
>> > >> >
>> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin used
>> > >> > by
>> > >> > Cayenne, and I use it in many projects, initially with svn as target
>> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with git
>> > >> > also,
>> > >> > when GitHub pages became popular. But I see that it's not the right
>> > >> > choice at Attic because it's not the most common Attic culture.
>> > >>
>> > >> Attic does not produce source code.
>> > >> The only output from its SCM is the website.
>> > >
>> > > yes, like any other website that I showed: source code here is a markup
>> > > language (be it Markdown, xdoc, static content, or anything else)
>> > > Attic is really exactly the same
>> > >
>> > > What makes Attic different is that Attic does not have any other repo
>> > > for
>> > > "programming" code: that's true, but does not change anything regarding
>> > > site>
>> > >
>> > >> AFAICT, all the above examples have a branch which contains the source
>> > >> for building the website.
>> > >
>> > > yes, I explained I chose them exactly for that reason
>> > >
>> > > I can show you Airavata site, which is quite simple and did the other
>> > > choice: https://github.com/apache/airavata-site
>> > >
>> > > source is in source, output is in content in the same branch
>> >
>> > This is equivalent to Attic.
>> >
>> > > it could have been: source in master branch, content in asf-site branch
>> > > which is the most common setup in GitHub pages (= what people nowadays
>> > > know a lot)
>> > >
>> > >> 1) The source is edited.
>> > >> 2) Run the build script to create the output in a clean subdirectory
>> > >> 3) Copy the subdirectory tree to the asf-site branch
>> > >> 4) commit the asf-site branch
>> > >> 5) The entire asf-site branch is then published via pubsub.
>> > >>
>> > >> What Attic does currently is:
>> > >> 1) & 2) as above
>> > >> 3) commit the changes
>> > >> 5) as above
>> > >>
>> > >> i.e. there is no need to copy the generated output anywhere because it
>> > >> is part of the same repo.
>> > >>
>> > >> This works because svnpubsub is set up to get its source from the
>> > >> docs/ subdirectory
>> > >
>> > > yes, the setup with source and output in the same svn repo or Git branch
>> > > makes it simple to checkout, but it mixes 2 types of files (source and
>> > > generated)
>> > >
>> > > separating source and generated in 2 separate locations (separate svn
>> > > root
>> > > or different branches in the same git repo) makes things more clear, at
>> > > the cost of an extra step to check out the generated content then update
>> > > with the updated content
>> >
>> > The workspace still contains both source and generated output in the
>> > examples I have seen.
>> >
>> > I assume it is ignored by SVN/Git so does not get committed or show up
>> > as a local change.
>>
>> I showed you Cayenne, Freemarker and Accumulo that are not like this.
>> Here we go back to Freemarker:
>> - source: https://github.com/apache/freemarker-site
>> - output: https://github.com/apache/freemarker-site/tree/asf-site
>>
>> > > yes: choose your issue
>> > > personally, I prefer the second setup (clear but a little harder to
>> > > setup)
>> > > I don't like having mixed content in one repo (source and generated
>> > > output)
>> > >
>> > > If everybody understands that these 2 setups a completely equivalent but
>> > > really prefer the mixed one (just to avoid a second checkout), I'll let
>> > > you
>> > > go: I don't have any problem myself, I make a strong difference between
>> > > source directory and output directory
>> >
>> > There's still mixed content in local workspaces unless you generate
>> > the output in a separate tree.
>> >
>> > > But if people start to edit output directory instead of source (like it
>> > > is
>> > > so easy to do in the mixed content setup), you're at risk
>> >
>> > It's also possible to checkin the generated output if it's not
>> > properly ignored in a 2 branch version.
>> > And then wonder why the site does not get updated.
>>
>> that's why in general there is a .gitignore or svn:ignore that is properly
>> configured
>>
>> > >> I don't know if gitpubsub can take its input from a subdirectory of a
>> > >> branch.
>> > >
>> > > it can: see Airavata site
>> >
>> > Ah - I see now.
>> >
>> > The webserver defines the site to be under content/, so the branch can
>> > contain other files in parallel directories.
>> >
>> > >> If not, then we will have to change strategy in order to use Git.
>> > >> Otherwise, we have a choice.
>> > >
>> > > we have a choice
>> >
>> > Yes.
>> >
>> > I prefer the status quo, not least because it involves fewer changes
>> > (I think only renaming docs/ to content/ if we move to Git).
>> > Using multiple repos would involve updating instructions as well.
>>
>> no, it's not multiple repos but multiple branches of the same repo
>>
>> > But if the majority want to change I won't object.
>>
>> since you are the guy who does the buidbot script, that does the commit,
>> you'll have to be confident that you can code the multi-branch option
>>
>> > > Regards,
>> > >
>> > > Hervé
>> > >
>> > >> > Regards,
>> > >> >
>> > >> > Hervé
>> > >> >
>> > >> >
>> > >> > [1]
>> > >> > https://help.github.com/articles/configuring-a-publishing-source-for-> > >> > gi
>> > >> > th
>> > >> > ub-pages/
>> > >> >
>> > >> > [2]
>> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment/modul
>> > >> > es
>> > >> > /g
>> > >> > itwcsub/files/config/gitwcsub.cfg
>> > >> >
>> > >> > [3] https://github.com/apache/cayenne-website/
>> > >> >
>> > >> > [4] https://github.com/apache/freemarker-site
>> > >> >
>> > >> > [5] https://github.com/apache/accumulo-website
>> > >> >
>> > >> > [6]
>> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tip
>> > >> > s.
>> > >> > ht
>> > >> > ml#Git_branch
>
>
Re: Separate branch for site (was: Move attic to git,)
Posted by Hervé BOUTEMY <he...@free.fr>.
better than a discussion: a demo
here is the website:
https://asf-attic.github.io/
its source: https://github.com/asf-attic/asf-attic.github.io/tree/source
its output: https://github.com/asf-attic/asf-attic.github.io/tree/master
the only thing that is not included here is the code for the CI to checkout
the output branch and update content after rebuild
But what is done here with GitHub GitPubSub equivalent can be done exactly the
same way at Apache Software Foundation
Regards,
Hervé
Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit :
> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit :
> > On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr> wrote:
> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr> wrote:
> > >> > first, I want to reassure everybody: this is a discussion, to get
> > >> > common
> > >> > knowledge of how things work in other projects then may work in the
> > >> > future for Attic if we decide to do an equivalent setup
> > >>
> > >> +1
> > >>
> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
> > >> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY <he...@free.fr>
>
> wrote:
> > >> >> ...
> > >> >>
> > >> >> >> In Git, this would naturally be in a separate branch named
> > >> >> >> "asf-site"
> > >> >>
> > >> >> How would that work for Attic?
> > >> >>
> > >> >> Where would the source files used to generate the site be held?
> > >> >
> > >> > There are multiple ways of doing, and GitHub documented it as clearly
> > >> > as
> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly what
> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and
> > >> > technical
> > >> > details on the build solution turned off)
> > >> >
> > >> > The 2 common ways are:
> > >> > 1. publish html from separate branch (which would be by default
> > >> > "asf-site"
> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a
> > >> > subdirectory
> > >> > on master branch (you see Attic current pattern?)
> > >> >
> > >> > I find the first option a lot more clear from a build+scm perspective
> > >> > than
> > >> > the second one. This will avoid the exact same discussion we have
> > >> > currently at Attic with svn to know who commits the generated content
> > >> > (&
> > >> > when as a consequence): - CI after source-only commit?
> > >> > - or user who builds on his machine then commits simultaneously
> > >> > source
> > >> > and
> > >> > generated content?
> > >> >
> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at many
> > >> > ASF
> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to show a
> > >> > case
> > >> > where:
> > >> > - master branch is a source branch, with markup and a build script
> > >> > - asf-site branch is a completely separate branch that contains
> > >> > generated
> > >> > html It uses Maven scm-publish plugin to update asf-git branch with
> > >> > generated html [6]
> > >> >
> > >> > But there is also Freemarker [4], that has a simple README telling
> > >> > "To
> > >> > publish the built site, commit the output into the "asf-site"
> > >> > branch".
> > >> >
> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to
> > >> > publish
> > >> > generated output to asf-site branch with a git-hook that I don't
> > >> > fully
> > >> > understand, but that maybe Attic members will prefer since it seems
> > >> > it's
> > >> > more the common culture here
> > >> >
> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin used
> > >> > by
> > >> > Cayenne, and I use it in many projects, initially with svn as target
> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with git
> > >> > also,
> > >> > when GitHub pages became popular. But I see that it's not the right
> > >> > choice at Attic because it's not the most common Attic culture.
> > >>
> > >> Attic does not produce source code.
> > >> The only output from its SCM is the website.
> > >
> > > yes, like any other website that I showed: source code here is a markup
> > > language (be it Markdown, xdoc, static content, or anything else)
> > > Attic is really exactly the same
> > >
> > > What makes Attic different is that Attic does not have any other repo
> > > for
> > > "programming" code: that's true, but does not change anything regarding
> > > site>
> > >
> > >> AFAICT, all the above examples have a branch which contains the source
> > >> for building the website.
> > >
> > > yes, I explained I chose them exactly for that reason
> > >
> > > I can show you Airavata site, which is quite simple and did the other
> > > choice: https://github.com/apache/airavata-site
> > >
> > > source is in source, output is in content in the same branch
> >
> > This is equivalent to Attic.
> >
> > > it could have been: source in master branch, content in asf-site branch
> > > which is the most common setup in GitHub pages (= what people nowadays
> > > know a lot)
> > >
> > >> 1) The source is edited.
> > >> 2) Run the build script to create the output in a clean subdirectory
> > >> 3) Copy the subdirectory tree to the asf-site branch
> > >> 4) commit the asf-site branch
> > >> 5) The entire asf-site branch is then published via pubsub.
> > >>
> > >> What Attic does currently is:
> > >> 1) & 2) as above
> > >> 3) commit the changes
> > >> 5) as above
> > >>
> > >> i.e. there is no need to copy the generated output anywhere because it
> > >> is part of the same repo.
> > >>
> > >> This works because svnpubsub is set up to get its source from the
> > >> docs/ subdirectory
> > >
> > > yes, the setup with source and output in the same svn repo or Git branch
> > > makes it simple to checkout, but it mixes 2 types of files (source and
> > > generated)
> > >
> > > separating source and generated in 2 separate locations (separate svn
> > > root
> > > or different branches in the same git repo) makes things more clear, at
> > > the cost of an extra step to check out the generated content then update
> > > with the updated content
> >
> > The workspace still contains both source and generated output in the
> > examples I have seen.
> >
> > I assume it is ignored by SVN/Git so does not get committed or show up
> > as a local change.
>
> I showed you Cayenne, Freemarker and Accumulo that are not like this.
> Here we go back to Freemarker:
> - source: https://github.com/apache/freemarker-site
> - output: https://github.com/apache/freemarker-site/tree/asf-site
>
> > > yes: choose your issue
> > > personally, I prefer the second setup (clear but a little harder to
> > > setup)
> > > I don't like having mixed content in one repo (source and generated
> > > output)
> > >
> > > If everybody understands that these 2 setups a completely equivalent but
> > > really prefer the mixed one (just to avoid a second checkout), I'll let
> > > you
> > > go: I don't have any problem myself, I make a strong difference between
> > > source directory and output directory
> >
> > There's still mixed content in local workspaces unless you generate
> > the output in a separate tree.
> >
> > > But if people start to edit output directory instead of source (like it
> > > is
> > > so easy to do in the mixed content setup), you're at risk
> >
> > It's also possible to checkin the generated output if it's not
> > properly ignored in a 2 branch version.
> > And then wonder why the site does not get updated.
>
> that's why in general there is a .gitignore or svn:ignore that is properly
> configured
>
> > >> I don't know if gitpubsub can take its input from a subdirectory of a
> > >> branch.
> > >
> > > it can: see Airavata site
> >
> > Ah - I see now.
> >
> > The webserver defines the site to be under content/, so the branch can
> > contain other files in parallel directories.
> >
> > >> If not, then we will have to change strategy in order to use Git.
> > >> Otherwise, we have a choice.
> > >
> > > we have a choice
> >
> > Yes.
> >
> > I prefer the status quo, not least because it involves fewer changes
> > (I think only renaming docs/ to content/ if we move to Git).
> > Using multiple repos would involve updating instructions as well.
>
> no, it's not multiple repos but multiple branches of the same repo
>
> > But if the majority want to change I won't object.
>
> since you are the guy who does the buidbot script, that does the commit,
> you'll have to be confident that you can code the multi-branch option
>
> > > Regards,
> > >
> > > Hervé
> > >
> > >> > Regards,
> > >> >
> > >> > Hervé
> > >> >
> > >> >
> > >> > [1]
> > >> > https://help.github.com/articles/configuring-a-publishing-source-for-> > >> > gi
> > >> > th
> > >> > ub-pages/
> > >> >
> > >> > [2]
> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment/modul
> > >> > es
> > >> > /g
> > >> > itwcsub/files/config/gitwcsub.cfg
> > >> >
> > >> > [3] https://github.com/apache/cayenne-website/
> > >> >
> > >> > [4] https://github.com/apache/freemarker-site
> > >> >
> > >> > [5] https://github.com/apache/accumulo-website
> > >> >
> > >> > [6]
> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tip
> > >> > s.
> > >> > ht
> > >> > ml#Git_branch
Re: Separate branch for site (was: Move attic to git,)
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit :
> On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr> wrote:
> > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
> >> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr> wrote:
> >> > first, I want to reassure everybody: this is a discussion, to get
> >> > common
> >> > knowledge of how things work in other projects then may work in the
> >> > future for Attic if we decide to do an equivalent setup
> >>
> >> +1
> >>
> >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
> >> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
> >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY <he...@free.fr>
wrote:
> >> >> ...
> >> >>
> >> >> >> In Git, this would naturally be in a separate branch named
> >> >> >> "asf-site"
> >> >>
> >> >> How would that work for Attic?
> >> >>
> >> >> Where would the source files used to generate the site be held?
> >> >
> >> > There are multiple ways of doing, and GitHub documented it as clearly
> >> > as
> >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly what
> >> > GitHub calls "GitHub Pages", with marketing bells turned on and
> >> > technical
> >> > details on the build solution turned off)
> >> >
> >> > The 2 common ways are:
> >> > 1. publish html from separate branch (which would be by default
> >> > "asf-site"
> >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a
> >> > subdirectory
> >> > on master branch (you see Attic current pattern?)
> >> >
> >> > I find the first option a lot more clear from a build+scm perspective
> >> > than
> >> > the second one. This will avoid the exact same discussion we have
> >> > currently at Attic with svn to know who commits the generated content
> >> > (&
> >> > when as a consequence): - CI after source-only commit?
> >> > - or user who builds on his machine then commits simultaneously source
> >> > and
> >> > generated content?
> >> >
> >> > Then looking at ASF gitwcsub configuration [2], I had a look at many
> >> > ASF
> >> > projects: the 2 ways are used. I picked Cayenne [3] case to show a case
> >> > where:
> >> > - master branch is a source branch, with markup and a build script
> >> > - asf-site branch is a completely separate branch that contains
> >> > generated
> >> > html It uses Maven scm-publish plugin to update asf-git branch with
> >> > generated html [6]
> >> >
> >> > But there is also Freemarker [4], that has a simple README telling "To
> >> > publish the built site, commit the output into the "asf-site" branch".
> >> >
> >> > Or Accumulo [5] which uses Jekyll and has some instructions to publish
> >> > generated output to asf-site branch with a git-hook that I don't fully
> >> > understand, but that maybe Attic members will prefer since it seems
> >> > it's
> >> > more the common culture here
> >> >
> >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin used
> >> > by
> >> > Cayenne, and I use it in many projects, initially with svn as target
> >> > source control (in 2012, for svnpubsub & Apache CMS) then with git
> >> > also,
> >> > when GitHub pages became popular. But I see that it's not the right
> >> > choice at Attic because it's not the most common Attic culture.
> >>
> >> Attic does not produce source code.
> >> The only output from its SCM is the website.
> >
> > yes, like any other website that I showed: source code here is a markup
> > language (be it Markdown, xdoc, static content, or anything else)
> > Attic is really exactly the same
> >
> > What makes Attic different is that Attic does not have any other repo for
> > "programming" code: that's true, but does not change anything regarding
> > site>
> >> AFAICT, all the above examples have a branch which contains the source
> >> for building the website.
> >
> > yes, I explained I chose them exactly for that reason
> >
> > I can show you Airavata site, which is quite simple and did the other
> > choice: https://github.com/apache/airavata-site
> >
> > source is in source, output is in content in the same branch
>
> This is equivalent to Attic.
>
> > it could have been: source in master branch, content in asf-site branch
> > which is the most common setup in GitHub pages (= what people nowadays
> > know a lot)
> >
> >> 1) The source is edited.
> >> 2) Run the build script to create the output in a clean subdirectory
> >> 3) Copy the subdirectory tree to the asf-site branch
> >> 4) commit the asf-site branch
> >> 5) The entire asf-site branch is then published via pubsub.
> >>
> >> What Attic does currently is:
> >> 1) & 2) as above
> >> 3) commit the changes
> >> 5) as above
> >>
> >> i.e. there is no need to copy the generated output anywhere because it
> >> is part of the same repo.
> >>
> >> This works because svnpubsub is set up to get its source from the
> >> docs/ subdirectory
> >
> > yes, the setup with source and output in the same svn repo or Git branch
> > makes it simple to checkout, but it mixes 2 types of files (source and
> > generated)
> >
> > separating source and generated in 2 separate locations (separate svn root
> > or different branches in the same git repo) makes things more clear, at
> > the cost of an extra step to check out the generated content then update
> > with the updated content
>
> The workspace still contains both source and generated output in the
> examples I have seen.
>
> I assume it is ignored by SVN/Git so does not get committed or show up
> as a local change.
I showed you Cayenne, Freemarker and Accumulo that are not like this.
Here we go back to Freemarker:
- source: https://github.com/apache/freemarker-site
- output: https://github.com/apache/freemarker-site/tree/asf-site
>
> > yes: choose your issue
> > personally, I prefer the second setup (clear but a little harder to setup)
> > I don't like having mixed content in one repo (source and generated
> > output)
> >
> > If everybody understands that these 2 setups a completely equivalent but
> > really prefer the mixed one (just to avoid a second checkout), I'll let
> > you
> > go: I don't have any problem myself, I make a strong difference between
> > source directory and output directory
>
> There's still mixed content in local workspaces unless you generate
> the output in a separate tree.
>
> > But if people start to edit output directory instead of source (like it is
> > so easy to do in the mixed content setup), you're at risk
>
> It's also possible to checkin the generated output if it's not
> properly ignored in a 2 branch version.
> And then wonder why the site does not get updated.
that's why in general there is a .gitignore or svn:ignore that is properly
configured
>
> >> I don't know if gitpubsub can take its input from a subdirectory of a
> >> branch.
> >
> > it can: see Airavata site
>
> Ah - I see now.
>
> The webserver defines the site to be under content/, so the branch can
> contain other files in parallel directories.
>
> >> If not, then we will have to change strategy in order to use Git.
> >> Otherwise, we have a choice.
> >
> > we have a choice
>
> Yes.
>
> I prefer the status quo, not least because it involves fewer changes
> (I think only renaming docs/ to content/ if we move to Git).
> Using multiple repos would involve updating instructions as well.
no, it's not multiple repos but multiple branches of the same repo
>
> But if the majority want to change I won't object.
since you are the guy who does the buidbot script, that does the commit,
you'll have to be confident that you can code the multi-branch option
>
> > Regards,
> >
> > Hervé
> >
> >> > Regards,
> >> >
> >> > Hervé
> >> >
> >> >
> >> > [1]
> >> > https://help.github.com/articles/configuring-a-publishing-source-for-gi
> >> > th
> >> > ub-pages/
> >> >
> >> > [2]
> >> > https://github.com/apache/infrastructure-puppet/blob/deployment/modules
> >> > /g
> >> > itwcsub/files/config/gitwcsub.cfg
> >> >
> >> > [3] https://github.com/apache/cayenne-website/
> >> >
> >> > [4] https://github.com/apache/freemarker-site
> >> >
> >> > [5] https://github.com/apache/accumulo-website
> >> >
> >> > [6]
> >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tips.
> >> > ht
> >> > ml#Git_branch
Re: Separate branch for site (was: Move attic to git,)
Posted by sebb <se...@gmail.com>.
On 29 April 2018 at 11:33, Hervé BOUTEMY <he...@free.fr> wrote:
> Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
>> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr> wrote:
>> > first, I want to reassure everybody: this is a discussion, to get common
>> > knowledge of how things work in other projects then may work in the
>> > future for Attic if we decide to do an equivalent setup
>> +1
>>
>> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
>> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
>> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY <he...@free.fr> wrote:
>> >> ...
>> >>
>> >> >> In Git, this would naturally be in a separate branch named "asf-site"
>> >>
>> >> How would that work for Attic?
>> >>
>> >> Where would the source files used to generate the site be held?
>> >
>> > There are multiple ways of doing, and GitHub documented it as clearly as
>> > possible [2] (yes, what we do at ASF with GitPubSub is exactly what
>> > GitHub calls "GitHub Pages", with marketing bells turned on and technical
>> > details on the build solution turned off)
>> >
>> > The 2 common ways are:
>> > 1. publish html from separate branch (which would be by default "asf-site"
>> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a subdirectory
>> > on master branch (you see Attic current pattern?)
>> >
>> > I find the first option a lot more clear from a build+scm perspective than
>> > the second one. This will avoid the exact same discussion we have
>> > currently at Attic with svn to know who commits the generated content (&
>> > when as a consequence): - CI after source-only commit?
>> > - or user who builds on his machine then commits simultaneously source and
>> > generated content?
>> >
>> > Then looking at ASF gitwcsub configuration [2], I had a look at many ASF
>> > projects: the 2 ways are used. I picked Cayenne [3] case to show a case
>> > where:
>> > - master branch is a source branch, with markup and a build script
>> > - asf-site branch is a completely separate branch that contains generated
>> > html It uses Maven scm-publish plugin to update asf-git branch with
>> > generated html [6]
>> >
>> > But there is also Freemarker [4], that has a simple README telling "To
>> > publish the built site, commit the output into the "asf-site" branch".
>> >
>> > Or Accumulo [5] which uses Jekyll and has some instructions to publish
>> > generated output to asf-site branch with a git-hook that I don't fully
>> > understand, but that maybe Attic members will prefer since it seems it's
>> > more the common culture here
>> >
>> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin used by
>> > Cayenne, and I use it in many projects, initially with svn as target
>> > source control (in 2012, for svnpubsub & Apache CMS) then with git also,
>> > when GitHub pages became popular. But I see that it's not the right
>> > choice at Attic because it's not the most common Attic culture.
>> Attic does not produce source code.
>> The only output from its SCM is the website.
> yes, like any other website that I showed: source code here is a markup
> language (be it Markdown, xdoc, static content, or anything else)
> Attic is really exactly the same
>
> What makes Attic different is that Attic does not have any other repo for
> "programming" code: that's true, but does not change anything regarding site
>
>>
>> AFAICT, all the above examples have a branch which contains the source
>> for building the website.
> yes, I explained I chose them exactly for that reason
>
> I can show you Airavata site, which is quite simple and did the other choice:
> https://github.com/apache/airavata-site
>
> source is in source, output is in content in the same branch
This is equivalent to Attic.
> it could have been: source in master branch, content in asf-site branch
> which is the most common setup in GitHub pages (= what people nowadays know a
> lot)
>
>>
>> 1) The source is edited.
>> 2) Run the build script to create the output in a clean subdirectory
>> 3) Copy the subdirectory tree to the asf-site branch
>> 4) commit the asf-site branch
>> 5) The entire asf-site branch is then published via pubsub.
>>
>> What Attic does currently is:
>> 1) & 2) as above
>> 3) commit the changes
>> 5) as above
>>
>> i.e. there is no need to copy the generated output anywhere because it
>> is part of the same repo.
>>
>> This works because svnpubsub is set up to get its source from the
>> docs/ subdirectory
> yes, the setup with source and output in the same svn repo or Git branch makes
> it simple to checkout, but it mixes 2 types of files (source and generated)
>
> separating source and generated in 2 separate locations (separate svn root or
> different branches in the same git repo) makes things more clear, at the cost
> of an extra step to check out the generated content then update with the
> updated content
The workspace still contains both source and generated output in the
examples I have seen.
I assume it is ignored by SVN/Git so does not get committed or show up
as a local change.
> yes: choose your issue
> personally, I prefer the second setup (clear but a little harder to setup)
> I don't like having mixed content in one repo (source and generated output)
>
> If everybody understands that these 2 setups a completely equivalent but
> really prefer the mixed one (just to avoid a second checkout), I'll let you
> go: I don't have any problem myself, I make a strong difference between source
> directory and output directory
There's still mixed content in local workspaces unless you generate
the output in a separate tree.
> But if people start to edit output directory instead of source (like it is so
> easy to do in the mixed content setup), you're at risk
It's also possible to checkin the generated output if it's not
properly ignored in a 2 branch version.
And then wonder why the site does not get updated.
>>
>> I don't know if gitpubsub can take its input from a subdirectory of a
>> branch.
> it can: see Airavata site
Ah - I see now.
The webserver defines the site to be under content/, so the branch can
contain other files in parallel directories.
>> If not, then we will have to change strategy in order to use Git.
>> Otherwise, we have a choice.
> we have a choice
Yes.
I prefer the status quo, not least because it involves fewer changes
(I think only renaming docs/ to content/ if we move to Git).
Using multiple repos would involve updating instructions as well.
But if the majority want to change I won't object.
> Regards,
>
> Hervé
>
>>
>> > Regards,
>> >
>> > Hervé
>> >
>> >
>> > [1]
>> > https://help.github.com/articles/configuring-a-publishing-source-for-gith
>> > ub-pages/
>> >
>> > [2]
>> > https://github.com/apache/infrastructure-puppet/blob/deployment/modules/g
>> > itwcsub/files/config/gitwcsub.cfg
>> >
>> > [3] https://github.com/apache/cayenne-website/
>> >
>> > [4] https://github.com/apache/freemarker-site
>> >
>> > [5] https://github.com/apache/accumulo-website
>> >
>> > [6]
>> > https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tips.ht
>> > ml#Git_branch
>
>
Re: Separate branch for site (was: Move attic to git,)
Posted by Hervé BOUTEMY <he...@free.fr>.
Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit :
> On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr> wrote:
> > first, I want to reassure everybody: this is a discussion, to get common
> > knowledge of how things work in other projects then may work in the
> > future for Attic if we decide to do an equivalent setup
> +1
>
> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
> >> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY <he...@free.fr> wrote:
> >> ...
> >>
> >> >> In Git, this would naturally be in a separate branch named "asf-site"
> >>
> >> How would that work for Attic?
> >>
> >> Where would the source files used to generate the site be held?
> >
> > There are multiple ways of doing, and GitHub documented it as clearly as
> > possible [2] (yes, what we do at ASF with GitPubSub is exactly what
> > GitHub calls "GitHub Pages", with marketing bells turned on and technical
> > details on the build solution turned off)
> >
> > The 2 common ways are:
> > 1. publish html from separate branch (which would be by default "asf-site"
> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a subdirectory
> > on master branch (you see Attic current pattern?)
> >
> > I find the first option a lot more clear from a build+scm perspective than
> > the second one. This will avoid the exact same discussion we have
> > currently at Attic with svn to know who commits the generated content (&
> > when as a consequence): - CI after source-only commit?
> > - or user who builds on his machine then commits simultaneously source and
> > generated content?
> >
> > Then looking at ASF gitwcsub configuration [2], I had a look at many ASF
> > projects: the 2 ways are used. I picked Cayenne [3] case to show a case
> > where:
> > - master branch is a source branch, with markup and a build script
> > - asf-site branch is a completely separate branch that contains generated
> > html It uses Maven scm-publish plugin to update asf-git branch with
> > generated html [6]
> >
> > But there is also Freemarker [4], that has a simple README telling "To
> > publish the built site, commit the output into the "asf-site" branch".
> >
> > Or Accumulo [5] which uses Jekyll and has some instructions to publish
> > generated output to asf-site branch with a git-hook that I don't fully
> > understand, but that maybe Attic members will prefer since it seems it's
> > more the common culture here
> >
> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin used by
> > Cayenne, and I use it in many projects, initially with svn as target
> > source control (in 2012, for svnpubsub & Apache CMS) then with git also,
> > when GitHub pages became popular. But I see that it's not the right
> > choice at Attic because it's not the most common Attic culture.
> Attic does not produce source code.
> The only output from its SCM is the website.
yes, like any other website that I showed: source code here is a markup
language (be it Markdown, xdoc, static content, or anything else)
Attic is really exactly the same
What makes Attic different is that Attic does not have any other repo for
"programming" code: that's true, but does not change anything regarding site
>
> AFAICT, all the above examples have a branch which contains the source
> for building the website.
yes, I explained I chose them exactly for that reason
I can show you Airavata site, which is quite simple and did the other choice:
https://github.com/apache/airavata-site
source is in source, output is in content in the same branch
it could have been: source in master branch, content in asf-site branch
which is the most common setup in GitHub pages (= what people nowadays know a
lot)
>
> 1) The source is edited.
> 2) Run the build script to create the output in a clean subdirectory
> 3) Copy the subdirectory tree to the asf-site branch
> 4) commit the asf-site branch
> 5) The entire asf-site branch is then published via pubsub.
>
> What Attic does currently is:
> 1) & 2) as above
> 3) commit the changes
> 5) as above
>
> i.e. there is no need to copy the generated output anywhere because it
> is part of the same repo.
>
> This works because svnpubsub is set up to get its source from the
> docs/ subdirectory
yes, the setup with source and output in the same svn repo or Git branch makes
it simple to checkout, but it mixes 2 types of files (source and generated)
separating source and generated in 2 separate locations (separate svn root or
different branches in the same git repo) makes things more clear, at the cost
of an extra step to check out the generated content then update with the
updated content
yes: choose your issue
personally, I prefer the second setup (clear but a little harder to setup)
I don't like having mixed content in one repo (source and generated output)
If everybody understands that these 2 setups a completely equivalent but
really prefer the mixed one (just to avoid a second checkout), I'll let you
go: I don't have any problem myself, I make a strong difference between source
directory and output directory
But if people start to edit output directory instead of source (like it is so
easy to do in the mixed content setup), you're at risk
>
> I don't know if gitpubsub can take its input from a subdirectory of a
> branch.
it can: see Airavata site
> If not, then we will have to change strategy in order to use Git.
> Otherwise, we have a choice.
we have a choice
Regards,
Hervé
>
> > Regards,
> >
> > Hervé
> >
> >
> > [1]
> > https://help.github.com/articles/configuring-a-publishing-source-for-gith
> > ub-pages/
> >
> > [2]
> > https://github.com/apache/infrastructure-puppet/blob/deployment/modules/g
> > itwcsub/files/config/gitwcsub.cfg
> >
> > [3] https://github.com/apache/cayenne-website/
> >
> > [4] https://github.com/apache/freemarker-site
> >
> > [5] https://github.com/apache/accumulo-website
> >
> > [6]
> > https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tips.ht
> > ml#Git_branch
Re: Separate branch for site (was: Move attic to git,)
Posted by sebb <se...@gmail.com>.
On 29 April 2018 at 09:41, Hervé BOUTEMY <he...@free.fr> wrote:
> first, I want to reassure everybody: this is a discussion, to get common knowledge of how things work in other projects then may work in the future for Attic if we decide to do an equivalent setup
+1
> Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
>> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
>> > On 28 April 2018 at 12:37, Hervé BOUTEMY <he...@free.fr> wrote:
>> ...
>>
>> >> In Git, this would naturally be in a separate branch named "asf-site"
>>
>> How would that work for Attic?
>>
>> Where would the source files used to generate the site be held?
>
> There are multiple ways of doing, and GitHub documented it as clearly as possible [2]
> (yes, what we do at ASF with GitPubSub is exactly what GitHub calls "GitHub Pages", with marketing bells turned on and technical details on the build solution turned off)
>
> The 2 common ways are:
> 1. publish html from separate branch (which would be by default "asf-site" at ASF, and is "gh-pages" at GitHub)
> 2. publish html from a subdirectory on master branch (you see Attic current pattern?)
>
> I find the first option a lot more clear from a build+scm perspective than the second one.
> This will avoid the exact same discussion we have currently at Attic with svn to know who commits the generated content (& when as a consequence):
> - CI after source-only commit?
> - or user who builds on his machine then commits simultaneously source and generated content?
>
> Then looking at ASF gitwcsub configuration [2], I had a look at many ASF projects: the 2 ways are used.
> I picked Cayenne [3] case to show a case where:
> - master branch is a source branch, with markup and a build script
> - asf-site branch is a completely separate branch that contains generated html
> It uses Maven scm-publish plugin to update asf-git branch with generated html [6]
>
> But there is also Freemarker [4], that has a simple README telling "To publish the built site, commit the output into the "asf-site" branch".
>
> Or Accumulo [5] which uses Jekyll and has some instructions to publish generated output to asf-site branch with a git-hook that I don't fully understand, but that maybe Attic members will prefer since it seems it's more the common culture here
>
> Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin used by Cayenne, and I use it in many projects, initially with svn as target source control (in 2012, for svnpubsub & Apache CMS) then with git also, when GitHub pages became popular.
> But I see that it's not the right choice at Attic because it's not the most common Attic culture.
Attic does not produce source code.
The only output from its SCM is the website.
AFAICT, all the above examples have a branch which contains the source
for building the website.
1) The source is edited.
2) Run the build script to create the output in a clean subdirectory
3) Copy the subdirectory tree to the asf-site branch
4) commit the asf-site branch
5) The entire asf-site branch is then published via pubsub.
What Attic does currently is:
1) & 2) as above
3) commit the changes
5) as above
i.e. there is no need to copy the generated output anywhere because it
is part of the same repo.
This works because svnpubsub is set up to get its source from the
docs/ subdirectory
I don't know if gitpubsub can take its input from a subdirectory of a branch.
If not, then we will have to change strategy in order to use Git.
Otherwise, we have a choice.
> Regards,
>
> Hervé
>
>
> [1] https://help.github.com/articles/configuring-a-publishing-source-for-github-pages/
>
> [2] https://github.com/apache/infrastructure-puppet/blob/deployment/modules/gitwcsub/files/config/gitwcsub.cfg
>
> [3] https://github.com/apache/cayenne-website/
>
> [4] https://github.com/apache/freemarker-site
>
> [5] https://github.com/apache/accumulo-website
>
> [6] https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tips.html#Git_branch
>
Re: Separate branch for site (was: Move attic to git,)
Posted by Hervé BOUTEMY <he...@free.fr>.
first, I want to reassure everybody: this is a discussion, to get common knowledge of how things work in other projects then may work in the future for Attic if we decide to do an equivalent setup
Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit :
> On 28 April 2018 at 12:48, sebb <se...@gmail.com> wrote:
> > On 28 April 2018 at 12:37, Hervé BOUTEMY <he...@free.fr> wrote:
> ...
>
> >> In Git, this would naturally be in a separate branch named "asf-site"
>
> How would that work for Attic?
>
> Where would the source files used to generate the site be held?
There are multiple ways of doing, and GitHub documented it as clearly as possible [2]
(yes, what we do at ASF with GitPubSub is exactly what GitHub calls "GitHub Pages", with marketing bells turned on and technical details on the build solution turned off)
The 2 common ways are:
1. publish html from separate branch (which would be by default "asf-site" at ASF, and is "gh-pages" at GitHub)
2. publish html from a subdirectory on master branch (you see Attic current pattern?)
I find the first option a lot more clear from a build+scm perspective than the second one.
This will avoid the exact same discussion we have currently at Attic with svn to know who commits the generated content (& when as a consequence):
- CI after source-only commit?
- or user who builds on his machine then commits simultaneously source and generated content?
Then looking at ASF gitwcsub configuration [2], I had a look at many ASF projects: the 2 ways are used.
I picked Cayenne [3] case to show a case where:
- master branch is a source branch, with markup and a build script
- asf-site branch is a completely separate branch that contains generated html
It uses Maven scm-publish plugin to update asf-git branch with generated html [6]
But there is also Freemarker [4], that has a simple README telling "To publish the built site, commit the output into the "asf-site" branch".
Or Accumulo [5] which uses Jekyll and has some instructions to publish generated output to asf-site branch with a git-hook that I don't fully understand, but that maybe Attic members will prefer since it seems it's more the common culture here
Notice: I'm a Maven guy, I co-wrote the Maven scm-publish plugin used by Cayenne, and I use it in many projects, initially with svn as target source control (in 2012, for svnpubsub & Apache CMS) then with git also, when GitHub pages became popular.
But I see that it's not the right choice at Attic because it's not the most common Attic culture.
Regards,
Hervé
[1] https://help.github.com/articles/configuring-a-publishing-source-for-github-pages/
[2] https://github.com/apache/infrastructure-puppet/blob/deployment/modules/gitwcsub/files/config/gitwcsub.cfg
[3] https://github.com/apache/cayenne-website/
[4] https://github.com/apache/freemarker-site
[5] https://github.com/apache/accumulo-website
[6] https://maven.apache.org/plugins/maven-scm-publish-plugin/various-tips.html#Git_branch