You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by Alex Heneveld <al...@cloudsoftcorp.com> on 2017/11/01 12:26:28 UTC
Re: New `website` branch in brooklyn-docs
Richard, All-
I think it's wrong to split user guide and other parts of the web site.
I don't see a good reason for it apart from us liking more bifurcations
along technology lines (and these are evolving). I don't see why
sub-dirs aren't appropriate or are difficult. (And was it really
necessary to abandon the subdirs in order to support gitbooks?)
To me, `brooklyn-docs` is the canonical collection of resources that
explain Brooklyn - what it is, how to use it, examples, reference, etc.
The user experiences this through our web site. It is a coherent piece.
The guide is a large chunk of this which has some unique aspects -- it
is built differently (gitbooks, and generates PDF) and we maintain old
versions of it (but not the website) but that is it. To me that's not
sufficient justification for the weight and tedium of a separate project.
Our project divisions should be things that are easily explained to new
people coming to the website. Explaining the difference between "guide"
and "other-things-on-the-site" at this level is a bit obscure and
certainly not important. A division makes things harder for a casual
contributor wanting to fix something (they have to know that difference
and grep in two projects/branches). It also as noted makes it harder for
us to add docs re features if these need to be done in two places.
Thomas says we know how to have multiple PRs but they are still irritating!
Finally: the lifecycle of the two IS pretty much the same as I see it.
There are two classes of changes:
* new features - these only apply the next release, so are not
published - normal workflow is to push to master and not publish
* fixes and non-code enhancements (eg listing new members) - these
apply to the next release and the current version (and in the case of
guide possibly older versions though updating this is less important) -
normal workflow is to push to master branch and last-released branch,
then publish last-released branch
These are the SAME for guide and for non-guide website content. The idea
that we only have one branch for website feels wrong, as it either
discourages non-guide updates for new features, or it risks publishing
these updates before they make it in to a released version. Efforts in
that direction will backfire and make the risks Richard speaks of (wrong
content live on our website) more likely IMO. The one version
difference is that older versions of the guide are included when
publishing whereas older versions of non-guide content is not, but
scripts take care of that, and it's a detail most users and developers
don't need to know about it, so it doesn't seeem a good reason to
introduce a project boundary.
So... if the publish/version lifecycle is the same, and change sets to
both are common, and they make a coherent piece, then focussing on this
dichotomy and splitting up the projects seems unfriendly to doc
consumers and to contributors -- and these should be the primary audience.
Best
Alex
On 31/10/2017 15:19, Richard Downer wrote:
> Hi Alex, Mark, sorry for the late reply.
>
> Alex - you make fair points. It does somewhat obscure the location of the
> website and complicate raising PRs.
>
> I think that a separate repo is probably the best long-term solution, but I
> didn't want to rush into it - creating repos is cheap (Infra can do that
> relatively easily) but deleting repos is nearly impossible for legal policy
> reasons. So I wanted to make sure we were making the right decision for a
> docs/website split before we commit to a separate repo.
>
> I'm not convinced of your argument about website_0.12 etc. IMO the website
> has a lifecycle which is independent of releases and as a general rule
> there should only ever be one website branch. Having multiple copies of the
> website on different branches with content for different purposes is a
> recipe for confusion. Such confusion risks updates on the website appearing
> and disappearing when the site is regenerated different branches, either
> accidentally or when content is not merged to all appropriate branches. If
> we do want to have a holding space for "v.Next" content on the main site,
> then I think there are other ways of handling this.
>
> Could I suggest we remain with the current setup (different branch in docs
> repo) for the short term, until we are absolutely happy with our new
> arrangement, and then switch to a new brooklyn-website repo (or alternative
> if we don't like the new arrangement)? My thoughts are that this would
> happen once we've made a 1.0 release and tested out all the process, or to
> set a deadline if a 1.0 release starts getting pushed out, but I'm happy to
> hear thoughts on this.
>
> Richard.
>
>
> On 27 October 2017 at 10:15, Mark McKenna <m4...@apache.org> wrote:
>
>> Richard , Alex
>>
>> I think its fine on a branch short term (few weeks)
>>
>> I think we should follow the pattern from other projects and split our
>> website into its own repository.
>>
>> Alex I think the technology of the website and the docs should diverge ie
>> we choose the best technology for each.
>>
>> RE PRs ... splitting the site / docs makes the most sense as we deal with
>> making prs across Brooklyn's numerous repositories
>>
>>
>> Mark
>>
>>
>>
>> On 27 October 2017 at 09:01, Alex Heneveld <alex.heneveld@cloudsoftcorp.
>> com>
>> wrote:
>>
>>> Sorry - I really don't like website being hidden on a branch.
>>>
>>> Why?
>>>
>>> * It's non-obvious and one more thing to remember (or forget, or explain)
>>> * Primary reason for branching (in my view) is lifecycle, and here the
>> two
>>> have very similar versioning and release lifecycles: master for both will
>>> be relevant to master of codebase, where eg v0.12.0 will be relevant to
>>> last release of 0.12.0 and what is live; there will be times we want to
>>> update what is live with something more reason relevant to that version,
>>> but we'll do that by pushing to 0.12, and we can release (publish) that
>>> immediately unlike code, but what we _can't_ assume for both is that
>> master
>>> can always be published, because we may have added unreleased features to
>>> docs for both site and guide; with site in a `website` branch we will
>> have
>>> to have branches there eg `website_0.12` alongside `0.12` of guide
>>> * PRs become more difficult, remembering which branch to open them
>> against
>>> and merge against
>>> * PRs become even more difficult if adding a feature which goes onto both
>>> the site and into the docs (which we should do more of), we add to one
>>> branch, switch branches add to the other, then open PRs for both branches
>>> * Ideally they converge to use the same technology (not both gitbook and
>>> markdown) so if in the same project they can share themes and utils etc;
>> if
>>> in different branches we have to cherry-pick between branches (always
>>> opening and reviewing 2 identical PRs!)
>>>
>>> My ideal would be different subdirs but if they really need to be
>> separate
>>> then we could do a different project (resolves reasons 1 2 3 and
>> improves 4
>>> and 5)
>>>
>>> Best
>>> Alex
>>>
>>>
>>>
>>> On 26/10/2017 14:04, Richard Downer wrote:
>>>
>>>> All,
>>>>
>>>> Following on from Thomas's gitbook work[1] - now merged[2] - it means
>> that
>>>> the brooklyn-docs `master` branch does not contain the public website
>>>> (just
>>>> the documentation).
>>>>
>>>> In those threads it was discussed that most branches would remove the
>>>> website, and a separate branch would be used to store the public website
>>>> (a
>>>> bit like how GitHub pages used a `gh-pages` branch for a repository's
>>>> publis website). I'm assuming that the acceptance and merging of
>> gitbooks
>>>> is passive consensus of this concept too.
>>>>
>>>> Therefore, I've created a new branch in the Apache brooklyn-docs repo
>>>> called `website`, which is taken from `master` immediately prior to the
>>>> merge of the gitbook conversion.
>>>>
>>>> If anybody disagrees with this passive consensus approach and thinks
>> that
>>>> this branch is a bad idea, then please feel free to speak up. After all,
>>>> source control means that things can always be undone :-)
>>>>
>>>> Expect a forthcoming PR on the `website` branch to remove the
>>>> documentation
>>>> files and leave just the website files (like the opposite of Thomas's
>> PR).
>>>> Richard.
>>>>
>>>> [1]
>>>> https://lists.apache.org/thread.html/760a3e2fdefaff8d8ac4ea3
>>>> f98a45060fbaa0d886fa57c8f44e4742e@%3Cdev.brooklyn.apache.org%3E
>>>> [2] https://github.com/apache/brooklyn-docs/pull/222
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: <ri...@apache.org>
>>>> Date: 26 October 2017 at 13:50
>>>> Subject: [brooklyn-docs] Git Push Summary
>>>> To: commits@brooklyn.apache.org
>>>>
>>>>
>>>> Repository: brooklyn-docs
>>>> Updated Branches:
>>>> refs/heads/website [created] e1d08e53c
>>>>
>>>>
Re: New `website` branch in brooklyn-docs
Posted by Alex Heneveld <al...@cloudsoftcorp.com>.
Great points.
> "Non-guide updates for new features" appears to be the crux of why we
> disagree (please correct me if I am wrong).
100% agree. Like your positioning "what do we optimize for", and the
question how much of non-guide items are release-specific.
Main non-guide items are:
* learn more - theory
* learn more - features list
* learn more - catalog
* download links
* community resources
* developer guide - IDE setup, code structure, and code standards
The majority of these are things that should update immediately, I'm now
persuaded by you of that and consequently slightly more comfortable with
a project or branch bifurcation.
One remaining question is whether we want to encourage more
release-specific info to go onto the non-guide website -- but I could be
convinced we want to force most such updates into the guide so that
historic versions are available. We can still reference content in the
guide from the main website. "Getting started" for instance does this.
Any changes to that structure would require an embargoed change to
non-guide website but this could be handled with PRs as you suggest.
Final point to note is that a fix to the guide still requires two PRs
against two branches so there is an asymmetry here as opposed to the
website. Still feels simpler to me to have the same consistent process
for guide fixes as for non-guide website fixes, and do the bifurcation
in directories rather than project or branch ... but I don't feel as
strongly as I did and willing to support you & Thomas if I'm a singleton
in my opinion!
Thanks for exploring with me Richard.
Best
Alex
On 01/11/2017 15:45, Richard Downer wrote:
> Hi Alex,
>
> ObDisclaimer: I have an opinion. I am not suggesting that it's a correct
> one :-)
>
>
> On 1 November 2017 at 12:26, Alex Heneveld <al...@cloudsoftcorp.com>
> wrote:
>
>> I think it's wrong to split user guide and other parts of the web site.
>>
>> I don't see a good reason for it apart from us liking more bifurcations
>> along technology lines (and these are evolving). I don't see why sub-dirs
>> aren't appropriate or are difficult. (And was it really necessary to
>> abandon the subdirs in order to support gitbooks?)
>>
> My motivation here is unrelated to the technology. GitBooks does not make
> any impact on this decision.
>
> My motivation here can be simply stated as:
>
> 1. There will be multiple published versions of the documentation
> (potentially multiple active simultaneously, and many historic versions)
> 2. There will only ever be one published website (no matter when URL
> manipulations you do, you will never find a different version of the
> website on our server)
>
> Furthermore:
>
> 3. The documentation is the product of a release that has been made and
> voted on
> 4. The website is not the product of a release
>
> Given these differences, we should not assume that the documentation and
> the website are treated in the same way.
>
>
>> To me, `brooklyn-docs` is the canonical collection of resources that
>> explain Brooklyn - what it is, how to use it, examples, reference, etc.
>> The user experiences this through our web site. It is a coherent piece.
>> The guide is a large chunk of this which has some unique aspects -- it is
>> built differently (gitbooks, and generates PDF) and we maintain old
>> versions of it (but not the website) but that is it.
>
> Yes, the docs and the website are both forms of information, but they have
> different forces acting on them. Consider the OOP "counterexample" of
> Rectangles and Squares[1] - intuitively it makes sense to write `public
> class Square extends Rectangle` until you realise that squares have
> constraints that break the abstraction. Here I don't think it makes sense
> to think of `public class Documentation extends Website` or vice-versa, as
> both have constraints unique to each. I don't think we should be fighting
> those constraints by trying to make a Square into a Rectangle; instead we
> should just accept that they have differences and deal with them
> differently.
>
> There are two classes of changes:
>> * new features - these only apply the next release, so are not published
>
> This is a rare event. To my knowledge we've never had a website update that
> had to be synchronised with a release, other than the obvious one of
> updating links to the downloads. However it *is* a worthwhile thing to be
> able to support. I think that there are other ways to do it, such as
> tagging the PR title.
>
> * fixes and non-code enhancements (eg listing new members) - these apply
>> to the next release and the current version (and in the case of guide
>> possibly older versions though updating this is less important) - normal
>> workflow is to push to master branch and last-released branch, then publish
>> last-released branch
>>
> This is a common event in the website history so far. But under this
> workflow, an immediate change to the website would require either *two*
> PRs, or to push the effort onto the committer to complete the merge between
> the branches. It also requires that the PR submitter *knows* that `master`
> is *not* for immediate updates. The submitter would either need to know
> which is the correct branch to use, or again push that responsibility back
> onto the committer to make sure the PR is merged into the correct places.
>
> Given that website updates generally want to go live immediately -
> potentially because it's a high impact fix, such as a typo or dead link on
> the homepage - it feels like we should optimise for this route, but this
> scenario carries more process weight than the "new features" scenario.
>
> The idea that we only have one branch for website feels wrong, as it either
>> discourages non-guide updates for new features, or it risks publishing
>> these updates before they make it in to a released version.
>
> "Non-guide updates for new features" appears to be the crux of why we
> disagree (please correct me if I am wrong).
>
> The two models are contradictory in this respect:
>
> 1. The seperate-branch-or-repo model discourages non-guide updates for new
> features as you say, but optimises for immediate website updates
> 2. The website-in-a-subdir model optimises for non-guide updates for new
> features, but discourages immediate website updates (branch knowledge,
> multiple PRs and an on-the-ball committer required)
>
> In the past we have had to make immediate website updates, including
> high-impact fixes to errors on the homepage or download page. But to my
> knowledge we have not had non-guide updates with a publication embargo. IMO
> it makes sense to optimise for the more common scenario of immediate
> updates.
>
> I think it's also worth mentioning that our website is actually pretty
> small, and maybe 90% of those pages are not related to features. There's
> very little content that genuinely is dependent on a Brooklyn release. Many
> feature-related content is actually a link to /v/latest/... So even if we
> do want to do more of these kinds of update, it will remain a small number.
>
> This is what I mean by "it optimises for rare conditions at the expense of
> common conditions". In the history of the Brooklyn website, the majority of
> changes are for immediate release. A passer-by looking to make a fix to our
> website would almost certainly be looking to make an immediate release. But
> under this workflow, the common case is the harder one to do.
>
> Efforts in that direction will backfire and make the risks Richard speaks
>> of (wrong content live on our website) more likely IMO.
>
> I disagree. If you have multiple branches containing the website it's more
> likely that:
> - a submitter makes a PR against the wrong branch
> - a committer merges a PR into the wrong branch, or does not merge it into
> all applicable branches
> - a pre/post-release merge between website branches is not done, or is done
> incorrectly
> - a website update is generated from the wrong branch
>
> I've been in this scenario - I've tried to update the Brooklyn website with
> newly-added content, only to run `svn diff` and find that other content has
> disappeared, and I've lost hours if not days trying to keep the published
> website coherent. It's a mess, and a plurality of branches has been
> responsible for website problems.
>
> OTOH, the risk around accidentally publishing an update for a new feature
> too early are small, simply because we generally don't make that content
> :-) And if we do want to make more of those types of updates, we can
> surely come up with a lighter-weight process than having multiple branches.
>
> --
>
> To put the "immediate update required" scenario into perspective, I'd like
> to go back to an example you wrote earlier in this thread:
>
> * Primary reason for branching (in my view) is lifecycle, and here the two
>
> have very similar versioning and release lifecycles: master for both will
>> be relevant to master of codebase, where eg v0.12.0 will be relevant to
>> last release of 0.12.0 and what is live; there will be times we want to
>> update what is live with something more reason relevant to that version,
>> but we'll do that by pushing to 0.12, and we can release (publish) that
>> immediately
>
> My unhappiness here is the statement "v0.12.0 will be relevant to last
> release of 0.12.0 and what is live" - this requires that the *submitter*
> knows that if an immediate website update is required, that he must make
> his PR against branch v0.12.0, and the committer must also merge it to that
> branch, and build the website from that branch. This is putting the onus on
> the *submitter* - and, to a lesser extent, the committer - to know this.
>
> But there's another problem. In the hopefully not-too-distant future, we
> will have more users using Brooklyn in production, and we are more likely
> to hit "immediate bug-fix release required" problems, and we must support
> users on a previous version.
>
> Say we've published versions 1.0.0 and 1.1.0. We have many users in
> production on 1.0.0, but not many on 1.1.0 as it's just been released. Then
> we discover that one of the bugs we fixed in 1.1.0 was actually a security
> vulnerability. Our 1.0.0 production users don't want to move up a whole
> version to get a security vulnerability fixed, so we backport that fix and
> make a new 1.0.1 release.
>
> Q: Where do we document the CVE security advisories that urgently need to
> go on the live website, and any subsequent live website updates? 1.0.1 (the
> version we just released), master (seen by default when you go to GitHub),
> 1.1.0 (the highest-numbered release but not the most recent release), or a
> combination?
>
> Yes we can come up with a policy that explains this, but again it's causing
> ambiguity and putting the onus on the PR submitter to know that policy
> exists and to follow it.
>
> --
>
> In summary: keep it simple. There's only one website. So there shouldn't be
> multiple website branches. It's in `master`, because if someone sees and
> fixes a problem with the website that's where they will submit a PR. No
> branches, no merges, no multiple PRs.
>
> In the (currently rare) event of a website update that needs to be held
> back for the next release, clearly say so in the PR title so that a
> committer doesn't merge it. It's common in Brooklyn to mark a PR as "WIP"
> or "For review only"; non-guide updates for new features could be handled
> in the same way.
>
>
> To repeat my earlier disclaimer, this is just an opinion, and writing it in
> an email does not imply that it is correct! I am always open to listening
> to the alternatives, and I am often persuaded change my mind :-)
>
> Richard.
>
>
> [1] https://en.wikipedia.org/wiki/Circle-ellipse_problem
>
>
>
>>
>>
>> On 31/10/2017 15:19, Richard Downer wrote:
>>
>>> Hi Alex, Mark, sorry for the late reply.
>>>
>>> Alex - you make fair points. It does somewhat obscure the location of the
>>> website and complicate raising PRs.
>>>
>>> I think that a separate repo is probably the best long-term solution, but
>>> I
>>> didn't want to rush into it - creating repos is cheap (Infra can do that
>>> relatively easily) but deleting repos is nearly impossible for legal
>>> policy
>>> reasons. So I wanted to make sure we were making the right decision for a
>>> docs/website split before we commit to a separate repo.
>>>
>>> I'm not convinced of your argument about website_0.12 etc. IMO the website
>>> has a lifecycle which is independent of releases and as a general rule
>>> there should only ever be one website branch. Having multiple copies of
>>> the
>>> website on different branches with content for different purposes is a
>>> recipe for confusion. Such confusion risks updates on the website
>>> appearing
>>> and disappearing when the site is regenerated different branches, either
>>> accidentally or when content is not merged to all appropriate branches. If
>>> we do want to have a holding space for "v.Next" content on the main site,
>>> then I think there are other ways of handling this.
>>>
>>> Could I suggest we remain with the current setup (different branch in docs
>>> repo) for the short term, until we are absolutely happy with our new
>>> arrangement, and then switch to a new brooklyn-website repo (or
>>> alternative
>>> if we don't like the new arrangement)? My thoughts are that this would
>>> happen once we've made a 1.0 release and tested out all the process, or to
>>> set a deadline if a 1.0 release starts getting pushed out, but I'm happy
>>> to
>>> hear thoughts on this.
>>>
>>> Richard.
>>>
>>>
>>> On 27 October 2017 at 10:15, Mark McKenna <m4...@apache.org> wrote:
>>>
>>> Richard , Alex
>>>> I think its fine on a branch short term (few weeks)
>>>>
>>>> I think we should follow the pattern from other projects and split our
>>>> website into its own repository.
>>>>
>>>> Alex I think the technology of the website and the docs should diverge ie
>>>> we choose the best technology for each.
>>>>
>>>> RE PRs ... splitting the site / docs makes the most sense as we deal with
>>>> making prs across Brooklyn's numerous repositories
>>>>
>>>>
>>>> Mark
>>>>
>>>>
>>>>
>>>> On 27 October 2017 at 09:01, Alex Heneveld <alex.heneveld@cloudsoftcorp.
>>>> com>
>>>> wrote:
>>>>
>>>> Sorry - I really don't like website being hidden on a branch.
>>>>> Why?
>>>>>
>>>>> * It's non-obvious and one more thing to remember (or forget, or
>>>>> explain)
>>>>> * Primary reason for branching (in my view) is lifecycle, and here the
>>>>>
>>>> two
>>>>
>>>>> have very similar versioning and release lifecycles: master for both
>>>>> will
>>>>> be relevant to master of codebase, where eg v0.12.0 will be relevant to
>>>>> last release of 0.12.0 and what is live; there will be times we want to
>>>>> update what is live with something more reason relevant to that version,
>>>>> but we'll do that by pushing to 0.12, and we can release (publish) that
>>>>> immediately unlike code, but what we _can't_ assume for both is that
>>>>>
>>>> master
>>>>
>>>>> can always be published, because we may have added unreleased features
>>>>> to
>>>>> docs for both site and guide; with site in a `website` branch we will
>>>>>
>>>> have
>>>>
>>>>> to have branches there eg `website_0.12` alongside `0.12` of guide
>>>>> * PRs become more difficult, remembering which branch to open them
>>>>>
>>>> against
>>>>
>>>>> and merge against
>>>>> * PRs become even more difficult if adding a feature which goes onto
>>>>> both
>>>>> the site and into the docs (which we should do more of), we add to one
>>>>> branch, switch branches add to the other, then open PRs for both
>>>>> branches
>>>>> * Ideally they converge to use the same technology (not both gitbook and
>>>>> markdown) so if in the same project they can share themes and utils etc;
>>>>>
>>>> if
>>>>
>>>>> in different branches we have to cherry-pick between branches (always
>>>>> opening and reviewing 2 identical PRs!)
>>>>>
>>>>> My ideal would be different subdirs but if they really need to be
>>>>>
>>>> separate
>>>>
>>>>> then we could do a different project (resolves reasons 1 2 3 and
>>>>>
>>>> improves 4
>>>>
>>>>> and 5)
>>>>>
>>>>> Best
>>>>> Alex
>>>>>
>>>>>
>>>>>
>>>>> On 26/10/2017 14:04, Richard Downer wrote:
>>>>>
>>>>> All,
>>>>>> Following on from Thomas's gitbook work[1] - now merged[2] - it means
>>>>>>
>>>>> that
>>>>> the brooklyn-docs `master` branch does not contain the public website
>>>>>> (just
>>>>>> the documentation).
>>>>>>
>>>>>> In those threads it was discussed that most branches would remove the
>>>>>> website, and a separate branch would be used to store the public
>>>>>> website
>>>>>> (a
>>>>>> bit like how GitHub pages used a `gh-pages` branch for a repository's
>>>>>> publis website). I'm assuming that the acceptance and merging of
>>>>>>
>>>>> gitbooks
>>>>> is passive consensus of this concept too.
>>>>>> Therefore, I've created a new branch in the Apache brooklyn-docs repo
>>>>>> called `website`, which is taken from `master` immediately prior to the
>>>>>> merge of the gitbook conversion.
>>>>>>
>>>>>> If anybody disagrees with this passive consensus approach and thinks
>>>>>>
>>>>> that
>>>>> this branch is a bad idea, then please feel free to speak up. After all,
>>>>>> source control means that things can always be undone :-)
>>>>>>
>>>>>> Expect a forthcoming PR on the `website` branch to remove the
>>>>>> documentation
>>>>>> files and leave just the website files (like the opposite of Thomas's
>>>>>>
>>>>> PR).
>>>>> Richard.
>>>>>> [1]
>>>>>> https://lists.apache.org/thread.html/760a3e2fdefaff8d8ac4ea3
>>>>>> f98a45060fbaa0d886fa57c8f44e4742e@%3Cdev.brooklyn.apache.org%3E
>>>>>> [2] https://github.com/apache/brooklyn-docs/pull/222
>>>>>>
>>>>>> ---------- Forwarded message ----------
>>>>>> From: <ri...@apache.org>
>>>>>> Date: 26 October 2017 at 13:50
>>>>>> Subject: [brooklyn-docs] Git Push Summary
>>>>>> To: commits@brooklyn.apache.org
>>>>>>
>>>>>>
>>>>>> Repository: brooklyn-docs
>>>>>> Updated Branches:
>>>>>> refs/heads/website [created] e1d08e53c
>>>>>>
>>>>>>
>>>>>>
Re: New `website` branch in brooklyn-docs
Posted by Richard Downer <ri...@apache.org>.
Hi Alex,
ObDisclaimer: I have an opinion. I am not suggesting that it's a correct
one :-)
On 1 November 2017 at 12:26, Alex Heneveld <al...@cloudsoftcorp.com>
wrote:
> I think it's wrong to split user guide and other parts of the web site.
>
> I don't see a good reason for it apart from us liking more bifurcations
> along technology lines (and these are evolving). I don't see why sub-dirs
> aren't appropriate or are difficult. (And was it really necessary to
> abandon the subdirs in order to support gitbooks?)
>
My motivation here is unrelated to the technology. GitBooks does not make
any impact on this decision.
My motivation here can be simply stated as:
1. There will be multiple published versions of the documentation
(potentially multiple active simultaneously, and many historic versions)
2. There will only ever be one published website (no matter when URL
manipulations you do, you will never find a different version of the
website on our server)
Furthermore:
3. The documentation is the product of a release that has been made and
voted on
4. The website is not the product of a release
Given these differences, we should not assume that the documentation and
the website are treated in the same way.
> To me, `brooklyn-docs` is the canonical collection of resources that
> explain Brooklyn - what it is, how to use it, examples, reference, etc.
> The user experiences this through our web site. It is a coherent piece.
> The guide is a large chunk of this which has some unique aspects -- it is
> built differently (gitbooks, and generates PDF) and we maintain old
> versions of it (but not the website) but that is it.
Yes, the docs and the website are both forms of information, but they have
different forces acting on them. Consider the OOP "counterexample" of
Rectangles and Squares[1] - intuitively it makes sense to write `public
class Square extends Rectangle` until you realise that squares have
constraints that break the abstraction. Here I don't think it makes sense
to think of `public class Documentation extends Website` or vice-versa, as
both have constraints unique to each. I don't think we should be fighting
those constraints by trying to make a Square into a Rectangle; instead we
should just accept that they have differences and deal with them
differently.
There are two classes of changes:
>
> * new features - these only apply the next release, so are not published
This is a rare event. To my knowledge we've never had a website update that
had to be synchronised with a release, other than the obvious one of
updating links to the downloads. However it *is* a worthwhile thing to be
able to support. I think that there are other ways to do it, such as
tagging the PR title.
* fixes and non-code enhancements (eg listing new members) - these apply
> to the next release and the current version (and in the case of guide
> possibly older versions though updating this is less important) - normal
> workflow is to push to master branch and last-released branch, then publish
> last-released branch
>
This is a common event in the website history so far. But under this
workflow, an immediate change to the website would require either *two*
PRs, or to push the effort onto the committer to complete the merge between
the branches. It also requires that the PR submitter *knows* that `master`
is *not* for immediate updates. The submitter would either need to know
which is the correct branch to use, or again push that responsibility back
onto the committer to make sure the PR is merged into the correct places.
Given that website updates generally want to go live immediately -
potentially because it's a high impact fix, such as a typo or dead link on
the homepage - it feels like we should optimise for this route, but this
scenario carries more process weight than the "new features" scenario.
The idea that we only have one branch for website feels wrong, as it either
> discourages non-guide updates for new features, or it risks publishing
> these updates before they make it in to a released version.
"Non-guide updates for new features" appears to be the crux of why we
disagree (please correct me if I am wrong).
The two models are contradictory in this respect:
1. The seperate-branch-or-repo model discourages non-guide updates for new
features as you say, but optimises for immediate website updates
2. The website-in-a-subdir model optimises for non-guide updates for new
features, but discourages immediate website updates (branch knowledge,
multiple PRs and an on-the-ball committer required)
In the past we have had to make immediate website updates, including
high-impact fixes to errors on the homepage or download page. But to my
knowledge we have not had non-guide updates with a publication embargo. IMO
it makes sense to optimise for the more common scenario of immediate
updates.
I think it's also worth mentioning that our website is actually pretty
small, and maybe 90% of those pages are not related to features. There's
very little content that genuinely is dependent on a Brooklyn release. Many
feature-related content is actually a link to /v/latest/... So even if we
do want to do more of these kinds of update, it will remain a small number.
This is what I mean by "it optimises for rare conditions at the expense of
common conditions". In the history of the Brooklyn website, the majority of
changes are for immediate release. A passer-by looking to make a fix to our
website would almost certainly be looking to make an immediate release. But
under this workflow, the common case is the harder one to do.
Efforts in that direction will backfire and make the risks Richard speaks
> of (wrong content live on our website) more likely IMO.
I disagree. If you have multiple branches containing the website it's more
likely that:
- a submitter makes a PR against the wrong branch
- a committer merges a PR into the wrong branch, or does not merge it into
all applicable branches
- a pre/post-release merge between website branches is not done, or is done
incorrectly
- a website update is generated from the wrong branch
I've been in this scenario - I've tried to update the Brooklyn website with
newly-added content, only to run `svn diff` and find that other content has
disappeared, and I've lost hours if not days trying to keep the published
website coherent. It's a mess, and a plurality of branches has been
responsible for website problems.
OTOH, the risk around accidentally publishing an update for a new feature
too early are small, simply because we generally don't make that content
:-) And if we do want to make more of those types of updates, we can
surely come up with a lighter-weight process than having multiple branches.
--
To put the "immediate update required" scenario into perspective, I'd like
to go back to an example you wrote earlier in this thread:
* Primary reason for branching (in my view) is lifecycle, and here the two
have very similar versioning and release lifecycles: master for both will
> be relevant to master of codebase, where eg v0.12.0 will be relevant to
> last release of 0.12.0 and what is live; there will be times we want to
> update what is live with something more reason relevant to that version,
> but we'll do that by pushing to 0.12, and we can release (publish) that
> immediately
My unhappiness here is the statement "v0.12.0 will be relevant to last
release of 0.12.0 and what is live" - this requires that the *submitter*
knows that if an immediate website update is required, that he must make
his PR against branch v0.12.0, and the committer must also merge it to that
branch, and build the website from that branch. This is putting the onus on
the *submitter* - and, to a lesser extent, the committer - to know this.
But there's another problem. In the hopefully not-too-distant future, we
will have more users using Brooklyn in production, and we are more likely
to hit "immediate bug-fix release required" problems, and we must support
users on a previous version.
Say we've published versions 1.0.0 and 1.1.0. We have many users in
production on 1.0.0, but not many on 1.1.0 as it's just been released. Then
we discover that one of the bugs we fixed in 1.1.0 was actually a security
vulnerability. Our 1.0.0 production users don't want to move up a whole
version to get a security vulnerability fixed, so we backport that fix and
make a new 1.0.1 release.
Q: Where do we document the CVE security advisories that urgently need to
go on the live website, and any subsequent live website updates? 1.0.1 (the
version we just released), master (seen by default when you go to GitHub),
1.1.0 (the highest-numbered release but not the most recent release), or a
combination?
Yes we can come up with a policy that explains this, but again it's causing
ambiguity and putting the onus on the PR submitter to know that policy
exists and to follow it.
--
In summary: keep it simple. There's only one website. So there shouldn't be
multiple website branches. It's in `master`, because if someone sees and
fixes a problem with the website that's where they will submit a PR. No
branches, no merges, no multiple PRs.
In the (currently rare) event of a website update that needs to be held
back for the next release, clearly say so in the PR title so that a
committer doesn't merge it. It's common in Brooklyn to mark a PR as "WIP"
or "For review only"; non-guide updates for new features could be handled
in the same way.
To repeat my earlier disclaimer, this is just an opinion, and writing it in
an email does not imply that it is correct! I am always open to listening
to the alternatives, and I am often persuaded change my mind :-)
Richard.
[1] https://en.wikipedia.org/wiki/Circle-ellipse_problem
>
>
>
> On 31/10/2017 15:19, Richard Downer wrote:
>
>> Hi Alex, Mark, sorry for the late reply.
>>
>> Alex - you make fair points. It does somewhat obscure the location of the
>> website and complicate raising PRs.
>>
>> I think that a separate repo is probably the best long-term solution, but
>> I
>> didn't want to rush into it - creating repos is cheap (Infra can do that
>> relatively easily) but deleting repos is nearly impossible for legal
>> policy
>> reasons. So I wanted to make sure we were making the right decision for a
>> docs/website split before we commit to a separate repo.
>>
>> I'm not convinced of your argument about website_0.12 etc. IMO the website
>> has a lifecycle which is independent of releases and as a general rule
>> there should only ever be one website branch. Having multiple copies of
>> the
>> website on different branches with content for different purposes is a
>> recipe for confusion. Such confusion risks updates on the website
>> appearing
>> and disappearing when the site is regenerated different branches, either
>> accidentally or when content is not merged to all appropriate branches. If
>> we do want to have a holding space for "v.Next" content on the main site,
>> then I think there are other ways of handling this.
>>
>> Could I suggest we remain with the current setup (different branch in docs
>> repo) for the short term, until we are absolutely happy with our new
>> arrangement, and then switch to a new brooklyn-website repo (or
>> alternative
>> if we don't like the new arrangement)? My thoughts are that this would
>> happen once we've made a 1.0 release and tested out all the process, or to
>> set a deadline if a 1.0 release starts getting pushed out, but I'm happy
>> to
>> hear thoughts on this.
>>
>> Richard.
>>
>>
>> On 27 October 2017 at 10:15, Mark McKenna <m4...@apache.org> wrote:
>>
>> Richard , Alex
>>>
>>> I think its fine on a branch short term (few weeks)
>>>
>>> I think we should follow the pattern from other projects and split our
>>> website into its own repository.
>>>
>>> Alex I think the technology of the website and the docs should diverge ie
>>> we choose the best technology for each.
>>>
>>> RE PRs ... splitting the site / docs makes the most sense as we deal with
>>> making prs across Brooklyn's numerous repositories
>>>
>>>
>>> Mark
>>>
>>>
>>>
>>> On 27 October 2017 at 09:01, Alex Heneveld <alex.heneveld@cloudsoftcorp.
>>> com>
>>> wrote:
>>>
>>> Sorry - I really don't like website being hidden on a branch.
>>>>
>>>> Why?
>>>>
>>>> * It's non-obvious and one more thing to remember (or forget, or
>>>> explain)
>>>> * Primary reason for branching (in my view) is lifecycle, and here the
>>>>
>>> two
>>>
>>>> have very similar versioning and release lifecycles: master for both
>>>> will
>>>> be relevant to master of codebase, where eg v0.12.0 will be relevant to
>>>> last release of 0.12.0 and what is live; there will be times we want to
>>>> update what is live with something more reason relevant to that version,
>>>> but we'll do that by pushing to 0.12, and we can release (publish) that
>>>> immediately unlike code, but what we _can't_ assume for both is that
>>>>
>>> master
>>>
>>>> can always be published, because we may have added unreleased features
>>>> to
>>>> docs for both site and guide; with site in a `website` branch we will
>>>>
>>> have
>>>
>>>> to have branches there eg `website_0.12` alongside `0.12` of guide
>>>> * PRs become more difficult, remembering which branch to open them
>>>>
>>> against
>>>
>>>> and merge against
>>>> * PRs become even more difficult if adding a feature which goes onto
>>>> both
>>>> the site and into the docs (which we should do more of), we add to one
>>>> branch, switch branches add to the other, then open PRs for both
>>>> branches
>>>> * Ideally they converge to use the same technology (not both gitbook and
>>>> markdown) so if in the same project they can share themes and utils etc;
>>>>
>>> if
>>>
>>>> in different branches we have to cherry-pick between branches (always
>>>> opening and reviewing 2 identical PRs!)
>>>>
>>>> My ideal would be different subdirs but if they really need to be
>>>>
>>> separate
>>>
>>>> then we could do a different project (resolves reasons 1 2 3 and
>>>>
>>> improves 4
>>>
>>>> and 5)
>>>>
>>>> Best
>>>> Alex
>>>>
>>>>
>>>>
>>>> On 26/10/2017 14:04, Richard Downer wrote:
>>>>
>>>> All,
>>>>>
>>>>> Following on from Thomas's gitbook work[1] - now merged[2] - it means
>>>>>
>>>> that
>>>
>>>> the brooklyn-docs `master` branch does not contain the public website
>>>>> (just
>>>>> the documentation).
>>>>>
>>>>> In those threads it was discussed that most branches would remove the
>>>>> website, and a separate branch would be used to store the public
>>>>> website
>>>>> (a
>>>>> bit like how GitHub pages used a `gh-pages` branch for a repository's
>>>>> publis website). I'm assuming that the acceptance and merging of
>>>>>
>>>> gitbooks
>>>
>>>> is passive consensus of this concept too.
>>>>>
>>>>> Therefore, I've created a new branch in the Apache brooklyn-docs repo
>>>>> called `website`, which is taken from `master` immediately prior to the
>>>>> merge of the gitbook conversion.
>>>>>
>>>>> If anybody disagrees with this passive consensus approach and thinks
>>>>>
>>>> that
>>>
>>>> this branch is a bad idea, then please feel free to speak up. After all,
>>>>> source control means that things can always be undone :-)
>>>>>
>>>>> Expect a forthcoming PR on the `website` branch to remove the
>>>>> documentation
>>>>> files and leave just the website files (like the opposite of Thomas's
>>>>>
>>>> PR).
>>>
>>>> Richard.
>>>>>
>>>>> [1]
>>>>> https://lists.apache.org/thread.html/760a3e2fdefaff8d8ac4ea3
>>>>> f98a45060fbaa0d886fa57c8f44e4742e@%3Cdev.brooklyn.apache.org%3E
>>>>> [2] https://github.com/apache/brooklyn-docs/pull/222
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: <ri...@apache.org>
>>>>> Date: 26 October 2017 at 13:50
>>>>> Subject: [brooklyn-docs] Git Push Summary
>>>>> To: commits@brooklyn.apache.org
>>>>>
>>>>>
>>>>> Repository: brooklyn-docs
>>>>> Updated Branches:
>>>>> refs/heads/website [created] e1d08e53c
>>>>>
>>>>>
>>>>>
>