Re: Support for (Java) Pattern Matching

[Jesse Glick <ty...@gmail.com>]

FWIW I have been working exclusively in GitHub for years now, and my
advice is to have every change be a PR coming from a repository fork.
Once you are accustomed to this workflow, it is not onerous, and
provides a natural place to track discussion of changes and
relationships to other changes. It also ensures that repositories of
record are not polluted with dozens of long-obsolete branches; there
are only PRs, which are either open or merged or closed without merge,
and a handful of origin branches reflecting official release lines.
Committers are just people trusted to push the Merge button.

(My project also generally demands that a PR not be merged until there
has been at least one code review from another committer; and an
automated build of that PR has passed all tests, so that the target
branch is always “clean”.)

On Fri, Oct 20, 2017 at 4:50 PM, Jan Lahoda <la...@gmail.com> wrote:
> if multiple people (can)
> contribute to a branch and the branch has a fair chance to end up in the
> mainline, it is IMO easier to keep it in an ASF repo, than on GitHub.

Even for this case I do not recommend using “origin PRs”. If you are
proposing a change, create a topic branch in your fork and file the
PR. If someone else has minor comments or suggestions, they can review
and you can decide how to process their feedback. If they have
significant changes to propose which are too complex to comfortably
describe verbally, they can simply create a child branch and file a PR
_against your PR_. If you agree with what they propose—perhaps after
some discussion and refinement—you just merge the sub-PR, and their
changes are incorporated. But the author of the PR remains of control.
When someone else can push directly to the branch of your PR, they may
well be “putting words in your mouth”.

Once you start to use GitHub the way it is designed, your development
style changes, and for the better.

Re: Support for (Java) Pattern Matching

[Jan Lahoda <la...@gmail.com>]

On Wed, Nov 1, 2017 at 5:49 PM, Jesse Glick <ty...@gmail.com> wrote:

> On Wed, Nov 1, 2017 at 8:35 AM, Jan Lahoda <la...@gmail.com> wrote:
> > you propose to open the PR from the
> > personal fork and leave it open for moths or years so that the code is
> > "perpetually" submitted to the ASF and can (hopefully!) be used, while
> the
> > actual development is still happening on a branch in a personal fork
> (which
> > is the source for the PR). Sounds like a workaround to me (the pull
> request
> > is not really requesting anything to be pulled for most of the lifetime
> of
> > the PR).
>
> A PR should be a unit of work that could plausibly be merged and
> released. If you are releasing betas from some `patterns` branch in
> origin, then most PRs to it would just be things like fixing some bug
> in pattern support. I presume these would not typically be stalled for
> more than a few weeks on end, even if a vacation intervenes. The
> initial basic support for patterns might be one largish PR.
>
> > it definitely seems smoother to me to do "git push" then to do "git
> push", go to a browser, open github, open a PR, and possibly wait and merge
> it to an ASF repo
>
> Well, sure. There are CLI tools to automate steps, but really the
> benefits of peer review more than counterbalance any technical
> inconvenience in my experience. Even when you do not get any feedback
> on a PR and wind up merging it yourself as a committer, knowing that
> someone _might_ be reviewing it forces you to think a bit more about
> presenting your work.
>

On one hand yes, on the other one: even the pushed (merged) code is still
open, and anyone can review it and provide feedback. The PR can make it
more convenient in some cases, but a commit may be reviewed any time
anyway, so I'd assume such forces to be in place all the time. Overall, I
am afraid I am not aware of a feedback neither the jdk-javac nor the
jdk/amber/patterns branch, and I am not sure if just using a PR would make
a difference in that.

Anyway, I am thinking of trying to use the long-term work-in-progress PR
for the impl. of
https://cwiki.apache.org/confluence/display/NETBEANS/JDK+Modules+Support+in+NetBeans+Module+System,
I hope that may lead to some more feedback.

Jan


> Obviously every development team has different policies and styles,
> and you can find ways to accommodate them all; it is a question of
> what culture you want to define.
>

Re: Support for (Java) Pattern Matching

[Jesse Glick <ty...@gmail.com>]

On Wed, Nov 1, 2017 at 8:35 AM, Jan Lahoda <la...@gmail.com> wrote:
> you propose to open the PR from the
> personal fork and leave it open for moths or years so that the code is
> "perpetually" submitted to the ASF and can (hopefully!) be used, while the
> actual development is still happening on a branch in a personal fork (which
> is the source for the PR). Sounds like a workaround to me (the pull request
> is not really requesting anything to be pulled for most of the lifetime of
> the PR).

A PR should be a unit of work that could plausibly be merged and
released. If you are releasing betas from some `patterns` branch in
origin, then most PRs to it would just be things like fixing some bug
in pattern support. I presume these would not typically be stalled for
more than a few weeks on end, even if a vacation intervenes. The
initial basic support for patterns might be one largish PR.

> it definitely seems smoother to me to do "git push" then to do "git
push", go to a browser, open github, open a PR, and possibly wait and merge
it to an ASF repo

Well, sure. There are CLI tools to automate steps, but really the
benefits of peer review more than counterbalance any technical
inconvenience in my experience. Even when you do not get any feedback
on a PR and wind up merging it yourself as a committer, knowing that
someone _might_ be reviewing it forces you to think a bit more about
presenting your work.

Obviously every development team has different policies and styles,
and you can find ways to accommodate them all; it is a question of
what culture you want to define.

Re: Support for (Java) Pattern Matching

[Jan Lahoda <la...@gmail.com>]

On Wed, Nov 1, 2017 at 12:51 PM, Jesse Glick <ty...@gmail.com> wrote:

> On Wed, Nov 1, 2017 at 2:31 AM, Jan Lahoda <la...@gmail.com> wrote:
> > when one opens a pull request against
> > apache/(incubating-)netbeans, then I think the code can be considered to
> be
> > submitted to ASF (per iCLA).
>
> IANAL but that sounds right.
>
> > for a long-running branch in a
> > personal repository, if the author of the branch cannot send the pull
> > request to merge it for whatever reason
>
> I recommend just filing a PR the moment you start the branch and have
> some initial commits. It can be marked as a work in progress / not for
> immediate merge / etc. with various labels. (It is possible to create
> PR search permalinks on GitHub which filter by label.) If your
> experiment never comes to anything, close the PR if you like. You can
> always reopen it and pick up where you left off, with review comments
> and everything intact.
>

So, if I understand it correctly, you propose to open the PR from the
personal fork and leave it open for moths or years so that the code is
"perpetually" submitted to the ASF and can (hopefully!) be used, while the
actual development is still happening on a branch in a personal fork (which
is the source for the PR). Sounds like a workaround to me (the pull request
is not really requesting anything to be pulled for most of the lifetime of
the PR).


>
> (Normally when merging a PR I would delete the corresponding fork
> branch, both on GitHub and in my local clone, to tidy up; but if you
> are closing a PR you expect to maybe resume work on later, you can
> just leave the branch in the remote. You could probably even recreate
> the branch based on the last commit hash visible in the PR, though I
> cannot recall ever trying that.)
>
> On this topic: the Eclipse foundation apparently has some tool they
> run automatically on PRs which verifies that all commits have been
> signed and that the signature matches someone with a CLA on file. This
> shows up as a GitHub status check, so you can see right in the PR
> whether the contribution is coming from someone who has done the right
> paperwork. If as a new contributor you forget about this, there will
> be a red X until you file a CLA, fix your Git settings, and amend your
> commit to include a signature.
>
> > how about releases from the branches. It may be desirable to do a
> > "preview" "release" for the branch (e.g. "pattern" preview; although I'd
> > prefer if we handled Java language features in the master, but even if we
> > succeed in that, it will take a long time to achieve). I don't think we
> can
> > release from a personal github.
>
> Certainly you would not want to cut official releases from a fork
> branch. If this is an official preview of some kind—like a beta
> release?—then there should I suppose be some origin branch for that
> purpose. You can still use PRs to propose specific additions to that
> branch; you just set the target branch accordingly when filing the PR.
>

I guess I am a little bit lost in the terminology here. If
incubator-netbeans/master is the branch from which the official final
releases are made, there may be a different branch say "patterns" somewhere
that contains support for Java language patterns. It is conceivable to me
that we may do an official (in the ASF sense) "patterns preview" release,
where people could try patterns, and there could be more bugs than in the
final release. AFAIK. This was being done for a long time for various
upcoming Java features. If we can have the future Java features supported
in master, that'd be great, but for that we either need some significant
improvements on the jdk-javac branch, or some other realistic proposal to
achieve that.

Jan


>
> It really depends on what your policy for distributing experimental
> changes is. As you say, using feature flags or hidden settings to
> enable unstable code in normal releases is the simplest approach for
> infrastructure, but might not be practical until the code has been
> refactored enough to allow this kind of behavior to be switched
> dynamically. So if you are publishing, essentially, forks of official
> modules, you are going to run into tricky questions anyway. What
> version number could you pick which is newer than the official
> ancestor merge base, but will be older than the official release
> version when the feature is stabilized, and incomparable to unrelated
> feature forks? Etc.
>

Re: Support for (Java) Pattern Matching

[Jesse Glick <ty...@gmail.com>]

On Wed, Nov 1, 2017 at 2:31 AM, Jan Lahoda <la...@gmail.com> wrote:
> when one opens a pull request against
> apache/(incubating-)netbeans, then I think the code can be considered to be
> submitted to ASF (per iCLA).

IANAL but that sounds right.

> for a long-running branch in a
> personal repository, if the author of the branch cannot send the pull
> request to merge it for whatever reason

I recommend just filing a PR the moment you start the branch and have
some initial commits. It can be marked as a work in progress / not for
immediate merge / etc. with various labels. (It is possible to create
PR search permalinks on GitHub which filter by label.) If your
experiment never comes to anything, close the PR if you like. You can
always reopen it and pick up where you left off, with review comments
and everything intact.

(Normally when merging a PR I would delete the corresponding fork
branch, both on GitHub and in my local clone, to tidy up; but if you
are closing a PR you expect to maybe resume work on later, you can
just leave the branch in the remote. You could probably even recreate
the branch based on the last commit hash visible in the PR, though I
cannot recall ever trying that.)

On this topic: the Eclipse foundation apparently has some tool they
run automatically on PRs which verifies that all commits have been
signed and that the signature matches someone with a CLA on file. This
shows up as a GitHub status check, so you can see right in the PR
whether the contribution is coming from someone who has done the right
paperwork. If as a new contributor you forget about this, there will
be a red X until you file a CLA, fix your Git settings, and amend your
commit to include a signature.

> how about releases from the branches. It may be desirable to do a
> "preview" "release" for the branch (e.g. "pattern" preview; although I'd
> prefer if we handled Java language features in the master, but even if we
> succeed in that, it will take a long time to achieve). I don't think we can
> release from a personal github.

Certainly you would not want to cut official releases from a fork
branch. If this is an official preview of some kind—like a beta
release?—then there should I suppose be some origin branch for that
purpose. You can still use PRs to propose specific additions to that
branch; you just set the target branch accordingly when filing the PR.

It really depends on what your policy for distributing experimental
changes is. As you say, using feature flags or hidden settings to
enable unstable code in normal releases is the simplest approach for
infrastructure, but might not be practical until the code has been
refactored enough to allow this kind of behavior to be switched
dynamically. So if you are publishing, essentially, forks of official
modules, you are going to run into tricky questions anyway. What
version number could you pick which is newer than the official
ancestor merge base, but will be older than the official release
version when the feature is stabilized, and incomparable to unrelated
feature forks? Etc.

Re: Support for (Java) Pattern Matching

[Jan Lahoda <la...@gmail.com>]

On Wed, Nov 1, 2017 at 12:49 PM, Neil C Smith <neilcsmith.net@googlemail.com
> wrote:

> On Wed, Nov 1, 2017 at 6:32 AM Jan Lahoda <la...@gmail.com> wrote:
>
> > -code provenance: when one opens a pull request against
> > apache/(incubating-)netbeans, then I think the code can be considered to
> be
> > submitted to ASF (per iCLA). Pushing a commit to a personal github
> > repository does not feel that way. So, for a long-running branch in a
> > personal repository, if the author of the branch cannot send the pull
> > request to merge it for whatever reason, how do we transfer the code to
> the
> > mainline (if we want it)? If the branch is in a ASF repo, it has already
> > been submitted to ASF, and anyone can continue without too much
> > (non-technical) issues.
> >
> > -how about releases from the branches. It may be desirable to do a
> > "preview" "release" for the branch (e.g. "pattern" preview; although I'd
> > prefer if we handled Java language features in the master, but even if we
> > succeed in that, it will take a long time to achieve). I don't think we
> can
> > release from a personal github.
> >
> >
> I'm tempted to say, with respect, you're making straw man arguments,
> although I think Jesse's email does imply longer lasting personal branches
> than might be desired.
>
> Git is distributed.  You have to have at least one clone (fork) to work
> with anyway, at least locally, so some commits happen elsewhere.  Your
> argument seems to rest on the fact that there would somehow be a difference
> in the scenarios between when you'd push to the main repo vs when you would
> push to your personal remote and file a PR.  Why?
>

(I believe this is different from what Jesse proposes because in your
proposal there would be only a short time between push to the personal repo
and a merge to the ASF repo.)

I don't have much against such approach (in a sense, it might be easier for
me if I'd just push to my github, and Geertjan merged the change for me).
OTOH, it definitely seems smoother to me to do "git push" then to do "git
push", go to a browser, open github, open a PR, and possibly wait and merge
it to an ASF repo. So I wonder if this is not discouraging contributions
unnecessarily.

Jan


>
> I'm 100% in agreement with the idea that all collaboration happens within
> the Apache repo (or within PR's to it before they merge) - even if this
> wasn't the Apache way.  But as Jesse says, this is a natural GitHub
> workflow, it's not onerous, and from the surveys I've seen narrowly the
> majority way of working (without starting Geertjan on the lack of merit of
> surveys! ;-) ).  That doesn't mean we have to choose it, but at some point
> we need a defined workflow that everyone knows how to use.
>
> Best wishes,
>
> Neil
> --
> Neil C Smith
> Artist & Technologist
> www.neilcsmith.net
>
> Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org
>

Re: Support for (Java) Pattern Matching

[Neil C Smith <ne...@googlemail.com>]

On Wed, Nov 1, 2017 at 6:32 AM Jan Lahoda <la...@gmail.com> wrote:

> -code provenance: when one opens a pull request against
> apache/(incubating-)netbeans, then I think the code can be considered to be
> submitted to ASF (per iCLA). Pushing a commit to a personal github
> repository does not feel that way. So, for a long-running branch in a
> personal repository, if the author of the branch cannot send the pull
> request to merge it for whatever reason, how do we transfer the code to the
> mainline (if we want it)? If the branch is in a ASF repo, it has already
> been submitted to ASF, and anyone can continue without too much
> (non-technical) issues.
>
> -how about releases from the branches. It may be desirable to do a
> "preview" "release" for the branch (e.g. "pattern" preview; although I'd
> prefer if we handled Java language features in the master, but even if we
> succeed in that, it will take a long time to achieve). I don't think we can
> release from a personal github.
>
>
I'm tempted to say, with respect, you're making straw man arguments,
although I think Jesse's email does imply longer lasting personal branches
than might be desired.

Git is distributed.  You have to have at least one clone (fork) to work
with anyway, at least locally, so some commits happen elsewhere.  Your
argument seems to rest on the fact that there would somehow be a difference
in the scenarios between when you'd push to the main repo vs when you would
push to your personal remote and file a PR.  Why?

I'm 100% in agreement with the idea that all collaboration happens within
the Apache repo (or within PR's to it before they merge) - even if this
wasn't the Apache way.  But as Jesse says, this is a natural GitHub
workflow, it's not onerous, and from the surveys I've seen narrowly the
majority way of working (without starting Geertjan on the lack of merit of
surveys! ;-) ).  That doesn't mean we have to choose it, but at some point
we need a defined workflow that everyone knows how to use.

Best wishes,

Neil
-- 
Neil C Smith
Artist & Technologist
www.neilcsmith.net

Praxis LIVE - hybrid visual IDE for creative coding - www.praxislive.org

Re: Support for (Java) Pattern Matching

[Jan Lahoda <la...@gmail.com>]

Hi Jesse,

Thanks for the comments! A comment is inline.

On Wed, Nov 1, 2017 at 1:54 AM, Jesse Glick <ty...@gmail.com> wrote:

> FWIW I have been working exclusively in GitHub for years now, and my
> advice is to have every change be a PR coming from a repository fork.
> Once you are accustomed to this workflow, it is not onerous, and
> provides a natural place to track discussion of changes and
> relationships to other changes. It also ensures that repositories of
> record are not polluted with dozens of long-obsolete branches; there
> are only PRs, which are either open or merged or closed without merge,
> and a handful of origin branches reflecting official release lines.
> Committers are just people trusted to push the Merge button.
>
> (My project also generally demands that a PR not be merged until there
> has been at least one code review from another committer; and an
> automated build of that PR has passed all tests, so that the target
> branch is always “clean”.)
>
> On Fri, Oct 20, 2017 at 4:50 PM, Jan Lahoda <la...@gmail.com> wrote:
> > if multiple people (can)
> > contribute to a branch and the branch has a fair chance to end up in the
> > mainline, it is IMO easier to keep it in an ASF repo, than on GitHub.
>
> Even for this case I do not recommend using “origin PRs”. If you are
> proposing a change, create a topic branch in your fork and file the
> PR. If someone else has minor comments or suggestions, they can review
> and you can decide how to process their feedback. If they have
> significant changes to propose which are too complex to comfortably
> describe verbally, they can simply create a child branch and file a PR
> _against your PR_. If you agree with what they propose—perhaps after
> some discussion and refinement—you just merge the sub-PR, and their
> changes are incorporated. But the author of the PR remains of control.
> When someone else can push directly to the branch of your PR, they may
> well be “putting words in your mouth”.
>

I think we are using this process for changes that are developed for a
short time and end up in the master branch. I suppose it would be easier to
me/us to simply use our github for changes that are being developed for a
long term and only merge when things are done, but I see considerable
problems with that:
-code provenance: when one opens a pull request against
apache/(incubating-)netbeans, then I think the code can be considered to be
submitted to ASF (per iCLA). Pushing a commit to a personal github
repository does not feel that way. So, for a long-running branch in a
personal repository, if the author of the branch cannot send the pull
request to merge it for whatever reason, how do we transfer the code to the
mainline (if we want it)? If the branch is in a ASF repo, it has already
been submitted to ASF, and anyone can continue without too much
(non-technical) issues.

-how about releases from the branches. It may be desirable to do a
"preview" "release" for the branch (e.g. "pattern" preview; although I'd
prefer if we handled Java language features in the master, but even if we
succeed in that, it will take a long time to achieve). I don't think we can
release from a personal github.

Jan


>
> Once you start to use GitHub the way it is designed, your development
> style changes, and for the better.
>