You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Karthik Kambatla <ka...@cloudera.com> on 2015/11/10 23:07:24 UTC
Need for force-push on feature branches
Hi folks,
Recently, Infra disabled force-pushes (and non-fast-forward pushes) to all
branches to avoid accidental overwrites or deletions.
I propose we reach out to Infra and ask for an exemption since our workflow
for feature branches involves deletions and force-pushes.
We should likely wait for a day or so to hear any concerns against this
request. Also, can someone volunteer following up on this? I am going away
on vacation shortly, and will have limited access to internet/email.
Karthik
Re: Need for force-push on feature branches
Posted by Sangjin Lee <sj...@apache.org>.
I already opened an INFRA JIRA for my specific issue (INFRA-10745
<https://issues.apache.org/jira/browse/INFRA-10745>), and also sent email
to them. Steve Loughran made me aware of the lockdown there.
We can reuse that JIRA for this discussion? I don't mind following up
later, as we're blocked (YARN-2928). If we need to change the branch naming
convention, then I think we might need a vote.
Also, deleting a branch is also disabled. That would have been another
avenue for rebase, but that's closed too.
Sangjin
On Tue, Nov 10, 2015 at 2:07 PM, Karthik Kambatla <ka...@cloudera.com>
wrote:
> Hi folks,
>
> Recently, Infra disabled force-pushes (and non-fast-forward pushes) to all
> branches to avoid accidental overwrites or deletions.
>
> I propose we reach out to Infra and ask for an exemption since our workflow
> for feature branches involves deletions and force-pushes.
>
> We should likely wait for a day or so to hear any concerns against this
> request. Also, can someone volunteer following up on this? I am going away
> on vacation shortly, and will have limited access to internet/email.
>
> Karthik
>
Re: Need for force-push on feature branches
Posted by Karthik Kambatla <ka...@cloudera.com>.
Just received an email today from Infra that force pushes are allowed
again. Relevant excerpt below:
--------------
First, If a forced commit is pushed, the subsequent commit email will
contain '[Forced Update!]' in the subject line. The hope here is that
it draws extra attention to the situation for a project community to
be aware, and take appropriate action if needed.
Second, we've changed the 'protected' portions of git to primarily
focus on refs/tags/rel - thus any tags under rel, will have their
entire commit history. This provides the provenance that the ASF needs
for releases, while still giving projects the ability to mold their
repository in the way they see fit.
--------------
On Thu, Jan 7, 2016 at 5:39 PM, Sangjin Lee <sj...@gmail.com> wrote:
> There was this on the infra email back in December:
>
> On 1 December 2015 at 13:55, David Nalley <da...@gnsa.us> wrote:
> >
> > As part of the steps to removing the git lockdown, the board has
> > tasked the President (and by extension Infrastructure) with defining:
> >
> > some ASF-wide convention for identifying which Git references are to
> > be treated as immutable within ASF controlled public copy of Git
> > repositories. At a minimum, this must include a reference for each
> > release.
> >
> > For historical perspective, we protected the trunk and master
> > branches, as well as any branches under /refs/heads/rel, and tags
> > under /refs/tags/rel/
> >
> > The thought being that master or trunk were the locations of
> > development that was going to end up in releases, and that releases
> > would be tagged something like tags/rel/4.6.0.
> >
> > In general practice, a large number of folks were not operating in
> > that manner. So, that brings us to today. At a minimum, the board is
> > expecting that we identify a way to keep release references immutable.
> >
> >
> > Thoughts, comments, flames?
> >
> > --David
>
>
> I don't know that there has been concrete action taken by the
> infrastructure.
>
> Regards,
> Sangjin
>
> On Thu, Jan 7, 2016 at 3:50 PM, Karthik Kambatla <ka...@cloudera.com>
> wrote:
>
> > Sangjin, Steve and others:
> >
> > just curious if you guys figured out a way to address/workaround this
> > hurdle - not being able to delete or force push. Should we reach out to
> > INFRA again?
> >
> > On Thu, Nov 12, 2015 at 10:21 AM, Allen Wittenauer <aw...@altiscale.com>
> > wrote:
> >
> > >
> > >
> > >
> > >
> > > > On Nov 12, 2015, at 10:14 AM, Sangjin Lee <sj...@gmail.com> wrote:
> > > >
> > > > I don't think we're proposing project-specific rules. It would be a
> > > > recognition of the git branch name prefix "feature/".
> > > >
> > > > If the file name had "HADOOP-67890-HADOOP-12345.001.patch" where
> > > > HADOOP-12345 was the feature JIRA but the git branch was
> > > > "feature/HADOOP-12345", if yetus didn't find a branch named
> > > "HADOOP-12345",
> > > > can it also look for "feature/HADOOP-12345" before failing the build?
> > > > HADOOP-12345 can be any branch JIRA.
> > >
> > >
> > > In a “everyone in the world including outside the ASF wants
> this
> > > magic branch matching happening” way, no, Yetus shouldn't do that. If
> > > Yetus added a flag to the JIRA plugin where this patch file naming
> stuff
> > > happens or make it a personality setting, then potentially yes. Yetus
> > > isn’t just Hadoop and it’s important to remember that when requesting
> > > additions.
> >
>
Re: Need for force-push on feature branches
Posted by Sangjin Lee <sj...@gmail.com>.
There was this on the infra email back in December:
On 1 December 2015 at 13:55, David Nalley <da...@gnsa.us> wrote:
>
> As part of the steps to removing the git lockdown, the board has
> tasked the President (and by extension Infrastructure) with defining:
>
> some ASF-wide convention for identifying which Git references are to
> be treated as immutable within ASF controlled public copy of Git
> repositories. At a minimum, this must include a reference for each
> release.
>
> For historical perspective, we protected the trunk and master
> branches, as well as any branches under /refs/heads/rel, and tags
> under /refs/tags/rel/
>
> The thought being that master or trunk were the locations of
> development that was going to end up in releases, and that releases
> would be tagged something like tags/rel/4.6.0.
>
> In general practice, a large number of folks were not operating in
> that manner. So, that brings us to today. At a minimum, the board is
> expecting that we identify a way to keep release references immutable.
>
>
> Thoughts, comments, flames?
>
> --David
I don't know that there has been concrete action taken by the
infrastructure.
Regards,
Sangjin
On Thu, Jan 7, 2016 at 3:50 PM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Sangjin, Steve and others:
>
> just curious if you guys figured out a way to address/workaround this
> hurdle - not being able to delete or force push. Should we reach out to
> INFRA again?
>
> On Thu, Nov 12, 2015 at 10:21 AM, Allen Wittenauer <aw...@altiscale.com>
> wrote:
>
> >
> >
> >
> >
> > > On Nov 12, 2015, at 10:14 AM, Sangjin Lee <sj...@gmail.com> wrote:
> > >
> > > I don't think we're proposing project-specific rules. It would be a
> > > recognition of the git branch name prefix "feature/".
> > >
> > > If the file name had "HADOOP-67890-HADOOP-12345.001.patch" where
> > > HADOOP-12345 was the feature JIRA but the git branch was
> > > "feature/HADOOP-12345", if yetus didn't find a branch named
> > "HADOOP-12345",
> > > can it also look for "feature/HADOOP-12345" before failing the build?
> > > HADOOP-12345 can be any branch JIRA.
> >
> >
> > In a “everyone in the world including outside the ASF wants this
> > magic branch matching happening” way, no, Yetus shouldn't do that. If
> > Yetus added a flag to the JIRA plugin where this patch file naming stuff
> > happens or make it a personality setting, then potentially yes. Yetus
> > isn’t just Hadoop and it’s important to remember that when requesting
> > additions.
>
Re: Need for force-push on feature branches
Posted by Karthik Kambatla <ka...@cloudera.com>.
Sangjin, Steve and others:
just curious if you guys figured out a way to address/workaround this
hurdle - not being able to delete or force push. Should we reach out to
INFRA again?
On Thu, Nov 12, 2015 at 10:21 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>
>
>
> > On Nov 12, 2015, at 10:14 AM, Sangjin Lee <sj...@gmail.com> wrote:
> >
> > I don't think we're proposing project-specific rules. It would be a
> > recognition of the git branch name prefix "feature/".
> >
> > If the file name had "HADOOP-67890-HADOOP-12345.001.patch" where
> > HADOOP-12345 was the feature JIRA but the git branch was
> > "feature/HADOOP-12345", if yetus didn't find a branch named
> "HADOOP-12345",
> > can it also look for "feature/HADOOP-12345" before failing the build?
> > HADOOP-12345 can be any branch JIRA.
>
>
> In a “everyone in the world including outside the ASF wants this
> magic branch matching happening” way, no, Yetus shouldn't do that. If
> Yetus added a flag to the JIRA plugin where this patch file naming stuff
> happens or make it a personality setting, then potentially yes. Yetus
> isn’t just Hadoop and it’s important to remember that when requesting
> additions.
Re: Need for force-push on feature branches
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 12, 2015, at 10:14 AM, Sangjin Lee <sj...@gmail.com> wrote:
>
> I don't think we're proposing project-specific rules. It would be a
> recognition of the git branch name prefix "feature/".
>
> If the file name had "HADOOP-67890-HADOOP-12345.001.patch" where
> HADOOP-12345 was the feature JIRA but the git branch was
> "feature/HADOOP-12345", if yetus didn't find a branch named "HADOOP-12345",
> can it also look for "feature/HADOOP-12345" before failing the build?
> HADOOP-12345 can be any branch JIRA.
In a “everyone in the world including outside the ASF wants this magic branch matching happening” way, no, Yetus shouldn't do that. If Yetus added a flag to the JIRA plugin where this patch file naming stuff happens or make it a personality setting, then potentially yes. Yetus isn’t just Hadoop and it’s important to remember that when requesting additions.
Re: Need for force-push on feature branches
Posted by Sangjin Lee <sj...@gmail.com>.
I don't think we're proposing project-specific rules. It would be a
recognition of the git branch name prefix "feature/".
If the file name had "HADOOP-67890-HADOOP-12345.001.patch" where
HADOOP-12345 was the feature JIRA but the git branch was
"feature/HADOOP-12345", if yetus didn't find a branch named "HADOOP-12345",
can it also look for "feature/HADOOP-12345" before failing the build?
HADOOP-12345 can be any branch JIRA.
On Thu, Nov 12, 2015 at 10:08 AM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> Implementing project-specific patch identification rules are definitely
> ‘not trivial’.
>
> FWIW, the documented ruleset Yetus supports is here:
>
>
> https://yetus.apache.org/documentation/latest/precommit-patchnames/
>
> (Altho, in reality, the code does support more than this but they are sort
> of weird looking, etc.)
>
> > On Nov 12, 2015, at 10:04 AM, Sangjin Lee <sj...@gmail.com> wrote:
> >
> > I suppose that would be fine too. Yetus just needs to add "feature/" to
> the
> > git branch name when it fails to find it as is. So it would require a
> > little work on yetus, but I guess should be pretty trivial?
> >
> > On Thu, Nov 12, 2015 at 9:59 AM, Steve Loughran <st...@hortonworks.com>
> > wrote:
> >
> >>
> >>> On 12 Nov 2015, at 17:49, Sangjin Lee <sj...@gmail.com> wrote:
> >>>
> >>> Yes, I completely understand about the git branch naming practice (in
> >> fact
> >>> that's what I normally do). I was commenting on our hadoop patch naming
> >>> convention. We are supposed to use patch names as
> >>> "<jira-id>-<branch-name-if-not-trunk>.<sequence>.patch".
> >>>
> >>> If we used "feature/HADOOP-12345" as the git branch name and the
> subtask
> >>> JIRA was HADOOP-67890 for example, the patch file name would be
> >>> "HADOOP-67890-feature/HADOOP-12345.001.patch", which would not be
> doable,
> >>> no? For that to work, we would need to have some kind of escaping
> >>> convention which jenkins and users follow.
> >>>
> >>
> >> aah, now I follow. wouldn't HADOOP-12345.001.patch be enough?
> >>
>
>
Re: Need for force-push on feature branches
Posted by Allen Wittenauer <aw...@altiscale.com>.
Implementing project-specific patch identification rules are definitely ‘not trivial’.
FWIW, the documented ruleset Yetus supports is here:
https://yetus.apache.org/documentation/latest/precommit-patchnames/
(Altho, in reality, the code does support more than this but they are sort of weird looking, etc.)
> On Nov 12, 2015, at 10:04 AM, Sangjin Lee <sj...@gmail.com> wrote:
>
> I suppose that would be fine too. Yetus just needs to add "feature/" to the
> git branch name when it fails to find it as is. So it would require a
> little work on yetus, but I guess should be pretty trivial?
>
> On Thu, Nov 12, 2015 at 9:59 AM, Steve Loughran <st...@hortonworks.com>
> wrote:
>
>>
>>> On 12 Nov 2015, at 17:49, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>> Yes, I completely understand about the git branch naming practice (in
>> fact
>>> that's what I normally do). I was commenting on our hadoop patch naming
>>> convention. We are supposed to use patch names as
>>> "<jira-id>-<branch-name-if-not-trunk>.<sequence>.patch".
>>>
>>> If we used "feature/HADOOP-12345" as the git branch name and the subtask
>>> JIRA was HADOOP-67890 for example, the patch file name would be
>>> "HADOOP-67890-feature/HADOOP-12345.001.patch", which would not be doable,
>>> no? For that to work, we would need to have some kind of escaping
>>> convention which jenkins and users follow.
>>>
>>
>> aah, now I follow. wouldn't HADOOP-12345.001.patch be enough?
>>
Re: Need for force-push on feature branches
Posted by Sangjin Lee <sj...@gmail.com>.
I suppose that would be fine too. Yetus just needs to add "feature/" to the
git branch name when it fails to find it as is. So it would require a
little work on yetus, but I guess should be pretty trivial?
On Thu, Nov 12, 2015 at 9:59 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 12 Nov 2015, at 17:49, Sangjin Lee <sj...@gmail.com> wrote:
> >
> > Yes, I completely understand about the git branch naming practice (in
> fact
> > that's what I normally do). I was commenting on our hadoop patch naming
> > convention. We are supposed to use patch names as
> > "<jira-id>-<branch-name-if-not-trunk>.<sequence>.patch".
> >
> > If we used "feature/HADOOP-12345" as the git branch name and the subtask
> > JIRA was HADOOP-67890 for example, the patch file name would be
> > "HADOOP-67890-feature/HADOOP-12345.001.patch", which would not be doable,
> > no? For that to work, we would need to have some kind of escaping
> > convention which jenkins and users follow.
> >
>
> aah, now I follow. wouldn't HADOOP-12345.001.patch be enough?
>
Re: Need for force-push on feature branches
Posted by Steve Loughran <st...@hortonworks.com>.
> On 12 Nov 2015, at 17:49, Sangjin Lee <sj...@gmail.com> wrote:
>
> Yes, I completely understand about the git branch naming practice (in fact
> that's what I normally do). I was commenting on our hadoop patch naming
> convention. We are supposed to use patch names as
> "<jira-id>-<branch-name-if-not-trunk>.<sequence>.patch".
>
> If we used "feature/HADOOP-12345" as the git branch name and the subtask
> JIRA was HADOOP-67890 for example, the patch file name would be
> "HADOOP-67890-feature/HADOOP-12345.001.patch", which would not be doable,
> no? For that to work, we would need to have some kind of escaping
> convention which jenkins and users follow.
>
aah, now I follow. wouldn't HADOOP-12345.001.patch be enough?
Re: Need for force-push on feature branches
Posted by Sangjin Lee <sj...@gmail.com>.
Yes, I completely understand about the git branch naming practice (in fact
that's what I normally do). I was commenting on our hadoop patch naming
convention. We are supposed to use patch names as
"<jira-id>-<branch-name-if-not-trunk>.<sequence>.patch".
If we used "feature/HADOOP-12345" as the git branch name and the subtask
JIRA was HADOOP-67890 for example, the patch file name would be
"HADOOP-67890-feature/HADOOP-12345.001.patch", which would not be doable,
no? For that to work, we would need to have some kind of escaping
convention which jenkins and users follow.
On Thu, Nov 12, 2015 at 1:23 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 11 Nov 2015, at 22:47, Sangjin Lee <sj...@apache.org> wrote:
> >
> > If we do use "feature/..." as the branch naming convention, it does pose
> an
> > issue with the patch naming due to the separator character ('/'). How
> about
> > "feature-..."?
>
> In most spark UIs, they get treated as subdirectories and easier to
> manager. Try it on something like sourcetree.
>
Re: Need for force-push on feature branches
Posted by Steve Loughran <st...@hortonworks.com>.
> On 11 Nov 2015, at 22:47, Sangjin Lee <sj...@apache.org> wrote:
>
> If we do use "feature/..." as the branch naming convention, it does pose an
> issue with the patch naming due to the separator character ('/'). How about
> "feature-..."?
In most spark UIs, they get treated as subdirectories and easier to manager. Try it on something like sourcetree.
Re: Need for force-push on feature branches
Posted by Sangjin Lee <sj...@apache.org>.
If we do use "feature/..." as the branch naming convention, it does pose an
issue with the patch naming due to the separator character ('/'). How about
"feature-..."?
On Tue, Nov 10, 2015 at 3:02 PM, Anu Engineer <ae...@hortonworks.com>
wrote:
> I ran into the same issue and filed an INFRA jira too.
>
> https://issues.apache.org/jira/browse/INFRA-10720
>
>
> So +1 for having the git control back
>
> Thanks
> Anu
>
>
>
>
>
> On 11/10/15, 2:45 PM, "Steve Loughran" <st...@hortonworks.com> wrote:
>
> >
> >> On 10 Nov 2015, at 22:07, Karthik Kambatla <ka...@cloudera.com> wrote:
> >>
> >> Hi folks,
> >>
> >> Recently, Infra disabled force-pushes (and non-fast-forward pushes) to
> all
> >> branches to avoid accidental overwrites or deletions.
> >>
> >> I propose we reach out to Infra and ask for an exemption since our
> workflow
> >> for feature branches involves deletions and force-pushes.
> >>
> >
> >I asked them for this exemption for all branches called feature/* earlier
> today; its consistent with the git flow branch naming.
> >
> >if all feature branches go in under there, people working on them are
> free to rebase as they choose.
> >
> >> We should likely wait for a day or so to hear any concerns against this
> >> request. Also, can someone volunteer following up on this? I am going
> away
> >> on vacation shortly, and will have limited access to internet/email.
> >>
> >
> >
> >
>
Re: Need for force-push on feature branches
Posted by Anu Engineer <ae...@hortonworks.com>.
I ran into the same issue and filed an INFRA jira too.
https://issues.apache.org/jira/browse/INFRA-10720
So +1 for having the git control back
Thanks
Anu
On 11/10/15, 2:45 PM, "Steve Loughran" <st...@hortonworks.com> wrote:
>
>> On 10 Nov 2015, at 22:07, Karthik Kambatla <ka...@cloudera.com> wrote:
>>
>> Hi folks,
>>
>> Recently, Infra disabled force-pushes (and non-fast-forward pushes) to all
>> branches to avoid accidental overwrites or deletions.
>>
>> I propose we reach out to Infra and ask for an exemption since our workflow
>> for feature branches involves deletions and force-pushes.
>>
>
>I asked them for this exemption for all branches called feature/* earlier today; its consistent with the git flow branch naming.
>
>if all feature branches go in under there, people working on them are free to rebase as they choose.
>
>> We should likely wait for a day or so to hear any concerns against this
>> request. Also, can someone volunteer following up on this? I am going away
>> on vacation shortly, and will have limited access to internet/email.
>>
>
>
>
Re: Need for force-push on feature branches
Posted by Steve Loughran <st...@hortonworks.com>.
> On 10 Nov 2015, at 22:07, Karthik Kambatla <ka...@cloudera.com> wrote:
>
> Hi folks,
>
> Recently, Infra disabled force-pushes (and non-fast-forward pushes) to all
> branches to avoid accidental overwrites or deletions.
>
> I propose we reach out to Infra and ask for an exemption since our workflow
> for feature branches involves deletions and force-pushes.
>
I asked them for this exemption for all branches called feature/* earlier today; its consistent with the git flow branch naming.
if all feature branches go in under there, people working on them are free to rebase as they choose.
> We should likely wait for a day or so to hear any concerns against this
> request. Also, can someone volunteer following up on this? I am going away
> on vacation shortly, and will have limited access to internet/email.
>