You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yetus.apache.org by Ted Yu <yu...@gmail.com> on 2017/11/27 01:48:25 UTC
precommit stalling across projects
Hi,
Looks like several Apache projects are not able to run QA jobs.
Here was an example:
https://builds.apache.org/job/PreCommit-YARN-Build/18663/console
hbase is also affected.
bq. ERROR: Unsure how to process YARN-6507.
The above came from precommit/core.d/patchfiles.sh
I am not familiar with Yetus codebase.
Can someone shed some light on the cause ?
Thanks
Re: precommit stalling across projects
Posted by Stack <st...@duboce.net>.
Thanks Allen.
S
On Mon, Nov 27, 2017 at 11:31 AM, Allen Wittenauer <aw@effectivemachines.com
> wrote:
>
> > On Nov 27, 2017, at 11:00 AM, Stack <st...@duboce.net> wrote:
> >
> > Mighty Allen:
> >
> > Do you know if there is anything special about hadoopqa account? You have
> > pointer to how we could setup an account of our own to do hbaseprecommit?
> > (I can write up a little bit of doc for yetus on how to.... after I go
> > through process).
>
>
> Before you do that, you might want to follow along INFRA-15558.
> There’s a bit of discussion around “how do we avoid this mess in the
> future” and some of that may dictate what people should probably be doing.
> I think in the end the recommendation will be “use ${projectname}qa” though.
Re: precommit stalling across projects
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Nov 27, 2017, at 11:00 AM, Stack <st...@duboce.net> wrote:
>
> Mighty Allen:
>
> Do you know if there is anything special about hadoopqa account? You have
> pointer to how we could setup an account of our own to do hbaseprecommit?
> (I can write up a little bit of doc for yetus on how to.... after I go
> through process).
Before you do that, you might want to follow along INFRA-15558. There’s a bit of discussion around “how do we avoid this mess in the future” and some of that may dictate what people should probably be doing. I think in the end the recommendation will be “use ${projectname}qa” though.
Re: precommit stalling across projects
Posted by Stack <st...@duboce.net>.
Mighty Allen:
Do you know if there is anything special about hadoopqa account? You have
pointer to how we could setup an account of our own to do hbaseprecommit?
(I can write up a little bit of doc for yetus on how to.... after I go
through process).
Thanks boss,
S
On Mon, Nov 27, 2017 at 10:34 AM, Allen Wittenauer <aw@effectivemachines.com
> wrote:
>
> > On Nov 27, 2017, at 7:49 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> > QA bot is back.
> >
> > Thanks Allan.
>
> You’re welcome, but probably premature. ;)
>
> Hadoop is no longer using the ‘hadoopqa’ account. Instead, it is
> now shared among:
>
> ACCUMULO
> AMBARI
> AVRO
> BOOKKEEPER
> HBASE
> HIVE
> GIRAPH
> KNOX
> LENS
> NIFI
> OOZIE
> ORC
> PHOENIX
> RATIS
> SENTRY
> TEZ
> ZOOKEEPER
>
> Hopefully it is a) obvious why this is bad b) everyone can
> appreciate the irony.
>
> At some point there will likely be a ‘day of reckoning’ where
> hadoopqa is turned off and all of these projects will fail until they set
> up their own JIRA account for their precommit results. A lot of this setup
> pre-dates Yetus as a TLP, but it’s probably on our collective plate to work
> up better documentation so that people don’t just clone existing jobs.
>
> We should probably also make more of an effort to pull
> precommit-admin into the Yetus source tree so that we can fix things like
> the global jenkins token.
Re: precommit stalling across projects
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Nov 27, 2017, at 7:49 AM, Ted Yu <yu...@gmail.com> wrote:
>
> QA bot is back.
>
> Thanks Allan.
You’re welcome, but probably premature. ;)
Hadoop is no longer using the ‘hadoopqa’ account. Instead, it is now shared among:
ACCUMULO
AMBARI
AVRO
BOOKKEEPER
HBASE
HIVE
GIRAPH
KNOX
LENS
NIFI
OOZIE
ORC
PHOENIX
RATIS
SENTRY
TEZ
ZOOKEEPER
Hopefully it is a) obvious why this is bad b) everyone can appreciate the irony.
At some point there will likely be a ‘day of reckoning’ where hadoopqa is turned off and all of these projects will fail until they set up their own JIRA account for their precommit results. A lot of this setup pre-dates Yetus as a TLP, but it’s probably on our collective plate to work up better documentation so that people don’t just clone existing jobs.
We should probably also make more of an effort to pull precommit-admin into the Yetus source tree so that we can fix things like the global jenkins token.
Re: precommit stalling across projects
Posted by Ted Yu <yu...@gmail.com>.
QA bot is back.
Thanks Allan.
On Sun, Nov 26, 2017 at 6:33 PM, Allen Wittenauer <aw...@effectivemachines.com>
wrote:
>
> > On Nov 26, 2017, at 6:29 PM, Allen Wittenauer <aw...@effectivemachines.com>
> wrote:
> >
> > For now, I’m thinking we should just keep it locked. Let’s see
> what crawls out of the woodwork. I’ll set up a yetusqa account later.
>
> Ha. I forgot. We already have a yetusqa account for our stuff.
>
>
>
Re: precommit stalling across projects
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Nov 26, 2017, at 6:29 PM, Allen Wittenauer <aw...@effectivemachines.com> wrote:
>
> For now, I’m thinking we should just keep it locked. Let’s see what crawls out of the woodwork. I’ll set up a yetusqa account later.
Ha. I forgot. We already have a yetusqa account for our stuff.
Re: precommit stalling across projects
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Nov 26, 2017, at 6:24 PM, Ted Yu <yu...@gmail.com> wrote:
>
> bq. screwed up the hadoopqa account and locked it out of JIRA
>
> Should an INFRA ticket be filed ?
For now, I’m thinking we should just keep it locked. Let’s see what crawls out of the woodwork. I’ll set up a yetusqa account later.
Re: precommit stalling across projects
Posted by Ted Yu <yu...@gmail.com>.
bq. screwed up the hadoopqa account and locked it out of JIRA
Should an INFRA ticket be filed ?
Cheers
On Sun, Nov 26, 2017 at 6:20 PM, Allen Wittenauer <aw...@effectivemachines.com>
wrote:
>
> > On Nov 26, 2017, at 5:48 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > Hi,
> > Looks like several Apache projects are not able to run QA jobs.
> >
> > Here was an example:
> > https://builds.apache.org/job/PreCommit-YARN-Build/18663/console
> >
> > hbase is also affected.
>
> ...
>
> > Can someone shed some light on the cause ?
>
> Given the data provided and without looking real hard, this
> usually means someone screwed up the hadoopqa account and locked it out of
> JIRA. Ideally, each project would create their own but given the number
> of projects that also just copy the hadoop precommit jobs blindly without
> understanding what is going on …
>
> I have no idea who actually ‘owns’ the hadoopqa account to
> unlock it. Probably Nigel. (I’d frankly be surprised if the Hadoop PMC
> does). I’m tempted to create a yetusqa account, give ownership to it to the
> yetus PMC, and then put that into (at least) the precommit-hadoop-build
> job. This way when folks just copy it they will also use yetusqa and make
> it easier for us to unlock. Might also be useful from a stats perspective?
>
>
>
Re: precommit stalling across projects
Posted by Ted Yu <yu...@gmail.com>.
QA bot is working again.
I triggered build for HBASE-19338 since it is high priority.
Please re-attach patch for QA to test your patch.
Cheers
On Sun, Nov 26, 2017 at 6:34 PM, Ted Yu <yu...@gmail.com> wrote:
> For those of you waiting for QA bot to come back, see the following thread.
>
> Allen advised waiting before logging INFRA ticket.
>
> FYI
>
> ---------- Forwarded message ----------
> From: Allen Wittenauer <aw...@effectivemachines.com>
> Date: Sun, Nov 26, 2017 at 6:28 PM
> Subject: Re: precommit stalling across projects
> To: dev@yetus.apache.org
>
>
>
> > On Nov 26, 2017, at 6:20 PM, Allen Wittenauer <aw...@effectivemachines.com>
> wrote:
> >
> >
> > Given the data provided and without looking real hard, this
> usually means someone screwed up the hadoopqa account and locked it out of
> JIRA.
>
> Yup, it’s locked. Tried from the UI. Hit the reset link. Maybe
> someone we know will get it.
>
> > I’m tempted to create a yetusqa account, give ownership to it to the
> yetus PMC, and then put that into (at least) the precommit-hadoop-build job.
>
> Looks like builds.apache.org is undergoing maintenance so no
> point in spending much time on this at the moment. I’ll check
> later/tomorrow.
>
>
>
>
Fwd: precommit stalling across projects
Posted by Ted Yu <yu...@gmail.com>.
For those of you waiting for QA bot to come back, see the following thread.
Allen advised waiting before logging INFRA ticket.
FYI
---------- Forwarded message ----------
From: Allen Wittenauer <aw...@effectivemachines.com>
Date: Sun, Nov 26, 2017 at 6:28 PM
Subject: Re: precommit stalling across projects
To: dev@yetus.apache.org
> On Nov 26, 2017, at 6:20 PM, Allen Wittenauer <aw...@effectivemachines.com>
wrote:
>
>
> Given the data provided and without looking real hard, this usually
means someone screwed up the hadoopqa account and locked it out of JIRA.
Yup, it’s locked. Tried from the UI. Hit the reset link. Maybe
someone we know will get it.
> I’m tempted to create a yetusqa account, give ownership to it to the
yetus PMC, and then put that into (at least) the precommit-hadoop-build job.
Looks like builds.apache.org is undergoing maintenance so no point
in spending much time on this at the moment. I’ll check later/tomorrow.
Re: precommit stalling across projects
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Nov 26, 2017, at 6:20 PM, Allen Wittenauer <aw...@effectivemachines.com> wrote:
>
>
> Given the data provided and without looking real hard, this usually means someone screwed up the hadoopqa account and locked it out of JIRA.
Yup, it’s locked. Tried from the UI. Hit the reset link. Maybe someone we know will get it.
> I’m tempted to create a yetusqa account, give ownership to it to the yetus PMC, and then put that into (at least) the precommit-hadoop-build job.
Looks like builds.apache.org is undergoing maintenance so no point in spending much time on this at the moment. I’ll check later/tomorrow.
Re: precommit stalling across projects
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Nov 26, 2017, at 5:48 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Hi,
> Looks like several Apache projects are not able to run QA jobs.
>
> Here was an example:
> https://builds.apache.org/job/PreCommit-YARN-Build/18663/console
>
> hbase is also affected.
...
> Can someone shed some light on the cause ?
Given the data provided and without looking real hard, this usually means someone screwed up the hadoopqa account and locked it out of JIRA. Ideally, each project would create their own but given the number of projects that also just copy the hadoop precommit jobs blindly without understanding what is going on …
I have no idea who actually ‘owns’ the hadoopqa account to unlock it. Probably Nigel. (I’d frankly be surprised if the Hadoop PMC does). I’m tempted to create a yetusqa account, give ownership to it to the yetus PMC, and then put that into (at least) the precommit-hadoop-build job. This way when folks just copy it they will also use yetusqa and make it easier for us to unlock. Might also be useful from a stats perspective?