You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sergey Shelukhin <se...@hortonworks.com> on 2018/05/03 19:40:23 UTC
Re: switching HiveQA to manual submission?
Ping? I just had 3-4 JIRAs again disappear from the queue mysteriously, for the 2nd or 3rd time in one week.
Given that there doesn’t appear to be major interest in improving the script in the community (other than some minor changes), should we just nuke it and have manual submission w/2 fields, a patch file and jira number? That way the tracking at least can be improved. And the patch choice to run or not run.
This would take less time than trying to appease the fickle spirits of HiveQA as it is now. And also probably result in somewhat shorter queues.
From: Sergey Shelukhin <se...@hortonworks.com>>
Date: Thursday, April 26, 2018 at 17:57
To: "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
Subject: switching HiveQA to manual submission?
Given the constant cluster.... with HiveQA, namely queue constantly losing or not picking up patches for whatever reason (not limited to dedup - I uploaded some patches today where there’s only one attached to the JIRA and still it didn’t get in the queue or got removed) I wonder if we should add manual submission option to HiveQA. Or make it the only option actually.
That way one can target a specific file and get a specific job link back that won’t got anywhere (one can hope)...
Re: switching HiveQA to manual submission?
Posted by Vihang Karajgaonkar <vi...@cloudera.com>.
The builds team acknowledged that for now on unscheduled/scheduled outages
will be emailed to builds@apache.org. Please subscribe to that list in case
you are interested to know about jenkins outages. This page
https://status.apache.org/ also reports the current status and past
incidents.
Thanks,
Vihang
On Thu, May 3, 2018 at 4:35 PM, Vihang Karajgaonkar <vi...@cloudera.com>
wrote:
> In case you didn't see the latest comment on the JIRA, here is the
> response I got from the builds team:
>
> Unfortunately we've been running into some bugs with Jenkins which is
>> causing us to restart the server. We're in the early stages of migrating to
>> a new host which we're confident will solve most of the issues we're seeing
>> now. Watch the builds@apache.org list https://lists.apache.org/list.
>> html?builds@apache.org for more information about the process going
>> forward. We'll also try and tweet about when we have problems and are
>> investigating them: https://twitter.com/infrabot
>>
>
> On Thu, May 3, 2018 at 2:41 PM, Vihang Karajgaonkar <vi...@cloudera.com>
> wrote:
>
>> I had sent an email to builds@apache.org (not sure if that is the right
>> list). Also created this JIRA https://issues.apache.org
>> /jira/browse/INFRA-16478 to see if there is some response.
>>
>> On Thu, May 3, 2018 at 2:21 PM, Vihang Karajgaonkar <vi...@cloudera.com>
>> wrote:
>>
>>> I saw patches getting disappeared also. I know the patch for HIVE-17824
>>> was not run since last 1 week. When we looked into this yesterday I found
>>> that for some reason the jenkins server was restarted overnight and we lose
>>> the pending queue. Does anyone know why this might be happening? Does
>>> jenkins restart happen if the pending queue is long enough or they really
>>> had some scheduled maintenance? If its a maintenance thing we should be
>>> able to find out the mailing list which informs users of this.
>>>
>>> Even if you submit the job manually, the current ptest is only able to
>>> serve one job at a time. We will still end up with long pending queue and
>>> any jenkins restart will drain the queue again. I think we need general
>>> improvements to Ptest to speed things up. The testing infrastructure is not
>>> able to keep up with the number of patches. Any suggestions to improve this
>>> situation are welcome but I don't know if submitting manually is really
>>> going to help with the problem.
>>>
>>>
>>> On Thu, May 3, 2018 at 12:53 PM, Кривенко Ігор <kr...@gmail.com>
>>> wrote:
>>>
>>>> My JIRAs also got disappeared.
>>>> But, what other developers which have no access to submit manually on
>>>> builds.apache.org will be doing?
>>>>
>>>> Thanks, Igor.
>>>>
>>>> 2018-05-03 22:47 GMT+03:00 Prasanth Jayachandran <
>>>> pjayachandran@hortonworks.com>:
>>>>
>>>> > My JIRAs also got disappeared mysteriously and had to manually submit
>>>> the
>>>> > patch.
>>>> > +1 on making it manual (atleast this will make the patch not
>>>> disappear and
>>>> > is more predictable/no babysitting).
>>>> >
>>>> > Thanks
>>>> > Prasanth
>>>> >
>>>> > > On May 3, 2018, at 12:40 PM, Sergey Shelukhin <
>>>> sergey@hortonworks.com>
>>>> > wrote:
>>>> > >
>>>> > > Ping? I just had 3-4 JIRAs again disappear from the queue
>>>> mysteriously,
>>>> > for the 2nd or 3rd time in one week.
>>>> > >
>>>> > > Given that there doesn’t appear to be major interest in improving
>>>> the
>>>> > script in the community (other than some minor changes), should we
>>>> just
>>>> > nuke it and have manual submission w/2 fields, a patch file and jira
>>>> > number? That way the tracking at least can be improved. And the patch
>>>> > choice to run or not run.
>>>> > > This would take less time than trying to appease the fickle spirits
>>>> of
>>>> > HiveQA as it is now. And also probably result in somewhat shorter
>>>> queues.
>>>> > >
>>>> > > From: Sergey Shelukhin <sergey@hortonworks.com<mailto:
>>>> > sergey@hortonworks.com>>
>>>> > > Date: Thursday, April 26, 2018 at 17:57
>>>> > > To: "dev@hive.apache.org<ma...@hive.apache.org>" <
>>>> > dev@hive.apache.org<ma...@hive.apache.org>>
>>>> > > Subject: switching HiveQA to manual submission?
>>>> > >
>>>> > > Given the constant cluster.... with HiveQA, namely queue constantly
>>>> > losing or not picking up patches for whatever reason (not limited to
>>>> dedup
>>>> > - I uploaded some patches today where there’s only one attached to
>>>> the JIRA
>>>> > and still it didn’t get in the queue or got removed) I wonder if we
>>>> should
>>>> > add manual submission option to HiveQA. Or make it the only option
>>>> actually.
>>>> > > That way one can target a specific file and get a specific job link
>>>> back
>>>> > that won’t got anywhere (one can hope)...
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>
Re: switching HiveQA to manual submission?
Posted by Vihang Karajgaonkar <vi...@cloudera.com>.
In case you didn't see the latest comment on the JIRA, here is the response
I got from the builds team:
Unfortunately we've been running into some bugs with Jenkins which is
> causing us to restart the server. We're in the early stages of migrating to
> a new host which we're confident will solve most of the issues we're seeing
> now. Watch the builds@apache.org list
> https://lists.apache.org/list.html?builds@apache.org for more information
> about the process going forward. We'll also try and tweet about when we
> have problems and are investigating them: https://twitter.com/infrabot
>
On Thu, May 3, 2018 at 2:41 PM, Vihang Karajgaonkar <vi...@cloudera.com>
wrote:
> I had sent an email to builds@apache.org (not sure if that is the right
> list). Also created this JIRA https://issues.apache.
> org/jira/browse/INFRA-16478 to see if there is some response.
>
> On Thu, May 3, 2018 at 2:21 PM, Vihang Karajgaonkar <vi...@cloudera.com>
> wrote:
>
>> I saw patches getting disappeared also. I know the patch for HIVE-17824
>> was not run since last 1 week. When we looked into this yesterday I found
>> that for some reason the jenkins server was restarted overnight and we lose
>> the pending queue. Does anyone know why this might be happening? Does
>> jenkins restart happen if the pending queue is long enough or they really
>> had some scheduled maintenance? If its a maintenance thing we should be
>> able to find out the mailing list which informs users of this.
>>
>> Even if you submit the job manually, the current ptest is only able to
>> serve one job at a time. We will still end up with long pending queue and
>> any jenkins restart will drain the queue again. I think we need general
>> improvements to Ptest to speed things up. The testing infrastructure is not
>> able to keep up with the number of patches. Any suggestions to improve this
>> situation are welcome but I don't know if submitting manually is really
>> going to help with the problem.
>>
>>
>> On Thu, May 3, 2018 at 12:53 PM, Кривенко Ігор <kr...@gmail.com>
>> wrote:
>>
>>> My JIRAs also got disappeared.
>>> But, what other developers which have no access to submit manually on
>>> builds.apache.org will be doing?
>>>
>>> Thanks, Igor.
>>>
>>> 2018-05-03 22:47 GMT+03:00 Prasanth Jayachandran <
>>> pjayachandran@hortonworks.com>:
>>>
>>> > My JIRAs also got disappeared mysteriously and had to manually submit
>>> the
>>> > patch.
>>> > +1 on making it manual (atleast this will make the patch not disappear
>>> and
>>> > is more predictable/no babysitting).
>>> >
>>> > Thanks
>>> > Prasanth
>>> >
>>> > > On May 3, 2018, at 12:40 PM, Sergey Shelukhin <
>>> sergey@hortonworks.com>
>>> > wrote:
>>> > >
>>> > > Ping? I just had 3-4 JIRAs again disappear from the queue
>>> mysteriously,
>>> > for the 2nd or 3rd time in one week.
>>> > >
>>> > > Given that there doesn’t appear to be major interest in improving the
>>> > script in the community (other than some minor changes), should we just
>>> > nuke it and have manual submission w/2 fields, a patch file and jira
>>> > number? That way the tracking at least can be improved. And the patch
>>> > choice to run or not run.
>>> > > This would take less time than trying to appease the fickle spirits
>>> of
>>> > HiveQA as it is now. And also probably result in somewhat shorter
>>> queues.
>>> > >
>>> > > From: Sergey Shelukhin <sergey@hortonworks.com<mailto:
>>> > sergey@hortonworks.com>>
>>> > > Date: Thursday, April 26, 2018 at 17:57
>>> > > To: "dev@hive.apache.org<ma...@hive.apache.org>" <
>>> > dev@hive.apache.org<ma...@hive.apache.org>>
>>> > > Subject: switching HiveQA to manual submission?
>>> > >
>>> > > Given the constant cluster.... with HiveQA, namely queue constantly
>>> > losing or not picking up patches for whatever reason (not limited to
>>> dedup
>>> > - I uploaded some patches today where there’s only one attached to the
>>> JIRA
>>> > and still it didn’t get in the queue or got removed) I wonder if we
>>> should
>>> > add manual submission option to HiveQA. Or make it the only option
>>> actually.
>>> > > That way one can target a specific file and get a specific job link
>>> back
>>> > that won’t got anywhere (one can hope)...
>>> >
>>> >
>>>
>>
>>
>
Re: switching HiveQA to manual submission?
Posted by Vihang Karajgaonkar <vi...@cloudera.com>.
I had sent an email to builds@apache.org (not sure if that is the right
list). Also created this JIRA
https://issues.apache.org/jira/browse/INFRA-16478 to see if there is some
response.
On Thu, May 3, 2018 at 2:21 PM, Vihang Karajgaonkar <vi...@cloudera.com>
wrote:
> I saw patches getting disappeared also. I know the patch for HIVE-17824
> was not run since last 1 week. When we looked into this yesterday I found
> that for some reason the jenkins server was restarted overnight and we lose
> the pending queue. Does anyone know why this might be happening? Does
> jenkins restart happen if the pending queue is long enough or they really
> had some scheduled maintenance? If its a maintenance thing we should be
> able to find out the mailing list which informs users of this.
>
> Even if you submit the job manually, the current ptest is only able to
> serve one job at a time. We will still end up with long pending queue and
> any jenkins restart will drain the queue again. I think we need general
> improvements to Ptest to speed things up. The testing infrastructure is not
> able to keep up with the number of patches. Any suggestions to improve this
> situation are welcome but I don't know if submitting manually is really
> going to help with the problem.
>
>
> On Thu, May 3, 2018 at 12:53 PM, Кривенко Ігор <kr...@gmail.com>
> wrote:
>
>> My JIRAs also got disappeared.
>> But, what other developers which have no access to submit manually on
>> builds.apache.org will be doing?
>>
>> Thanks, Igor.
>>
>> 2018-05-03 22:47 GMT+03:00 Prasanth Jayachandran <
>> pjayachandran@hortonworks.com>:
>>
>> > My JIRAs also got disappeared mysteriously and had to manually submit
>> the
>> > patch.
>> > +1 on making it manual (atleast this will make the patch not disappear
>> and
>> > is more predictable/no babysitting).
>> >
>> > Thanks
>> > Prasanth
>> >
>> > > On May 3, 2018, at 12:40 PM, Sergey Shelukhin <sergey@hortonworks.com
>> >
>> > wrote:
>> > >
>> > > Ping? I just had 3-4 JIRAs again disappear from the queue
>> mysteriously,
>> > for the 2nd or 3rd time in one week.
>> > >
>> > > Given that there doesn’t appear to be major interest in improving the
>> > script in the community (other than some minor changes), should we just
>> > nuke it and have manual submission w/2 fields, a patch file and jira
>> > number? That way the tracking at least can be improved. And the patch
>> > choice to run or not run.
>> > > This would take less time than trying to appease the fickle spirits of
>> > HiveQA as it is now. And also probably result in somewhat shorter
>> queues.
>> > >
>> > > From: Sergey Shelukhin <sergey@hortonworks.com<mailto:
>> > sergey@hortonworks.com>>
>> > > Date: Thursday, April 26, 2018 at 17:57
>> > > To: "dev@hive.apache.org<ma...@hive.apache.org>" <
>> > dev@hive.apache.org<ma...@hive.apache.org>>
>> > > Subject: switching HiveQA to manual submission?
>> > >
>> > > Given the constant cluster.... with HiveQA, namely queue constantly
>> > losing or not picking up patches for whatever reason (not limited to
>> dedup
>> > - I uploaded some patches today where there’s only one attached to the
>> JIRA
>> > and still it didn’t get in the queue or got removed) I wonder if we
>> should
>> > add manual submission option to HiveQA. Or make it the only option
>> actually.
>> > > That way one can target a specific file and get a specific job link
>> back
>> > that won’t got anywhere (one can hope)...
>> >
>> >
>>
>
>
Re: switching HiveQA to manual submission?
Posted by Vihang Karajgaonkar <vi...@cloudera.com>.
I saw patches getting disappeared also. I know the patch for HIVE-17824 was
not run since last 1 week. When we looked into this yesterday I found that
for some reason the jenkins server was restarted overnight and we lose the
pending queue. Does anyone know why this might be happening? Does jenkins
restart happen if the pending queue is long enough or they really had some
scheduled maintenance? If its a maintenance thing we should be able to find
out the mailing list which informs users of this.
Even if you submit the job manually, the current ptest is only able to
serve one job at a time. We will still end up with long pending queue and
any jenkins restart will drain the queue again. I think we need general
improvements to Ptest to speed things up. The testing infrastructure is not
able to keep up with the number of patches. Any suggestions to improve this
situation are welcome but I don't know if submitting manually is really
going to help with the problem.
On Thu, May 3, 2018 at 12:53 PM, Кривенко Ігор <kr...@gmail.com>
wrote:
> My JIRAs also got disappeared.
> But, what other developers which have no access to submit manually on
> builds.apache.org will be doing?
>
> Thanks, Igor.
>
> 2018-05-03 22:47 GMT+03:00 Prasanth Jayachandran <
> pjayachandran@hortonworks.com>:
>
> > My JIRAs also got disappeared mysteriously and had to manually submit the
> > patch.
> > +1 on making it manual (atleast this will make the patch not disappear
> and
> > is more predictable/no babysitting).
> >
> > Thanks
> > Prasanth
> >
> > > On May 3, 2018, at 12:40 PM, Sergey Shelukhin <se...@hortonworks.com>
> > wrote:
> > >
> > > Ping? I just had 3-4 JIRAs again disappear from the queue mysteriously,
> > for the 2nd or 3rd time in one week.
> > >
> > > Given that there doesn’t appear to be major interest in improving the
> > script in the community (other than some minor changes), should we just
> > nuke it and have manual submission w/2 fields, a patch file and jira
> > number? That way the tracking at least can be improved. And the patch
> > choice to run or not run.
> > > This would take less time than trying to appease the fickle spirits of
> > HiveQA as it is now. And also probably result in somewhat shorter queues.
> > >
> > > From: Sergey Shelukhin <sergey@hortonworks.com<mailto:
> > sergey@hortonworks.com>>
> > > Date: Thursday, April 26, 2018 at 17:57
> > > To: "dev@hive.apache.org<ma...@hive.apache.org>" <
> > dev@hive.apache.org<ma...@hive.apache.org>>
> > > Subject: switching HiveQA to manual submission?
> > >
> > > Given the constant cluster.... with HiveQA, namely queue constantly
> > losing or not picking up patches for whatever reason (not limited to
> dedup
> > - I uploaded some patches today where there’s only one attached to the
> JIRA
> > and still it didn’t get in the queue or got removed) I wonder if we
> should
> > add manual submission option to HiveQA. Or make it the only option
> actually.
> > > That way one can target a specific file and get a specific job link
> back
> > that won’t got anywhere (one can hope)...
> >
> >
>
Re: switching HiveQA to manual submission?
Posted by Кривенко Ігор <kr...@gmail.com>.
My JIRAs also got disappeared.
But, what other developers which have no access to submit manually on
builds.apache.org will be doing?
Thanks, Igor.
2018-05-03 22:47 GMT+03:00 Prasanth Jayachandran <
pjayachandran@hortonworks.com>:
> My JIRAs also got disappeared mysteriously and had to manually submit the
> patch.
> +1 on making it manual (atleast this will make the patch not disappear and
> is more predictable/no babysitting).
>
> Thanks
> Prasanth
>
> > On May 3, 2018, at 12:40 PM, Sergey Shelukhin <se...@hortonworks.com>
> wrote:
> >
> > Ping? I just had 3-4 JIRAs again disappear from the queue mysteriously,
> for the 2nd or 3rd time in one week.
> >
> > Given that there doesn’t appear to be major interest in improving the
> script in the community (other than some minor changes), should we just
> nuke it and have manual submission w/2 fields, a patch file and jira
> number? That way the tracking at least can be improved. And the patch
> choice to run or not run.
> > This would take less time than trying to appease the fickle spirits of
> HiveQA as it is now. And also probably result in somewhat shorter queues.
> >
> > From: Sergey Shelukhin <sergey@hortonworks.com<mailto:
> sergey@hortonworks.com>>
> > Date: Thursday, April 26, 2018 at 17:57
> > To: "dev@hive.apache.org<ma...@hive.apache.org>" <
> dev@hive.apache.org<ma...@hive.apache.org>>
> > Subject: switching HiveQA to manual submission?
> >
> > Given the constant cluster.... with HiveQA, namely queue constantly
> losing or not picking up patches for whatever reason (not limited to dedup
> - I uploaded some patches today where there’s only one attached to the JIRA
> and still it didn’t get in the queue or got removed) I wonder if we should
> add manual submission option to HiveQA. Or make it the only option actually.
> > That way one can target a specific file and get a specific job link back
> that won’t got anywhere (one can hope)...
>
>
Re: switching HiveQA to manual submission?
Posted by Prasanth Jayachandran <pj...@hortonworks.com>.
My JIRAs also got disappeared mysteriously and had to manually submit the patch.
+1 on making it manual (atleast this will make the patch not disappear and is more predictable/no babysitting).
Thanks
Prasanth
> On May 3, 2018, at 12:40 PM, Sergey Shelukhin <se...@hortonworks.com> wrote:
>
> Ping? I just had 3-4 JIRAs again disappear from the queue mysteriously, for the 2nd or 3rd time in one week.
>
> Given that there doesn’t appear to be major interest in improving the script in the community (other than some minor changes), should we just nuke it and have manual submission w/2 fields, a patch file and jira number? That way the tracking at least can be improved. And the patch choice to run or not run.
> This would take less time than trying to appease the fickle spirits of HiveQA as it is now. And also probably result in somewhat shorter queues.
>
> From: Sergey Shelukhin <se...@hortonworks.com>>
> Date: Thursday, April 26, 2018 at 17:57
> To: "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
> Subject: switching HiveQA to manual submission?
>
> Given the constant cluster.... with HiveQA, namely queue constantly losing or not picking up patches for whatever reason (not limited to dedup - I uploaded some patches today where there’s only one attached to the JIRA and still it didn’t get in the queue or got removed) I wonder if we should add manual submission option to HiveQA. Or make it the only option actually.
> That way one can target a specific file and get a specific job link back that won’t got anywhere (one can hope)...