You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Brock Noland <br...@cloudera.com> on 2014/02/21 16:11:21 UTC
Precommit queue
There was a ec2 spot price spike overnight which combined with
everyone trying to get patches in for the branching has resulted in a
massive queue:
http://bigtop01.cloudera.org:8080/view/Hive/job/PreCommit-HIVE-Build/
~25 builds in the queue
Brock
Re: Precommit queue
Posted by Brock Noland <br...@cloudera.com>.
FYI the changes 1) excluding TestWithBeelineArgs 2) not copying the
source dir on failed tests has improved run times by ~1.5 hours.
On Sat, Feb 22, 2014 at 10:34 AM, Brock Noland <br...@cloudera.com> wrote:
> Looking more into that test...I decided to skip it and open a blocker
> to look at it: https://issues.apache.org/jira/browse/HIVE-6488
>
> On Sat, Feb 22, 2014 at 10:26 AM, Brock Noland <br...@cloudera.com> wrote:
>> In other news, TestWithBeelineArgs created a 9GB hive.log, before
>> failing, in the last run:
>>
>> http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-1440/failed/TestBeeLineWithArgs/
>>
>> On Sat, Feb 22, 2014 at 10:23 AM, Brock Noland <br...@cloudera.com> wrote:
>>> Prices have been stable but do they do seem to be having capacity
>>> problems[1] as I have been unable to get machines when bidding
>>> $0.70/hr which is 10x the normal price. However, there is one item
>>> which is taking time for no good reason:
>>> https://issues.apache.org/jira/browse/HIVE-6487. I am going to put a
>>> break fix in for that this morning.
>>>
>>> Brock
>>>
>>> [1] An economically inclined observer would expect lower capacity to
>>> result in increased prices but that is not always the case. Whatever
>>> black magic they have running the spot market seems to have a lot of
>>> non-market logic.
>>>
>>>
>>> On Fri, Feb 21, 2014 at 10:40 PM, Thejas Nair <th...@hortonworks.com> wrote:
>>>> Hi Brock,
>>>> Do you know why the tests are taking almost twice as long in recent runs ?
>>>> Is it related to the ec2 spot price spikes ?
>>>> Thanks,
>>>> Thejas
>>>>
>>>>
>>>> On Fri, Feb 21, 2014 at 7:11 AM, Brock Noland <br...@cloudera.com> wrote:
>>>>
>>>>> There was a ec2 spot price spike overnight which combined with
>>>>> everyone trying to get patches in for the branching has resulted in a
>>>>> massive queue:
>>>>>
>>>>> http://bigtop01.cloudera.org:8080/view/Hive/job/PreCommit-HIVE-Build/
>>>>>
>>>>> ~25 builds in the queue
>>>>>
>>>>> Brock
>>>>>
>>>>
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender immediately
>>>> and delete it from your system. Thank You.
>>>
>>>
>>>
>>> --
>>> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: Precommit queue
Posted by Brock Noland <br...@cloudera.com>.
Looking more into that test...I decided to skip it and open a blocker
to look at it: https://issues.apache.org/jira/browse/HIVE-6488
On Sat, Feb 22, 2014 at 10:26 AM, Brock Noland <br...@cloudera.com> wrote:
> In other news, TestWithBeelineArgs created a 9GB hive.log, before
> failing, in the last run:
>
> http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-1440/failed/TestBeeLineWithArgs/
>
> On Sat, Feb 22, 2014 at 10:23 AM, Brock Noland <br...@cloudera.com> wrote:
>> Prices have been stable but do they do seem to be having capacity
>> problems[1] as I have been unable to get machines when bidding
>> $0.70/hr which is 10x the normal price. However, there is one item
>> which is taking time for no good reason:
>> https://issues.apache.org/jira/browse/HIVE-6487. I am going to put a
>> break fix in for that this morning.
>>
>> Brock
>>
>> [1] An economically inclined observer would expect lower capacity to
>> result in increased prices but that is not always the case. Whatever
>> black magic they have running the spot market seems to have a lot of
>> non-market logic.
>>
>>
>> On Fri, Feb 21, 2014 at 10:40 PM, Thejas Nair <th...@hortonworks.com> wrote:
>>> Hi Brock,
>>> Do you know why the tests are taking almost twice as long in recent runs ?
>>> Is it related to the ec2 spot price spikes ?
>>> Thanks,
>>> Thejas
>>>
>>>
>>> On Fri, Feb 21, 2014 at 7:11 AM, Brock Noland <br...@cloudera.com> wrote:
>>>
>>>> There was a ec2 spot price spike overnight which combined with
>>>> everyone trying to get patches in for the branching has resulted in a
>>>> massive queue:
>>>>
>>>> http://bigtop01.cloudera.org:8080/view/Hive/job/PreCommit-HIVE-Build/
>>>>
>>>> ~25 builds in the queue
>>>>
>>>> Brock
>>>>
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: Precommit queue
Posted by Brock Noland <br...@cloudera.com>.
In other news, TestWithBeelineArgs created a 9GB hive.log, before
failing, in the last run:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-1440/failed/TestBeeLineWithArgs/
On Sat, Feb 22, 2014 at 10:23 AM, Brock Noland <br...@cloudera.com> wrote:
> Prices have been stable but do they do seem to be having capacity
> problems[1] as I have been unable to get machines when bidding
> $0.70/hr which is 10x the normal price. However, there is one item
> which is taking time for no good reason:
> https://issues.apache.org/jira/browse/HIVE-6487. I am going to put a
> break fix in for that this morning.
>
> Brock
>
> [1] An economically inclined observer would expect lower capacity to
> result in increased prices but that is not always the case. Whatever
> black magic they have running the spot market seems to have a lot of
> non-market logic.
>
>
> On Fri, Feb 21, 2014 at 10:40 PM, Thejas Nair <th...@hortonworks.com> wrote:
>> Hi Brock,
>> Do you know why the tests are taking almost twice as long in recent runs ?
>> Is it related to the ec2 spot price spikes ?
>> Thanks,
>> Thejas
>>
>>
>> On Fri, Feb 21, 2014 at 7:11 AM, Brock Noland <br...@cloudera.com> wrote:
>>
>>> There was a ec2 spot price spike overnight which combined with
>>> everyone trying to get patches in for the branching has resulted in a
>>> massive queue:
>>>
>>> http://bigtop01.cloudera.org:8080/view/Hive/job/PreCommit-HIVE-Build/
>>>
>>> ~25 builds in the queue
>>>
>>> Brock
>>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: Precommit queue
Posted by Brock Noland <br...@cloudera.com>.
Prices have been stable but do they do seem to be having capacity
problems[1] as I have been unable to get machines when bidding
$0.70/hr which is 10x the normal price. However, there is one item
which is taking time for no good reason:
https://issues.apache.org/jira/browse/HIVE-6487. I am going to put a
break fix in for that this morning.
Brock
[1] An economically inclined observer would expect lower capacity to
result in increased prices but that is not always the case. Whatever
black magic they have running the spot market seems to have a lot of
non-market logic.
On Fri, Feb 21, 2014 at 10:40 PM, Thejas Nair <th...@hortonworks.com> wrote:
> Hi Brock,
> Do you know why the tests are taking almost twice as long in recent runs ?
> Is it related to the ec2 spot price spikes ?
> Thanks,
> Thejas
>
>
> On Fri, Feb 21, 2014 at 7:11 AM, Brock Noland <br...@cloudera.com> wrote:
>
>> There was a ec2 spot price spike overnight which combined with
>> everyone trying to get patches in for the branching has resulted in a
>> massive queue:
>>
>> http://bigtop01.cloudera.org:8080/view/Hive/job/PreCommit-HIVE-Build/
>>
>> ~25 builds in the queue
>>
>> Brock
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: Precommit queue
Posted by Thejas Nair <th...@hortonworks.com>.
Hi Brock,
Do you know why the tests are taking almost twice as long in recent runs ?
Is it related to the ec2 spot price spikes ?
Thanks,
Thejas
On Fri, Feb 21, 2014 at 7:11 AM, Brock Noland <br...@cloudera.com> wrote:
> There was a ec2 spot price spike overnight which combined with
> everyone trying to get patches in for the branching has resulted in a
> massive queue:
>
> http://bigtop01.cloudera.org:8080/view/Hive/job/PreCommit-HIVE-Build/
>
> ~25 builds in the queue
>
> Brock
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.