You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Andrew Purtell <ap...@apache.org> on 2013/06/10 20:14:16 UTC
Re: The Jenkins VMs are increasingly slow / overloaded
Since the situation with ASF Jenkins seems better (please correct me if
wrong), we are going to scale back HBase jobs on the Jenkins at
http://54.241.6.143/ to one execution per day. Please let me know if you
have any concerns.
On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <ap...@apache.org>wrote:
> I have checked with the folks here and until/unless the situation with ASF
> Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> us-west-1:
>
> - A m1.large instance running 24/7 hosting Jenkins
>
> - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> Jenkins with its EC2 plugin:
> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> would be launched on demand depending on how many concurrent executors are
> needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> inactive. Jenkins' EC2 plugin handles those details.
>
> It would remain to be seen if this is better than the ASF Jenkins VMs, but
> if the community is willing to try it, we'd be willing to set it up and
> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> possible to give all committers accounts for managing these resources. An
> m1.large has:
> - 7.5 GiB memory
> - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
> - 850 GB instance storage
> - 64-bit platform
> - I/O Performance: Moderate
>
>
> On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <nd...@gmail.com> wrote:
>
>> What, like travis-ci.org ?
>>
>> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <ap...@apache.org>
>> wrote:
>>
>> > Would the HBase dev community be open to alternate hosted Jenkins test
>> > infrastructure?
>> >
>> > --
>> > Best regards,
>> >
>> > - Andy
>> >
>> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> > (via Tom White)
>> >
>>
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: The Jenkins VMs are increasingly slow / overloaded
Posted by Stack <st...@duboce.net>.
Sounds good Andrew. Its been useful having another perspective.
On Mon, Jun 10, 2013 at 11:14 AM, Andrew Purtell <ap...@apache.org>wrote:
> Since the situation with ASF Jenkins seems better (please correct me if
> wrong), we are going to scale back HBase jobs on the Jenkins at
> http://54.241.6.143/ to one execution per day. Please let me know if you
> have any concerns.
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <apurtell@apache.org
> >wrote:
>
> > I have checked with the folks here and until/unless the situation with
> ASF
> > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > us-west-1:
> >
> > - A m1.large instance running 24/7 hosting Jenkins
> >
> > - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> > Jenkins with its EC2 plugin:
> > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > would be launched on demand depending on how many concurrent executors
> are
> > needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > inactive. Jenkins' EC2 plugin handles those details.
> >
> > It would remain to be seen if this is better than the ASF Jenkins VMs,
> but
> > if the community is willing to try it, we'd be willing to set it up and
> > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > possible to give all committers accounts for managing these resources. An
> > m1.large has:
> > - 7.5 GiB memory
> > - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
> > - 850 GB instance storage
> > - 64-bit platform
> > - I/O Performance: Moderate
> >
> >
> > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <nd...@gmail.com>
> wrote:
> >
> >> What, like travis-ci.org ?
> >>
> >> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <ap...@apache.org>
> >> wrote:
> >>
> >> > Would the HBase dev community be open to alternate hosted Jenkins test
> >> > infrastructure?
> >> >
> >> > --
> >> > Best regards,
> >> >
> >> > - Andy
> >> >
> >> > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >> > (via Tom White)
> >> >
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: The Jenkins VMs are increasingly slow / overloaded
Posted by Andrew Purtell <ap...@apache.org>.
Thanks Elliott.
This reminds me... to update, we've retained the SCM polling and daily
builds for 0.94 and 0.94-security. All of the other builds are set to
execute once per day at a random time.
On Mon, Jun 10, 2013 at 12:42 PM, Elliott Clark <ec...@apache.org> wrote:
> Thanks for putting so much into that to get us past that rough spot.
>
>
> On Mon, Jun 10, 2013 at 11:14 AM, Andrew Purtell <apurtell@apache.org
> >wrote:
>
> > Since the situation with ASF Jenkins seems better (please correct me if
> > wrong), we are going to scale back HBase jobs on the Jenkins at
> > http://54.241.6.143/ to one execution per day. Please let me know if you
> > have any concerns.
> >
> >
> > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <apurtell@apache.org
> > >wrote:
> >
> > > I have checked with the folks here and until/unless the situation with
> > ASF
> > > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > > us-west-1:
> > >
> > > - A m1.large instance running 24/7 hosting Jenkins
> > >
> > > - A pool of 5 m1.large instances serving as Jenkins slaves, managed
> by
> > > Jenkins with its EC2 plugin:
> > > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > > would be launched on demand depending on how many concurrent executors
> > are
> > > needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > > inactive. Jenkins' EC2 plugin handles those details.
> > >
> > > It would remain to be seen if this is better than the ASF Jenkins VMs,
> > but
> > > if the community is willing to try it, we'd be willing to set it up and
> > > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > > possible to give all committers accounts for managing these resources.
> An
> > > m1.large has:
> > > - 7.5 GiB memory
> > > - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> each)
> > > - 850 GB instance storage
> > > - 64-bit platform
> > > - I/O Performance: Moderate
> > >
> > >
> > > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <nd...@gmail.com>
> > wrote:
> > >
> > >> What, like travis-ci.org ?
> > >>
> > >> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <ap...@apache.org>
> > >> wrote:
> > >>
> > >> > Would the HBase dev community be open to alternate hosted Jenkins
> test
> > >> > infrastructure?
> > >> >
> > >> > --
> > >> > Best regards,
> > >> >
> > >> > - Andy
> > >> >
> > >> > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > >> > (via Tom White)
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: The Jenkins VMs are increasingly slow / overloaded
Posted by Elliott Clark <ec...@apache.org>.
Thanks for putting so much into that to get us past that rough spot.
On Mon, Jun 10, 2013 at 11:14 AM, Andrew Purtell <ap...@apache.org>wrote:
> Since the situation with ASF Jenkins seems better (please correct me if
> wrong), we are going to scale back HBase jobs on the Jenkins at
> http://54.241.6.143/ to one execution per day. Please let me know if you
> have any concerns.
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <apurtell@apache.org
> >wrote:
>
> > I have checked with the folks here and until/unless the situation with
> ASF
> > Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> > us-west-1:
> >
> > - A m1.large instance running 24/7 hosting Jenkins
> >
> > - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> > Jenkins with its EC2 plugin:
> > https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> > would be launched on demand depending on how many concurrent executors
> are
> > needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> > inactive. Jenkins' EC2 plugin handles those details.
> >
> > It would remain to be seen if this is better than the ASF Jenkins VMs,
> but
> > if the community is willing to try it, we'd be willing to set it up and
> > give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> > possible to give all committers accounts for managing these resources. An
> > m1.large has:
> > - 7.5 GiB memory
> > - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
> > - 850 GB instance storage
> > - 64-bit platform
> > - I/O Performance: Moderate
> >
> >
> > On Tue, Mar 19, 2013 at 5:23 PM, Nick Dimiduk <nd...@gmail.com>
> wrote:
> >
> >> What, like travis-ci.org ?
> >>
> >> On Tue, Mar 19, 2013 at 8:49 AM, Andrew Purtell <ap...@apache.org>
> >> wrote:
> >>
> >> > Would the HBase dev community be open to alternate hosted Jenkins test
> >> > infrastructure?
> >> >
> >> > --
> >> > Best regards,
> >> >
> >> > - Andy
> >> >
> >> > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> >> > (via Tom White)
> >> >
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: The Jenkins VMs are increasingly slow / overloaded
Posted by Andrew Purtell <ap...@apache.org>.
On Mon, Jul 15, 2013 at 3:30 PM, Ted Yu <yu...@gmail.com> wrote:
> Thanks for the notice, Andy.
>
> bq. End Time: 2013-07-17 00:00 PDT
>
> Should the End Time be on 07-18 (just guessing) ?
>
Yeah, I don't know... that's what they said.
Re: The Jenkins VMs are increasingly slow / overloaded
Posted by Ted Yu <yu...@gmail.com>.
Thanks for the notice, Andy.
bq. End Time: 2013-07-17 00:00 PDT
Should the End Time be on 07-18 (just guessing) ?
On Mon, Jul 15, 2013 at 3:27 PM, Andrew Purtell <ap...@apache.org> wrote:
> The EC2 Jenkins for HBase is still active, just running a reduced schedule.
>
> I have received the following notice from Amazon regarding the VM
> http://54.241.6.143/, FYI:
>
> Zone: us-west-1a
> Event Type: system-maintenance
> Description:
> Your instance network connections will be restarted during this window.
> Start Time: 2013-07-17 07:00 PDT
> End Time: 2013-07-17 00:00 PDT
>
>
> Will double check service after the maintenance window.
>
>
> On Mon, Jun 10, 2013 at 11:14 AM, Andrew Purtell <apurtell@apache.org
> >wrote:
>
> > Since the situation with ASF Jenkins seems better (please correct me if
> > wrong), we are going to scale back HBase jobs on the Jenkins at
> > http://54.241.6.143/ to one execution per day. Please let me know if you
> > have any concerns.
> >
> >
> > On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <apurtell@apache.org
> >wrote:
> >
> >> I have checked with the folks here and until/unless the situation with
> >> ASF Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
> >> us-west-1:
> >>
> >> - A m1.large instance running 24/7 hosting Jenkins
> >>
> >> - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
> >> Jenkins with its EC2 plugin:
> >> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
> >> would be launched on demand depending on how many concurrent executors
> are
> >> needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
> >> inactive. Jenkins' EC2 plugin handles those details.
> >>
> >> It would remain to be seen if this is better than the ASF Jenkins VMs,
> >> but if the community is willing to try it, we'd be willing to set it up
> and
> >> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
> >> possible to give all committers accounts for managing these resources.
> An
> >> m1.large has:
> >> - 7.5 GiB memory
> >> - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
> each)
> >> - 850 GB instance storage
> >> - 64-bit platform
> >> - I/O Performance: Moderate
> >>
> >
>
Re: The Jenkins VMs are increasingly slow / overloaded
Posted by Andrew Purtell <ap...@apache.org>.
The EC2 Jenkins for HBase is still active, just running a reduced schedule.
I have received the following notice from Amazon regarding the VM
http://54.241.6.143/, FYI:
Zone: us-west-1a
Event Type: system-maintenance
Description:
Your instance network connections will be restarted during this window.
Start Time: 2013-07-17 07:00 PDT
End Time: 2013-07-17 00:00 PDT
Will double check service after the maintenance window.
On Mon, Jun 10, 2013 at 11:14 AM, Andrew Purtell <ap...@apache.org>wrote:
> Since the situation with ASF Jenkins seems better (please correct me if
> wrong), we are going to scale back HBase jobs on the Jenkins at
> http://54.241.6.143/ to one execution per day. Please let me know if you
> have any concerns.
>
>
> On Tue, Mar 19, 2013 at 10:48 AM, Andrew Purtell <ap...@apache.org>wrote:
>
>> I have checked with the folks here and until/unless the situation with
>> ASF Jenkins VMs improves, we can provide a hosted Jenkins up in EC2, in
>> us-west-1:
>>
>> - A m1.large instance running 24/7 hosting Jenkins
>>
>> - A pool of 5 m1.large instances serving as Jenkins slaves, managed by
>> Jenkins with its EC2 plugin:
>> https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin . These
>> would be launched on demand depending on how many concurrent executors are
>> needed (we'd set up a 1 executor to 1 instance mapping) and reaped when
>> inactive. Jenkins' EC2 plugin handles those details.
>>
>> It would remain to be seen if this is better than the ASF Jenkins VMs,
>> but if the community is willing to try it, we'd be willing to set it up and
>> give out access. With EC2's IAM (http://aws.amazon.com/iam/) it's
>> possible to give all committers accounts for managing these resources. An
>> m1.large has:
>> - 7.5 GiB memory
>> - 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
>> - 850 GB instance storage
>> - 64-bit platform
>> - I/O Performance: Moderate
>>
>