You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Andrew Bayer <an...@gmail.com> on 2015/07/02 17:17:44 UTC
[UPDATE] builds.apache.org maintenance on July 2nd
Hi all -
builds.apache.org is still down - we're having problems rebuilding it. We
don't yet have a timeframe for when it'll be back up, but once it's been
reinstalled, I'll send another update. Thanks.
A.
On Thu, Jul 2, 2015 at 1:28 PM, Andrew Bayer <an...@gmail.com> wrote:
> Hooray - a correction. =) I've been able to get all the projects to backup
> in a reasonable timeframe, so we should hopefully not be missing builds.
> Sorry for the false alarm!
>
> A.
>
> On Thu, Jul 2, 2015 at 12:37 PM, Andrew Bayer <an...@gmail.com>
> wrote:
>
>> builds.apache.org has been down for the last two hours as we work on
>> finalizing the backup - due to the absolutely atrocious performance we've
>> seen on this box with ZFS on Linux and directory traversal, we may not be
>> able to back up Maven project type builds from around 5pm UTC yesterday
>> until Jenkins was brought offline at around 8:30am UTC today. To get all of
>> those backed up, it might take another 4 hours of rsyncing. The same build
>> numbers won't be repeated - just that any builds run in that time may not
>> be preserved. This will only be the case for Maven project type builds, not
>> freestyle jobs, and all previous builds and job configuration etc will
>> still be there regardless. More updates when we're further along.
>>
>> A.
>>
>> On Tue, Jun 30, 2015 at 11:52 AM, Andrew Bayer <ab...@apache.org> wrote:
>>
>>> A reminder - builds.apache.org will be down on July 2nd, starting
>>> around 9am UTC/11am CEST/10am BST/5am EDT/2am PDT. We'll be taking down
>>> Jenkins, backing it up, rebuilding the machine, and restoring Jenkins.
>>> Hopefully this will be done within 5-6 hours, but could take longer. We'll
>>> send updates on the status as we go. Thanks.
>>>
>>> If you have any questions please contact infrastructure@apache.org.
>>>
>>> A.
>>>
>>
>>
>
Re: [UPDATE] builds.apache.org maintenance on July 2nd
Posted by Andrew Bayer <an...@gmail.com>.
Hi all -
Regrettably, the restore from backup is taking dramatically longer than
expected - we're now not even halfway through, so I don't expect to be back
online until Saturday morning UTC.
A.
On Fri, Jul 3, 2015 at 9:35 AM, Andrew Bayer <an...@gmail.com> wrote:
> Hi all -
>
> Sorry for the delay in updating - we were finally able to get the OS
> installed and booting properly, and are now in the process of restoring
> Jenkins from backup. This'll take a few hours at least - sorry for the
> inconvenience, but we should be back up this evening Europe time at the
> outside.
>
> A.
>
> On Thu, Jul 2, 2015 at 5:17 PM, Andrew Bayer <an...@gmail.com>
> wrote:
>
>> Hi all -
>>
>> builds.apache.org is still down - we're having problems rebuilding it.
>> We don't yet have a timeframe for when it'll be back up, but once it's been
>> reinstalled, I'll send another update. Thanks.
>>
>> A.
>>
>>
>> On Thu, Jul 2, 2015 at 1:28 PM, Andrew Bayer <an...@gmail.com>
>> wrote:
>>
>>> Hooray - a correction. =) I've been able to get all the projects to
>>> backup in a reasonable timeframe, so we should hopefully not be missing
>>> builds. Sorry for the false alarm!
>>>
>>> A.
>>>
>>> On Thu, Jul 2, 2015 at 12:37 PM, Andrew Bayer <an...@gmail.com>
>>> wrote:
>>>
>>>> builds.apache.org has been down for the last two hours as we work on
>>>> finalizing the backup - due to the absolutely atrocious performance we've
>>>> seen on this box with ZFS on Linux and directory traversal, we may not be
>>>> able to back up Maven project type builds from around 5pm UTC yesterday
>>>> until Jenkins was brought offline at around 8:30am UTC today. To get all of
>>>> those backed up, it might take another 4 hours of rsyncing. The same build
>>>> numbers won't be repeated - just that any builds run in that time may not
>>>> be preserved. This will only be the case for Maven project type builds, not
>>>> freestyle jobs, and all previous builds and job configuration etc will
>>>> still be there regardless. More updates when we're further along.
>>>>
>>>> A.
>>>>
>>>> On Tue, Jun 30, 2015 at 11:52 AM, Andrew Bayer <ab...@apache.org>
>>>> wrote:
>>>>
>>>>> A reminder - builds.apache.org will be down on July 2nd, starting
>>>>> around 9am UTC/11am CEST/10am BST/5am EDT/2am PDT. We'll be taking down
>>>>> Jenkins, backing it up, rebuilding the machine, and restoring Jenkins.
>>>>> Hopefully this will be done within 5-6 hours, but could take longer. We'll
>>>>> send updates on the status as we go. Thanks.
>>>>>
>>>>> If you have any questions please contact infrastructure@apache.org.
>>>>>
>>>>> A.
>>>>>
>>>>
>>>>
>>>
>>
>
Re: [UPDATE] builds.apache.org maintenance on July 2nd
Posted by Andrew Bayer <an...@gmail.com>.
Hi all -
Sorry for the delay in updating - we were finally able to get the OS
installed and booting properly, and are now in the process of restoring
Jenkins from backup. This'll take a few hours at least - sorry for the
inconvenience, but we should be back up this evening Europe time at the
outside.
A.
On Thu, Jul 2, 2015 at 5:17 PM, Andrew Bayer <an...@gmail.com> wrote:
> Hi all -
>
> builds.apache.org is still down - we're having problems rebuilding it. We
> don't yet have a timeframe for when it'll be back up, but once it's been
> reinstalled, I'll send another update. Thanks.
>
> A.
>
>
> On Thu, Jul 2, 2015 at 1:28 PM, Andrew Bayer <an...@gmail.com>
> wrote:
>
>> Hooray - a correction. =) I've been able to get all the projects to
>> backup in a reasonable timeframe, so we should hopefully not be missing
>> builds. Sorry for the false alarm!
>>
>> A.
>>
>> On Thu, Jul 2, 2015 at 12:37 PM, Andrew Bayer <an...@gmail.com>
>> wrote:
>>
>>> builds.apache.org has been down for the last two hours as we work on
>>> finalizing the backup - due to the absolutely atrocious performance we've
>>> seen on this box with ZFS on Linux and directory traversal, we may not be
>>> able to back up Maven project type builds from around 5pm UTC yesterday
>>> until Jenkins was brought offline at around 8:30am UTC today. To get all of
>>> those backed up, it might take another 4 hours of rsyncing. The same build
>>> numbers won't be repeated - just that any builds run in that time may not
>>> be preserved. This will only be the case for Maven project type builds, not
>>> freestyle jobs, and all previous builds and job configuration etc will
>>> still be there regardless. More updates when we're further along.
>>>
>>> A.
>>>
>>> On Tue, Jun 30, 2015 at 11:52 AM, Andrew Bayer <ab...@apache.org>
>>> wrote:
>>>
>>>> A reminder - builds.apache.org will be down on July 2nd, starting
>>>> around 9am UTC/11am CEST/10am BST/5am EDT/2am PDT. We'll be taking down
>>>> Jenkins, backing it up, rebuilding the machine, and restoring Jenkins.
>>>> Hopefully this will be done within 5-6 hours, but could take longer. We'll
>>>> send updates on the status as we go. Thanks.
>>>>
>>>> If you have any questions please contact infrastructure@apache.org.
>>>>
>>>> A.
>>>>
>>>
>>>
>>
>