You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Lance Albertson <la...@osuosl.org> on 2019/09/17 18:12:35 UTC
[Hosting] DNS outage (currently temporarily resolved)
All,
Starting at around 10:30 AM PDT (1730 UTC), all three of our DNS servers
stopped responding to DNS requests. We temporarily recovered the services
starting at 10:53 AM PDT (1753 UTC). Initial investigated work suggests we
ran into this problem [1] due to the release of CentOS 7.7 today which
updated the version of bind. When our configuration management tool ran, it
automatically upgraded bind which caused this problem. For now we've
disabled our configuration management on the nodes pending a more permanent
solution. We'll try to minimize the outage impacts when we deploy the
permanent solution.
Apologies for the issues.
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820056
--
Lance Albertson
Director
Oregon State University | Open Source Lab
Re: [Hosting] DNS outage (currently temporarily resolved)
Posted by Lance Albertson <la...@osuosl.org>.
We have updated our configuration management to account for the changes in
Bind and will be deploying the change at 4pm PDT (2300 UTC) today (in just
about 30 minutes). Unfortunately, this may cause a momentary outage on a
specific DNS server when we push the update. To minimize the impact, we'll
be pushing this update one server at a time and confirming the fix is
working properly.
Thanks-
On Tue, Sep 17, 2019 at 11:12 AM Lance Albertson <la...@osuosl.org> wrote:
> All,
>
> Starting at around 10:30 AM PDT (1730 UTC), all three of our DNS servers
> stopped responding to DNS requests. We temporarily recovered the services
> starting at 10:53 AM PDT (1753 UTC). Initial investigated work suggests we
> ran into this problem [1] due to the release of CentOS 7.7 today which
> updated the version of bind. When our configuration management tool ran, it
> automatically upgraded bind which caused this problem. For now we've
> disabled our configuration management on the nodes pending a more permanent
> solution. We'll try to minimize the outage impacts when we deploy the
> permanent solution.
>
> Apologies for the issues.
>
> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820056
>
> --
> Lance Albertson
> Director
> Oregon State University | Open Source Lab
>
--
Lance Albertson
Director
Oregon State University | Open Source Lab