You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Keith W <ke...@gmail.com> on 2015/08/10 14:59:48 UTC

Name resolution issue on ubuntu jenkins slaves?

Hello Apache builds,

We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins slaves
since August 9th.  The test verifies the behaviour of the code when the
user specifies a hostname that does not exist in DNS.  For this purpose,
the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part)
which is assumed not resolve.  This test is longstanding and has been
running on the slaves for many years without issue.

Between August 9th and 10th, something appears to have changed on the
slaves, which is meaning that the lookup of the name is now returning an
IP.  This is causing the Java test to fail. I've investigated by
introducing shell commands into the job, and can see evidence of the same
problem at the UNIX level.  I am certainly seeing this on slaves ubuntu4
through ubuntu6.


$ host hg3sgaaw4lgihjs
hg3sgaaw4lgihjs has address 198.105.244.11
hg3sgaaw4lgihjs has address 198.105.254.11
Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN)

$ host hg3sgaaw4lgihjs2
hg3sgaaw4lgihjs2 has address 198.105.244.11
hg3sgaaw4lgihjs2 has address 198.105.254.11
Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN)


I considered changing the test to use a RFC-2606 Reserved Top Level DNS
Names  hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to
198.105.244.11 too.  The fact that an .invalid address is resolving makes
me suspect there is an environmental problem at the root cause.

host hg3sgaaw4lgihjs.invalid
hg3sgaaw4lgihjs.invalid has address 198.105.244.11
hg3sgaaw4lgihjs.invalid has address 198.105.254.11
Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN)

Is there a name resolution issue affecting these hosts?

Example job affected by the issue:

https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/

Kind regards, Keith Wall.

Re: Name resolution issue on ubuntu jenkins slaves?

Posted by Rajiv Chittajallu <ra...@yahoo-inc.com.INVALID>.
Using google public dns should be fine. 

Removed yahoo in colo resolver, 67.195.2.197 which is accessible from hosted apache nodes.  DHCPD configs is also updated to issue revolvers as 8.8.8.8/8.8.4.4 .
-rajive


      From: Andrew Bayer <an...@gmail.com>
 To: "builds@apache.org" <bu...@apache.org> 
Cc: Keith W <ke...@gmail.com>; Rajiv Chittajallu <ra...@yahoo-inc.com> 
 Sent: Thursday, August 20, 2015 10:23 AM
 Subject: Re: Name resolution issue on ubuntu jenkins slaves?
   
FYI, I've manually overwritten /etc/resolv.conf on all the H* and ubuntu-* slaves with something more valid.
A.


On Thu, Aug 20, 2015 at 1:16 PM, Andrew Bayer <an...@gmail.com> wrote:

Yup, http://drewgraybeal.blogspot.com/2015/05/level-3-dns-hijacking-4222-and-others.html - level3 is hijacking NXDOMAIN now. I think Y!'s DHCP is setting those DNS servers? Rajiv, can you comment?
A.
On Thu, Aug 20, 2015 at 10:32 AM, Andrew Bayer <an...@gmail.com> wrote:

So the Yahoo! provided slaves (H*, ubuntu-*) do have Level3's DNS servers in /etc/resolv.conf - we don't actually control that directly, it's part of the system setup Y! uses, I think. I'll ping our contact at Y! about changing them to use more standard DNS.
A.
On Thu, Aug 20, 2015 at 6:15 AM, Gavin McDonald <gm...@apache.org> wrote:


> On 20 Aug 2015, at 4:03 am, David Nalley <da...@gnsa.us> wrote:
>
> Whatever we are using for DNS on those build slaves is redirecting any
> NXDOMAIN to a Level3 search domain.
>
> Is this happening on the ubuntu-* slaves or on the dynamic build slaves?

It was mentioned earlier :

>  I am certainly seeing this on slaves ubuntu4
> through ubuntu6.

I have seen passes and failures on the dynamic slaves too, so seems inconsistent.

Gav…

>
> --David
>
> On Mon, Aug 10, 2015 at 8:59 AM, Keith W <ke...@gmail.com> wrote:
>> Hello Apache builds,
>>
>> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins slaves
>> since August 9th.  The test verifies the behaviour of the code when the
>> user specifies a hostname that does not exist in DNS.  For this purpose,
>> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part)
>> which is assumed not resolve.  This test is longstanding and has been
>> running on the slaves for many years without issue.
>>
>> Between August 9th and 10th, something appears to have changed on the
>> slaves, which is meaning that the lookup of the name is now returning an
>> IP.  This is causing the Java test to fail. I've investigated by
>> introducing shell commands into the job, and can see evidence of the same
>> problem at the UNIX level.  I am certainly seeing this on slaves ubuntu4
>> through ubuntu6.
>>
>>
>> $ host hg3sgaaw4lgihjs
>> hg3sgaaw4lgihjs has address 198.105.244.11
>> hg3sgaaw4lgihjs has address 198.105.254.11
>> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN)
>>
>> $ host hg3sgaaw4lgihjs2
>> hg3sgaaw4lgihjs2 has address 198.105.244.11
>> hg3sgaaw4lgihjs2 has address 198.105.254.11
>> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN)
>>
>>
>> I considered changing the test to use a RFC-2606 Reserved Top Level DNS
>> Names  hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to
>> 198.105.244.11 too.  The fact that an .invalid address is resolving makes
>> me suspect there is an environmental problem at the root cause.
>>
>> host hg3sgaaw4lgihjs.invalid
>> hg3sgaaw4lgihjs.invalid has address 198.105.244.11
>> hg3sgaaw4lgihjs.invalid has address 198.105.254.11
>> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN)
>>
>> Is there a name resolution issue affecting these hosts?
>>
>> Example job affected by the issue:
>>
>> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/
>>
>> Kind regards, Keith Wall.










  

Re: Name resolution issue on ubuntu jenkins slaves?

Posted by Andrew Bayer <an...@gmail.com>.
FYI, I've manually overwritten /etc/resolv.conf on all the H* and ubuntu-*
slaves with something more valid.

A.

On Thu, Aug 20, 2015 at 1:16 PM, Andrew Bayer <an...@gmail.com>
wrote:

> Yup,
> http://drewgraybeal.blogspot.com/2015/05/level-3-dns-hijacking-4222-and-others.html -
> level3 is hijacking NXDOMAIN now. I think Y!'s DHCP is setting those DNS
> servers? Rajiv, can you comment?
>
> A.
>
> On Thu, Aug 20, 2015 at 10:32 AM, Andrew Bayer <an...@gmail.com>
> wrote:
>
>> So the Yahoo! provided slaves (H*, ubuntu-*) do have Level3's DNS servers
>> in /etc/resolv.conf - we don't actually control that directly, it's part of
>> the system setup Y! uses, I think. I'll ping our contact at Y! about
>> changing them to use more standard DNS.
>>
>> A.
>>
>> On Thu, Aug 20, 2015 at 6:15 AM, Gavin McDonald <gm...@apache.org>
>> wrote:
>>
>>>
>>> > On 20 Aug 2015, at 4:03 am, David Nalley <da...@gnsa.us> wrote:
>>> >
>>> > Whatever we are using for DNS on those build slaves is redirecting any
>>> > NXDOMAIN to a Level3 search domain.
>>> >
>>> > Is this happening on the ubuntu-* slaves or on the dynamic build
>>> slaves?
>>>
>>> It was mentioned earlier :
>>>
>>> >  I am certainly seeing this on slaves ubuntu4
>>> > through ubuntu6.
>>>
>>> I have seen passes and failures on the dynamic slaves too, so seems
>>> inconsistent.
>>>
>>> Gav…
>>>
>>> >
>>> > --David
>>> >
>>> > On Mon, Aug 10, 2015 at 8:59 AM, Keith W <ke...@gmail.com> wrote:
>>> >> Hello Apache builds,
>>> >>
>>> >> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins
>>> slaves
>>> >> since August 9th.  The test verifies the behaviour of the code when
>>> the
>>> >> user specifies a hostname that does not exist in DNS.  For this
>>> purpose,
>>> >> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal
>>> part)
>>> >> which is assumed not resolve.  This test is longstanding and has been
>>> >> running on the slaves for many years without issue.
>>> >>
>>> >> Between August 9th and 10th, something appears to have changed on the
>>> >> slaves, which is meaning that the lookup of the name is now returning
>>> an
>>> >> IP.  This is causing the Java test to fail. I've investigated by
>>> >> introducing shell commands into the job, and can see evidence of the
>>> same
>>> >> problem at the UNIX level.  I am certainly seeing this on slaves
>>> ubuntu4
>>> >> through ubuntu6.
>>> >>
>>> >>
>>> >> $ host hg3sgaaw4lgihjs
>>> >> hg3sgaaw4lgihjs has address 198.105.244.11
>>> >> hg3sgaaw4lgihjs has address 198.105.254.11
>>> >> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN)
>>> >>
>>> >> $ host hg3sgaaw4lgihjs2
>>> >> hg3sgaaw4lgihjs2 has address 198.105.244.11
>>> >> hg3sgaaw4lgihjs2 has address 198.105.254.11
>>> >> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN)
>>> >>
>>> >>
>>> >> I considered changing the test to use a RFC-2606 Reserved Top Level
>>> DNS
>>> >> Names  hg3sgaaw4lgihjs.invalid but I notice that it too is resolving
>>> to
>>> >> 198.105.244.11 too.  The fact that an .invalid address is resolving
>>> makes
>>> >> me suspect there is an environmental problem at the root cause.
>>> >>
>>> >> host hg3sgaaw4lgihjs.invalid
>>> >> hg3sgaaw4lgihjs.invalid has address 198.105.244.11
>>> >> hg3sgaaw4lgihjs.invalid has address 198.105.254.11
>>> >> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN)
>>> >>
>>> >> Is there a name resolution issue affecting these hosts?
>>> >>
>>> >> Example job affected by the issue:
>>> >>
>>> >>
>>> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/
>>> >>
>>> >> Kind regards, Keith Wall.
>>>
>>>
>>>
>>
>

Re: Name resolution issue on ubuntu jenkins slaves?

Posted by Andrew Bayer <an...@gmail.com>.
Yup,
http://drewgraybeal.blogspot.com/2015/05/level-3-dns-hijacking-4222-and-others.html
-
level3 is hijacking NXDOMAIN now. I think Y!'s DHCP is setting those DNS
servers? Rajiv, can you comment?

A.

On Thu, Aug 20, 2015 at 10:32 AM, Andrew Bayer <an...@gmail.com>
wrote:

> So the Yahoo! provided slaves (H*, ubuntu-*) do have Level3's DNS servers
> in /etc/resolv.conf - we don't actually control that directly, it's part of
> the system setup Y! uses, I think. I'll ping our contact at Y! about
> changing them to use more standard DNS.
>
> A.
>
> On Thu, Aug 20, 2015 at 6:15 AM, Gavin McDonald <gm...@apache.org>
> wrote:
>
>>
>> > On 20 Aug 2015, at 4:03 am, David Nalley <da...@gnsa.us> wrote:
>> >
>> > Whatever we are using for DNS on those build slaves is redirecting any
>> > NXDOMAIN to a Level3 search domain.
>> >
>> > Is this happening on the ubuntu-* slaves or on the dynamic build slaves?
>>
>> It was mentioned earlier :
>>
>> >  I am certainly seeing this on slaves ubuntu4
>> > through ubuntu6.
>>
>> I have seen passes and failures on the dynamic slaves too, so seems
>> inconsistent.
>>
>> Gav…
>>
>> >
>> > --David
>> >
>> > On Mon, Aug 10, 2015 at 8:59 AM, Keith W <ke...@gmail.com> wrote:
>> >> Hello Apache builds,
>> >>
>> >> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins
>> slaves
>> >> since August 9th.  The test verifies the behaviour of the code when the
>> >> user specifies a hostname that does not exist in DNS.  For this
>> purpose,
>> >> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part)
>> >> which is assumed not resolve.  This test is longstanding and has been
>> >> running on the slaves for many years without issue.
>> >>
>> >> Between August 9th and 10th, something appears to have changed on the
>> >> slaves, which is meaning that the lookup of the name is now returning
>> an
>> >> IP.  This is causing the Java test to fail. I've investigated by
>> >> introducing shell commands into the job, and can see evidence of the
>> same
>> >> problem at the UNIX level.  I am certainly seeing this on slaves
>> ubuntu4
>> >> through ubuntu6.
>> >>
>> >>
>> >> $ host hg3sgaaw4lgihjs
>> >> hg3sgaaw4lgihjs has address 198.105.244.11
>> >> hg3sgaaw4lgihjs has address 198.105.254.11
>> >> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN)
>> >>
>> >> $ host hg3sgaaw4lgihjs2
>> >> hg3sgaaw4lgihjs2 has address 198.105.244.11
>> >> hg3sgaaw4lgihjs2 has address 198.105.254.11
>> >> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN)
>> >>
>> >>
>> >> I considered changing the test to use a RFC-2606 Reserved Top Level DNS
>> >> Names  hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to
>> >> 198.105.244.11 too.  The fact that an .invalid address is resolving
>> makes
>> >> me suspect there is an environmental problem at the root cause.
>> >>
>> >> host hg3sgaaw4lgihjs.invalid
>> >> hg3sgaaw4lgihjs.invalid has address 198.105.244.11
>> >> hg3sgaaw4lgihjs.invalid has address 198.105.254.11
>> >> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN)
>> >>
>> >> Is there a name resolution issue affecting these hosts?
>> >>
>> >> Example job affected by the issue:
>> >>
>> >>
>> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/
>> >>
>> >> Kind regards, Keith Wall.
>>
>>
>>
>

Re: Name resolution issue on ubuntu jenkins slaves?

Posted by Andrew Bayer <an...@gmail.com>.
So the Yahoo! provided slaves (H*, ubuntu-*) do have Level3's DNS servers
in /etc/resolv.conf - we don't actually control that directly, it's part of
the system setup Y! uses, I think. I'll ping our contact at Y! about
changing them to use more standard DNS.

A.

On Thu, Aug 20, 2015 at 6:15 AM, Gavin McDonald <gm...@apache.org>
wrote:

>
> > On 20 Aug 2015, at 4:03 am, David Nalley <da...@gnsa.us> wrote:
> >
> > Whatever we are using for DNS on those build slaves is redirecting any
> > NXDOMAIN to a Level3 search domain.
> >
> > Is this happening on the ubuntu-* slaves or on the dynamic build slaves?
>
> It was mentioned earlier :
>
> >  I am certainly seeing this on slaves ubuntu4
> > through ubuntu6.
>
> I have seen passes and failures on the dynamic slaves too, so seems
> inconsistent.
>
> Gav…
>
> >
> > --David
> >
> > On Mon, Aug 10, 2015 at 8:59 AM, Keith W <ke...@gmail.com> wrote:
> >> Hello Apache builds,
> >>
> >> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins
> slaves
> >> since August 9th.  The test verifies the behaviour of the code when the
> >> user specifies a hostname that does not exist in DNS.  For this purpose,
> >> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part)
> >> which is assumed not resolve.  This test is longstanding and has been
> >> running on the slaves for many years without issue.
> >>
> >> Between August 9th and 10th, something appears to have changed on the
> >> slaves, which is meaning that the lookup of the name is now returning an
> >> IP.  This is causing the Java test to fail. I've investigated by
> >> introducing shell commands into the job, and can see evidence of the
> same
> >> problem at the UNIX level.  I am certainly seeing this on slaves ubuntu4
> >> through ubuntu6.
> >>
> >>
> >> $ host hg3sgaaw4lgihjs
> >> hg3sgaaw4lgihjs has address 198.105.244.11
> >> hg3sgaaw4lgihjs has address 198.105.254.11
> >> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN)
> >>
> >> $ host hg3sgaaw4lgihjs2
> >> hg3sgaaw4lgihjs2 has address 198.105.244.11
> >> hg3sgaaw4lgihjs2 has address 198.105.254.11
> >> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN)
> >>
> >>
> >> I considered changing the test to use a RFC-2606 Reserved Top Level DNS
> >> Names  hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to
> >> 198.105.244.11 too.  The fact that an .invalid address is resolving
> makes
> >> me suspect there is an environmental problem at the root cause.
> >>
> >> host hg3sgaaw4lgihjs.invalid
> >> hg3sgaaw4lgihjs.invalid has address 198.105.244.11
> >> hg3sgaaw4lgihjs.invalid has address 198.105.254.11
> >> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN)
> >>
> >> Is there a name resolution issue affecting these hosts?
> >>
> >> Example job affected by the issue:
> >>
> >>
> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/
> >>
> >> Kind regards, Keith Wall.
>
>
>

Re: Name resolution issue on ubuntu jenkins slaves?

Posted by Gavin McDonald <gm...@apache.org>.
> On 20 Aug 2015, at 4:03 am, David Nalley <da...@gnsa.us> wrote:
> 
> Whatever we are using for DNS on those build slaves is redirecting any
> NXDOMAIN to a Level3 search domain.
> 
> Is this happening on the ubuntu-* slaves or on the dynamic build slaves?

It was mentioned earlier :

>  I am certainly seeing this on slaves ubuntu4
> through ubuntu6.

I have seen passes and failures on the dynamic slaves too, so seems inconsistent.

Gav…

> 
> --David
> 
> On Mon, Aug 10, 2015 at 8:59 AM, Keith W <ke...@gmail.com> wrote:
>> Hello Apache builds,
>> 
>> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins slaves
>> since August 9th.  The test verifies the behaviour of the code when the
>> user specifies a hostname that does not exist in DNS.  For this purpose,
>> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part)
>> which is assumed not resolve.  This test is longstanding and has been
>> running on the slaves for many years without issue.
>> 
>> Between August 9th and 10th, something appears to have changed on the
>> slaves, which is meaning that the lookup of the name is now returning an
>> IP.  This is causing the Java test to fail. I've investigated by
>> introducing shell commands into the job, and can see evidence of the same
>> problem at the UNIX level.  I am certainly seeing this on slaves ubuntu4
>> through ubuntu6.
>> 
>> 
>> $ host hg3sgaaw4lgihjs
>> hg3sgaaw4lgihjs has address 198.105.244.11
>> hg3sgaaw4lgihjs has address 198.105.254.11
>> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN)
>> 
>> $ host hg3sgaaw4lgihjs2
>> hg3sgaaw4lgihjs2 has address 198.105.244.11
>> hg3sgaaw4lgihjs2 has address 198.105.254.11
>> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN)
>> 
>> 
>> I considered changing the test to use a RFC-2606 Reserved Top Level DNS
>> Names  hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to
>> 198.105.244.11 too.  The fact that an .invalid address is resolving makes
>> me suspect there is an environmental problem at the root cause.
>> 
>> host hg3sgaaw4lgihjs.invalid
>> hg3sgaaw4lgihjs.invalid has address 198.105.244.11
>> hg3sgaaw4lgihjs.invalid has address 198.105.254.11
>> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN)
>> 
>> Is there a name resolution issue affecting these hosts?
>> 
>> Example job affected by the issue:
>> 
>> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/
>> 
>> Kind regards, Keith Wall.



Re: Name resolution issue on ubuntu jenkins slaves?

Posted by David Nalley <da...@gnsa.us>.
Whatever we are using for DNS on those build slaves is redirecting any
NXDOMAIN to a Level3 search domain.

Is this happening on the ubuntu-* slaves or on the dynamic build slaves?

--David

On Mon, Aug 10, 2015 at 8:59 AM, Keith W <ke...@gmail.com> wrote:
> Hello Apache builds,
>
> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins slaves
> since August 9th.  The test verifies the behaviour of the code when the
> user specifies a hostname that does not exist in DNS.  For this purpose,
> the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part)
> which is assumed not resolve.  This test is longstanding and has been
> running on the slaves for many years without issue.
>
> Between August 9th and 10th, something appears to have changed on the
> slaves, which is meaning that the lookup of the name is now returning an
> IP.  This is causing the Java test to fail. I've investigated by
> introducing shell commands into the job, and can see evidence of the same
> problem at the UNIX level.  I am certainly seeing this on slaves ubuntu4
> through ubuntu6.
>
>
> $ host hg3sgaaw4lgihjs
> hg3sgaaw4lgihjs has address 198.105.244.11
> hg3sgaaw4lgihjs has address 198.105.254.11
> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN)
>
> $ host hg3sgaaw4lgihjs2
> hg3sgaaw4lgihjs2 has address 198.105.244.11
> hg3sgaaw4lgihjs2 has address 198.105.254.11
> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN)
>
>
> I considered changing the test to use a RFC-2606 Reserved Top Level DNS
> Names  hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to
> 198.105.244.11 too.  The fact that an .invalid address is resolving makes
> me suspect there is an environmental problem at the root cause.
>
> host hg3sgaaw4lgihjs.invalid
> hg3sgaaw4lgihjs.invalid has address 198.105.244.11
> hg3sgaaw4lgihjs.invalid has address 198.105.254.11
> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN)
>
> Is there a name resolution issue affecting these hosts?
>
> Example job affected by the issue:
>
> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/
>
> Kind regards, Keith Wall.

Re: Name resolution issue on ubuntu jenkins slaves?

Posted by ke...@gmail.com.
Hi Builds

Would someone be able the comment in the issue I describe below please?

--
Keith.

> On 10 Aug 2015, at 13:59, Keith W <ke...@gmail.com> wrote:
> 
> Hello Apache builds,
> 
> We (Apache Qpid) have been seeing a test fail on the Ubuntu Jenkins slaves since August 9th.  The test verifies the behaviour of the code when the user specifies a hostname that does not exist in DNS.  For this purpose, the test uses a random name 'hg3sgaaw4lgihjs' (without hierarchal part) which is assumed not resolve.  This test is longstanding and has been running on the slaves for many years without issue.
> 
> Between August 9th and 10th, something appears to have changed on the slaves, which is meaning that the lookup of the name is now returning an IP.  This is causing the Java test to fail. I've investigated by introducing shell commands into the job, and can see evidence of the same problem at the UNIX level.  I am certainly seeing this on slaves ubuntu4 through ubuntu6.
> 
> 
> $ host hg3sgaaw4lgihjs
> hg3sgaaw4lgihjs has address 198.105.244.11
> hg3sgaaw4lgihjs has address 198.105.254.11
> Host hg3sgaaw4lgihjs not found: 3(NXDOMAIN)
> 
> $ host hg3sgaaw4lgihjs2
> hg3sgaaw4lgihjs2 has address 198.105.244.11
> hg3sgaaw4lgihjs2 has address 198.105.254.11
> Host hg3sgaaw4lgihjs2 not found: 3(NXDOMAIN)
> 
> 
> I considered changing the test to use a RFC-2606 Reserved Top Level DNS Names  hg3sgaaw4lgihjs.invalid but I notice that it too is resolving to 198.105.244.11 too.  The fact that an .invalid address is resolving makes me suspect there is an environmental problem at the root cause.
> 
> host hg3sgaaw4lgihjs.invalid
> hg3sgaaw4lgihjs.invalid has address 198.105.244.11
> hg3sgaaw4lgihjs.invalid has address 198.105.254.11
> Host hg3sgaaw4lgihjs.invalid not found: 3(NXDOMAIN)
> 
> Is there a name resolution issue affecting these hosts?
> 
> Example job affected by the issue:
> 
> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-Test-JDK1.8/
> 
> Kind regards, Keith Wall.