Test failure on Java side in Ubuntu (0.14.0-rc1)

[Yunseong Lee <yu...@gmail.com>]

Hi,

In the voting thread in rc1, Gon reported that the tests fail in Ubuntu.
Actually, I experienced the same failure before, but I thought it could be
an environmental issue.

1.
The failed tests are in reef-io, especially regarding to Network. Taegeon
and I took a look at the log (reef-0.14.0-rc1-java-ubuntu.log in
attachment), and he pointed out that the first Exception was:

Caused by: java.net.SocketException: Too many open files
> at sun.nio.ch.Net.socket0(Native Method)
> at sun.nio.ch.Net.serverSocket(Net.java:445)
> at
> sun.nio.ch.ServerSocketChannelImpl.<init>(ServerSocketChannelImpl.java:88)
> at
> sun.nio.ch.SelectorProviderImpl.openServerSocketChannel(SelectorProviderImpl.java:56)
> at
> io.netty.channel.socket.nio.NioServerSocketChannel.newSocket(NioServerSocketChannel.java:57)
> ... 36 more


>From this block, the tests in the same package fail subsequently. It seems
because too many sockets are open up to the limit that OS can support.


2.
I've tested by checking out the previous commits, and the tests pass before
'[REEF-1124] Remove deprecated constructors of NameLookupClient' [1].

Those constructors were deprecated[2] by Sergey, and removed[3] by Andrew.
Sergey and Andrew, Do you have any idea about this problem?


3.
I've tried different version of Ubuntu in different machines (including
VMs), the tests never have passed in Ubuntu.

Interestingly, Travis CI[3] that runs on Ubuntu 12.04 has never failed,
which I encounter another failure with the same version of Ubuntu.

Could we make sure the CI settings are correct?



Any comment is more than welcome.

Regards,
Yunseong

[1] https://issues.apache.org/jira/browse/REEF-703
[2] https://issues.apache.org/jira/browse/REEF-1124
[3] https://travis-ci.org/apache/reef/jobs/111555784#L3

Re: Test failure on Java side in Ubuntu (0.14.0-rc1)

[Tae-Geon Um <ta...@gmail.com>]

Hi, yunseong

Thanks for your report! 

> On Feb 26, 2016, at 9:23 PM, Yunseong Lee <yu...@gmail.com> wrote:
> 
> Hi,
> 
> In the voting thread in rc1, Gon reported that the tests fail in Ubuntu.  Actually, I experienced the same failure before, but I thought it could be an environmental issue.
> 
> 1.
> The failed tests are in reef-io, especially regarding to Network. Taegeon and I took a look at the log (reef-0.14.0-rc1-java-ubuntu.log in attachment), and he pointed out that the first Exception was:
> 
> 	Caused by: java.net.SocketException: Too many open files
> 		at sun.nio.ch.Net.socket0(Native Method)
> 		at sun.nio.ch.Net.serverSocket(Net.java:445)
> 		at sun.nio.ch.ServerSocketChannelImpl.<init>(ServerSocketChannelImpl.java:88)
> 		at sun.nio.ch.SelectorProviderImpl.openServerSocketChannel(SelectorProviderImpl.java:56)
> 		at io.netty.channel.socket.nio.NioServerSocketChannel.newSocket(NioServerSocketChannel.java:57)
> 		... 36 more
> 
> From this block, the tests in the same package fail subsequently. It seems because too many sockets are open up to the limit that OS can support.
> 
> 
> 2.
> I've tested by checking out the previous commits, and the tests pass before '[REEF-1124] Remove deprecated constructors of NameLookupClient' [1].
> 
> Those constructors were deprecated[2] by Sergey, and removed[3] by Andrew. Sergey and Andrew, Do you have any idea about this problem? 
> 

I took a look at the "[REEF-1124] Remove deprecated constructors of NameLookupClient” 
and I found the only functional difference btw previous version and this PR is that this PR does not “share”  replyLookupQueue in NameClient.java. This PR does not give replyLookupQueue as an argument to the constructer of NameLookupClient, but previous version gives the queue as an argument. 

I’m not sure right now whether this is the root cause or not. And If it is, I’m not sure why this makes a problem right now.
I need some time to figure out.

> 
> 3.
> I've tried different version of Ubuntu in different machines (including VMs), the tests never have passed in Ubuntu.
> 
> Interestingly, Travis CI[3] that runs on Ubuntu 12.04 has never failed, which I encounter another failure with the same version of Ubuntu.
> 
> Could we make sure the CI settings are correct? 
> 
> 
> 
> Any comment is more than welcome.
> 
> Regards,
> Yunseong
> 
> [1] https://issues.apache.org/jira/browse/REEF-703 <https://issues.apache.org/jira/browse/REEF-703>
> [2] https://issues.apache.org/jira/browse/REEF-1124 <https://issues.apache.org/jira/browse/REEF-1124>
> [3] https://travis-ci.org/apache/reef/jobs/111555784#L3 <https://travis-ci.org/apache/reef/jobs/111555784#L3>
Thanks,
Taegeon