[jira] [Created] (AVRO-1538) NettyTransceiver descriptor leaks

["Platon Vai (JIRA)" <ji...@apache.org>]

Platon Vai created AVRO-1538:
--------------------------------

             Summary: NettyTransceiver descriptor leaks
                 Key: AVRO-1538
                 URL: https://issues.apache.org/jira/browse/AVRO-1538
             Project: Avro
          Issue Type: Bug
          Components: java
    Affects Versions: 1.7.6, 1.7.5
         Environment: openSuse 13.1 64 bit, jdk1.7.0_51 64 bit
            Reporter: Platon Vai


NettyTransceiver created without channel factory results in file descriptors leak while connecting with IOException (java.io.IOException: Error connecting to, NettyTransceiver.java:280).

It seems very similar to https://issues.apache.org/jira/browse/FLUME-2221. But I am not flume user.

To reproduce - run the test below (IP:9999 was nobody listened.)
In my enviroment (run from IDEA, openSuse 13.1 64bit, all system settings are by default) stable fails after 146 cycles (count = 146). 

    @Test
    public void testAvroNettyTest() {
        for (int count = 0; count < 4500; count++) {
            try {
                NettyTransceiver apiClient = new NettyTransceiver(new InetSocketAddress("<IP>", 9999), 5000L);
            } catch (Exception exc) {
                System.out.println(String.format("=========================== %d ===========================", count));
                exc.printStackTrace();

                if ( exc.getMessage().contains("Too many open files")
                        || exc.getCause().getMessage().contains("Too many open files")  ) {
                    System.out.println("Found [Too many open files].");
                    break;
                }
            }
        }
    }

And test that OK
    @Test
    public void testAvroCustomNettyTest() throws Exception {
        for (int count = 0; count < 4500; count++) {
            NioClientSocketChannelFactory channelFactory = null;
            ExecutorService bossExecutor = null, workerExecutor = null;
            try {
                bossExecutor = Executors.newCachedThreadPool(new NettyTransceiverThreadFactory(
                        "Avro " + NettyTransceiver.class.getSimpleName() + " Boss"));

                workerExecutor = Executors.newCachedThreadPool(new NettyTransceiverThreadFactory(
                        "Avro " + NettyTransceiver.class.getSimpleName() + " I/O Worker"));

                channelFactory = new NioClientSocketChannelFactory(
                        bossExecutor,
                        workerExecutor
                        );

                NettyTransceiver apiClient = new NettyTransceiver(
                        new InetSocketAddress("<IP>", 9999),
                        channelFactory,
                        5000L);
            } catch (Exception exc) {
                System.out.println(String.format("=========================== %d ===========================", count));
                exc.printStackTrace();

                if ( channelFactory != null ) {
                    channelFactory.shutdown();
                }

                if ( exc.getMessage().contains("Too many open files")
                        || exc.getCause().getMessage().contains("Too many open files")  ) {
                    System.out.println("Found [Too many open files].");
                    break;
                }
            }
        }
    }

So workaround exists it would be cool if NettyTransceiver in next release will be more "friendly" (release all internal resources if initialization failed). 



--
This message was sent by Atlassian JIRA
(v6.2#6252)