[jira] [Created] (FLUME-3019) AvroSource throws OutOfMemory error when building an elastic client

["zhu.chenxiang (JIRA)" <ji...@apache.org>]

zhu.chenxiang created FLUME-3019:
------------------------------------

             Summary: AvroSource throws OutOfMemory error when building an elastic client
                 Key: FLUME-3019
                 URL: https://issues.apache.org/jira/browse/FLUME-3019
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
    Affects Versions: v1.7.0
         Environment: Linux version 4.1.7-15.23.amzn1.x86_64
            Reporter: zhu.chenxiang


I use two AvroSource nodes to receive data,The client use AvroSink  on aws elastic ec2.
I find that when Client in case of power failure or broken network，the netty server will not release the resource.
so when my client is elastic,the server still keep a lot of channels.After a long time running ,throws OutOfMemory error.
I try to add IdleStateHandler in AdvancedChannelPipelineFactory，but allChannels is a private variable in org.apache.avro.ipc.NettyServer，I cannot remove Channel from allChannels.

org.jboss.netty.util.Timer timer = new HashedWheelTimer();
pipeline.addLast("timeout", new IdleStateHandler(timer, 300, 0, 0));
pipeline.addLast("hearbeat", new Heartbeat());

    private class Heartbeat  extends IdleStateAwareChannelHandler {

      @Override
      public void channelIdle(ChannelHandlerContext ctx, IdleStateEvent e) throws Exception {
        super.channelIdle(ctx, e);
        if(e.getState() == IdleState.WRITER_IDLE){
          e.getChannel().close();
         //org.apache.avro.ipc.NettyServer allChannels
         //allChannels.remove(e.getChannel());
          logger.info("Connection to {} closed.",
                  e.getChannel().getRemoteAddress());
        }
      }
    }






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)