You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "zhu.chenxiang (JIRA)" <ji...@apache.org> on 2016/10/31 05:50:58 UTC
[jira] [Updated] (FLUME-3019) AvroSource throws OutOfMemory error
when elastic client established
[ https://issues.apache.org/jira/browse/FLUME-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhu.chenxiang updated FLUME-3019:
---------------------------------
Summary: AvroSource throws OutOfMemory error when elastic client established (was: AvroSource throws OutOfMemory error when connect an elastic client)
> AvroSource throws OutOfMemory error when elastic client established
> -------------------------------------------------------------------
>
> Key: FLUME-3019
> URL: https://issues.apache.org/jira/browse/FLUME-3019
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.6.0, v1.7.0
> Environment: Linux version 4.1.7-15.23.amzn1.x86_64
> Reporter: zhu.chenxiang
>
> I use two AvroSource nodes to receive data,The client use AvroSink on aws elastic ec2.
> I find that when Client in case of power failure or broken network，the netty server will not release the resource.
> so when my client is elastic,the server still keep a lot of channels.After a long time running ,throws OutOfMemory error.
> I try to add IdleStateHandler in AdvancedChannelPipelineFactory，but allChannels is a private variable in org.apache.avro.ipc.NettyServer，I cannot remove Channel from allChannels.
> org.jboss.netty.util.Timer timer = new HashedWheelTimer();
> pipeline.addLast("timeout", new IdleStateHandler(timer, 300, 0, 0));
> pipeline.addLast("hearbeat", new Heartbeat());
> private class Heartbeat extends IdleStateAwareChannelHandler {
> @Override
> public void channelIdle(ChannelHandlerContext ctx, IdleStateEvent e) throws Exception {
> super.channelIdle(ctx, e);
> if(e.getState() == IdleState.WRITER_IDLE){
> e.getChannel().close();
> //org.apache.avro.ipc.NettyServer allChannels
> //allChannels.remove(e.getChannel());
> logger.info("Connection to {} closed.",
> e.getChannel().getRemoteAddress());
> }
> }
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)