You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "E. Sammer (Resolved) (JIRA)" <ji...@apache.org> on 2011/12/04 04:04:41 UTC
[jira] [Resolved] (FLUME-861) AvroSource is failing on
ClosedChannelException
[ https://issues.apache.org/jira/browse/FLUME-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
E. Sammer resolved FLUME-861.
-----------------------------
Resolution: Duplicate
Fix Version/s: NG alpha 2
Assignee: E. Sammer
Duplicate of FLUME-827. Jarek: If FLUME-827 doesn't solve the issue please feel free to reopen this. Thanks!
> AvroSource is failing on ClosedChannelException
> -----------------------------------------------
>
> Key: FLUME-861
> URL: https://issues.apache.org/jira/browse/FLUME-861
> Project: Flume
> Issue Type: Bug
> Affects Versions: NG alpha 2
> Reporter: Jarek Jarcec Cecho
> Assignee: E. Sammer
> Fix For: NG alpha 2
>
> Attachments: hddev01.log, hddev01.properties, hddev02.log, hddev02.properties
>
>
> I've noticed that flume-ng is not reconnecting avro source/sink combination after restart of the sink part.
> Steps to reproduce:
> 1) Run flume-ng on two nodes
> 2) Start sending data
> 3) Turn off sink instance (I used CTRL+C)
> 4) Turn it on again
> 5) Avro source part will fail on:
> 2011-11-25 02:22:23,720 (Thread-1) [WARN - org.apache.avro.ipc.NettyTransceiver$NettyClientAvroHandler.exceptionCaught(NettyTransceiver.java:257)] Unexpected exception from downstream.
> java.nio.channels.ClosedChannelException
> (entire error message is present in hddev01.log)
> I've used two centos boxes where first was acting as an agent, reading local log file using ExecSource. Second machine was acting as a collector, waiting for input events and storing them on HDFS. Both machines were connected using AVRO sink+source combination. You can find configuration files for both machines with their logs as well attached to this JIRA bug.
> I've executed both flume-ng instances using following commands:
> ./bin/flume-ng node --conf conf/ --classpath flume-ng.jar --f conf/configuration.properties -n hddev01 > hddev01.log 2>&1
> ./bin/flume-ng node --conf conf/ --classpath flume-ng.jar --f conf/configuration.properties -n hddev02 > hddev02.log 2>&1
> Input file was created using following small bash script (it was executed after flume-ng was successfully started):
> for i in `seq -w 01 180`; do echo $i; echo Yoda-$i >> /var/log/jarcec; sleep 1s; done
> Please note that I applied patch from FLUME-858 in order to get DataStream file type working for this test, however I was observing same behavior without it as well (e.g. using default sequence file)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira