DirectDriver Error

[Cameron Gandevia <cg...@gmail.com>]

Hey

I am noticing the following exception in my collector logs. Whenever it
happens my collector goes down and does not reconnect. I am using trunk
build as of Oct 3. I have a feeling it some kind a race condition with the
number of file handles open. Has anyone else experienced this problem?

2011-10-04 14:31:43,386 [logicalNode collector0_log_dir-3497] INFO
 com.cloudera.flume.core.connector.DirectDriver - Connector logicalNode
collector0_log_dir-3497 exited with error: Blocked append interrupted by
rotation event
java.lang.InterruptedException: Blocked append interrupted by rotation event
        at
com.cloudera.flume.handlers.rolling.RollSink.append(RollSink.java:209)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.core.MaskDecorator.append(MaskDecorator.java:43)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.handlers.debug.InsistentOpenDecorator.append(InsistentOpenDecorator.java:169)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:71)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.handlers.debug.InsistentAppendDecorator.append(InsistentAppendDecorator.java:110)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.handlers.endtoend.AckChecksumChecker.append(AckChecksumChecker.java:113)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.handlers.batch.UnbatchingDecorator.append(UnbatchingDecorator.java:62)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.handlers.batch.GunzipDecorator.append(GunzipDecorator.java:81)
        at
com.cloudera.flume.collector.CollectorSink.append(CollectorSink.java:222)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.core.extractors.DateExtractor.append(DateExtractor.java:86)
        at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
        at
com.cloudera.flume.core.extractors.RegexExtractor.append(RegexExtractor.java:88)
        at
com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:133)
2011-10-04 14:31:43,387 [logicalNode collector0_log_dir-3497] INFO
 com.cloudera.flume.collector.CollectorSource - closed
2011-10-04 14:31:44,393 [logicalNode collector0_log_dir-3497] INFO
 com.cloudera.flume.handlers.thrift.ThriftEventSource - Closed server on
port 36892...
2011-10-04 14:31:44,393 [logicalNode collector0_log_dir-3497] INFO
 com.cloudera.flume.handlers.thrift.ThriftEventSource - Queue still has 1000
elements ...
2011-10-04 14:31:54,402 [logicalNode collector0_log_dir-3497] WARN
 com.cloudera.flume.handlers.thrift.ThriftEventSource - Close timed out due
to no progress.  Closing despite having 1000 values still enqueued
2011-10-04 14:31:54,402 [logicalNode collector0_log_dir-3497] INFO
 com.cloudera.flume.handlers.rolling.RollSink - closing RollSink
'escapedCustomDfs("hdfs://
van-mang-perf-hadoop-namenode1.globalrelay.net:8020/rawLogs/%{dateyear}-%{datemonth}-%{dateday}/%{datehr}00/%{host}","raw-%{rolltag}"
)'
2011-10-04 14:31:54,402 [logicalNode collector0_log_dir-3497] INFO
 com.cloudera.flume.handlers.rolling.RollSink - double close
'escapedCustomDfs("hdfs://
van-mang-perf-hadoop-namenode1.globalrelay.net:8020/rawLogs/%{dateyear}-%{datemonth}-%{dateday}/%{datehr}00/%{host}","raw-%{rolltag}"
)'
2011-10-04 14:31:54,402 [logicalNode collector0_log_dir-3497] ERROR
com.cloudera.flume.core.connector.DirectDriver - Exiting driver logicalNode
collector0_log_dir-3497 in error state CollectorSource | RegexExtractor
because Blocked append interrupted by rotation event