You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Julian Alcala (Created) (JIRA)" <ji...@apache.org> on 2012/02/23 21:58:52 UTC
[jira] [Created] (FLUME-993) Tail thread nterrupte: null / Tail
thread exiting With S3 bucket
Tail thread nterrupte: null / Tail thread exiting With S3 bucket
----------------------------------------------------------------
Key: FLUME-993
URL: https://issues.apache.org/jira/browse/FLUME-993
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v0.9.4
Environment: Debian Squeeze, S3, EC2
Reporter: Julian Alcala
Priority: Blocker
Hi Guys,
The following are logs from my flume agents. These agents are using a collector sink that pushes to s3. I am finding that my agents will not stay alive and get into the blow error state with the tail thread exiting. Please advise. Thanks much!
2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
@
9942,93 22%
2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
java.lang.InterruptedException
at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-05144
26931-0800.316601632689443.00000026
2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_ww
riting/20120222-051426931-0800.316601632689443.00000026
2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.31660163266
89443.00000026
2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaa
iloverManager$StateChangeDeco@65cccd3
2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PSTT
2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' ss
ink:'agentSink( "flume-collector", 35853 )' }
2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
@
9942,93 22%
2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
java.lang.InterruptedException
at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-051422
6931-0800.316601632689443.00000026
2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrr
iting/20120222-051426931-0800.316601632689443.00000026
2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.316601632688
9443.00000026
2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaii
loverManager$StateChangeDeco@65cccd3
2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PST
2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sinn
k:'agentSink( "flume-collector", 35853 )' }
2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
@
9942,93 22%
2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
java.lang.InterruptedException
at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-0514266
931-0800.316601632689443.00000026
2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrii
ting/20120222-051426931-0800.316601632689443.00000026
2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.3166016326899
443.00000026
2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaill
overManager$StateChangeDeco@65cccd3
2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PST 22
012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sink::
'agentSink( "flume-collector", 35853 )' }
2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
@
9942,93 22%
2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
java.lang.InterruptedException
at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (FLUME-993) Tail thread nterrupte: null / Tail
thread exiting With S3 bucket
Posted by "Alexander Lorenz-Alten (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Lorenz-Alten resolved FLUME-993.
------------------------------------------
Resolution: Won't Fix
> Tail thread nterrupte: null / Tail thread exiting With S3 bucket
> ----------------------------------------------------------------
>
> Key: FLUME-993
> URL: https://issues.apache.org/jira/browse/FLUME-993
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: Debian Squeeze, S3, EC2
> Reporter: Julian Alcala
> Priority: Blocker
>
> Hi Guys,
> The following are logs from my flume agents. These agents are using a collector sink that pushes to s3. I am finding that my agents will not stay alive and get into the blow error state with the tail thread exiting. Please advise. Thanks much!
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-05144
> 26931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_ww
> riting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.31660163266
> 89443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaa
> iloverManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PSTT
> 2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' ss
> ink:'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-051422
> 6931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrr
> iting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.316601632688
> 9443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaii
> loverManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PST
> 2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sinn
> k:'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-0514266
> 931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrii
> ting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.3166016326899
> 443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaill
> overManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PST 22
> 012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sink::
> 'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-993) Tail thread nterrupte: null / Tail
thread exiting With S3 bucket
Posted by "Clive Cox (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255445#comment-13255445 ]
Clive Cox commented on FLUME-993:
---------------------------------
I am seeing something very similar with a client tailing a log file which is sent to S3.
This is with a built 0.9.5 version of flume.
2012-04-16 11:13:25,810 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
java.lang.InterruptedException
at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:812)
at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
2012-04-16 11:13:25,811 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> Tail thread nterrupte: null / Tail thread exiting With S3 bucket
> ----------------------------------------------------------------
>
> Key: FLUME-993
> URL: https://issues.apache.org/jira/browse/FLUME-993
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: Debian Squeeze, S3, EC2
> Reporter: Julian Alcala
> Priority: Blocker
>
> Hi Guys,
> The following are logs from my flume agents. These agents are using a collector sink that pushes to s3. I am finding that my agents will not stay alive and get into the blow error state with the tail thread exiting. Please advise. Thanks much!
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-05144
> 26931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_ww
> riting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.31660163266
> 89443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaa
> iloverManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PSTT
> 2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' ss
> ink:'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-051422
> 6931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrr
> iting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.316601632688
> 9443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaii
> loverManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PST
> 2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sinn
> k:'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-0514266
> 931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrii
> ting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.3166016326899
> 443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaill
> overManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PST 22
> 012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sink::
> 'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (FLUME-993) Tail thread nterrupte: null / Tail
thread exiting With S3 bucket
Posted by "Alexander Lorenz-Alten (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Lorenz-Alten reopened FLUME-993:
------------------------------------------
> Tail thread nterrupte: null / Tail thread exiting With S3 bucket
> ----------------------------------------------------------------
>
> Key: FLUME-993
> URL: https://issues.apache.org/jira/browse/FLUME-993
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v0.9.4
> Environment: Debian Squeeze, S3, EC2
> Reporter: Julian Alcala
> Priority: Blocker
>
> Hi Guys,
> The following are logs from my flume agents. These agents are using a collector sink that pushes to s3. I am finding that my agents will not stay alive and get into the blow error state with the tail thread exiting. Please advise. Thanks much!
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-05144
> 26931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_ww
> riting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.31660163266
> 89443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaa
> iloverManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PSTT
> 2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' ss
> ink:'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-051422
> 6931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrr
> iting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.316601632688
> 9443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaii
> loverManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PST
> 2012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sinn
> k:'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,372 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink 'NaiveFileFailover'
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.handlers.hdfs.SeqfileEventSink: closed /tmp/flume-flume/agent/host/dfo_writing/20120222-0514266
> 931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: File lives in /tmp/flume-flume/agent/host/dfo_wrii
> ting/20120222-051426931-0800.316601632689443.00000026
> 2012-02-22 05:14:36,373 INFO com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: opening new file for 20120222-051426931-0800.3166016326899
> 443.00000026
> 2012-02-22 05:14:36,374 INFO com.cloudera.flume.agent.diskfailover.DiskFailoverSource: end of file com.cloudera.flume.agent.diskfailover.NaiveFileFaill
> overManager$StateChangeDeco@65cccd3
> 2012-02-22 05:14:36,581 WARN com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager: Double close (which is ok)
> 2012-02-22 05:14:36,581 INFO com.cloudera.flume.handlers.thrift.ThriftEventSink: ThriftEventSink on port 35853 closed
> 2012-02-22 05:14:36,792 INFO com.cloudera.flume.agent.LogicalNode: Node config successfully set to FlumeConfigData: {srcVer:'Wed Feb 22 05:14:34 PST 22
> 012' snkVer:'Wed Feb 22 05:14:34 PST 2012' ts='Wed Feb 22 05:14:34 PST 2012' flowId:'default-flow' source:'tail( "/log/log.log" )' sink::
> 'agentSink( "flume-collector", 35853 )' }
> 2012-02-22 05:14:36,826 WARN com.cloudera.flume.agent.durability.NaiveFileWALManager: Strange, shutting down but now reopening
> 2012-02-22 05:14:36,826 INFO com.cloudera.flume.agent.durability.NaiveFileWALManager: NaiveFileWALManager is now open
> @
> 9942,93 22%
> 2012-02-22 05:14:36,371 ERROR com.cloudera.flume.handlers.text.TailSource: Tail thread nterrupted: null
> java.lang.InterruptedException
> at java.util.concurrent.SynchronousQueue.put(SynchronousQueue.java:841)
> at com.cloudera.flume.handlers.text.Cursor.extractLines(Cursor.java:230)
> at com.cloudera.flume.handlers.text.Cursor.readAllFromChannel(Cursor.java:368)
> at com.cloudera.flume.handlers.text.Cursor.tailBody(Cursor.java:255)
> at com.cloudera.flume.handlers.text.TailSource$TailThread.run(TailSource.java:191)
> 2012-02-22 05:14:36,371 INFO com.cloudera.flume.handlers.text.TailSource: TailThread has exited
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira