You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flume.apache.org by "Yanzhi.liu" <40...@qq.com> on 2012/10/16 04:19:58 UTC

About file channel yum error

Hello everyone:
     I am using the flume 1.3.0.But the flume clusters was working after two days and the flume was stoped. And the flume.log was only error about the:
 14 Oct 2012 15:29:17,018 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.hdfs.HDFSEventSink.process:448)  - process failed
java.lang.OutOfMemoryError
        at java.io.RandomAccessFile.readBytes(Native Method)
        at java.io.RandomAccessFile.read(RandomAccessFile.java:338)
        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:397)
        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:377)
        at org.apache.flume.channel.file.LogFile.readDelimitedBuffer(LogFile.java:486)
        at org.apache.flume.channel.file.LogFileV3$RandomReader.doGet(LogFileV3.java:258)
        at org.apache.flume.channel.file.LogFile$RandomReader.get(LogFile.java:298)
        at org.apache.flume.channel.file.Log.get(Log.java:409)
        at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:447)
        at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
        at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:91)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:387)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:662)
 So I want to how to avoid the error.
 Thanks very much!
 My Name:
 Yanzhi Liu

Re: About file channel yum error

Posted by Brock Noland <br...@cloudera.com>.

What is your channel capacity and configured agent heap size?


On Monday, October 15, 2012 at 9:19 PM, Yanzhi.liu wrote:

> Hello everyone:
>     I am using the flume 1.3.0.But the flume clusters was working after two days and the flume was stoped. And the flume.log was only error about the:
> 14 Oct 2012 15:29:17,018 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.hdfs.HDFSEventSink.process:448)  - process failed
> java.lang.OutOfMemoryError
>         at java.io.RandomAccessFile.readBytes(Native Method)
>         at java.io.RandomAccessFile.read(RandomAccessFile.java:338)
>         at java.io.RandomAccessFile.readFully(RandomAccessFile.java:397)
>         at java.io.RandomAccessFile.readFully(RandomAccessFile.java:377)
>         at org.apache.flume.channel.file.LogFile.readDelimitedBuffer(LogFile.java:486)
>         at org.apache.flume.channel.file.LogFileV3$RandomReader.doGet(LogFileV3.java:258)
>         at org.apache.flume.channel.file.LogFile$RandomReader.get(LogFile.java:298)
>         at org.apache.flume.channel.file.Log.get(Log.java:409)
>         at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:447)
>         at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
>         at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:91)
>         at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:387)
>         at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>         at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>         at java.lang.Thread.run(Thread.java:662) 
> So I want to how to avoid the error.
> Thanks very much!
> My Name:
> Yanzhi Liu
> 
> 
> 
> 
> 
> 
> 
>