[jira] [Created] (FLUME-3185) Corrupt event found. Please run File Channel Integrity tool

["Santhosh Chandrasekaran (JIRA)" <ji...@apache.org>]

Santhosh Chandrasekaran created FLUME-3185:
----------------------------------------------

             Summary: Corrupt event found. Please run File Channel Integrity tool
                 Key: FLUME-3185
                 URL: https://issues.apache.org/jira/browse/FLUME-3185
             Project: Flume
          Issue Type: Bug
          Components: File Channel
    Affects Versions: 1.7.0
         Environment: PRODUCTION
            Reporter: Santhosh Chandrasekaran


We get the below exception in PROD.

08 Oct 2017 15:46:09,988 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:158)  - Unable to deliver event. Exception follows.
org.apache.flume.ChannelException: Take failed due to IO error [channel=chn-file-xml-dis-log]
        at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:534)
        at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
        at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
        at com.macys.daas.flume.sink.elasticsearch.CreateIndex.process(CreateIndex.java:150)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Corrupt event found. Please run File Channel Integrity tool.
        at org.apache.flume.channel.file.Log.get(Log.java:616) 
        at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:531)
        ... 6 more
Caused by: org.apache.flume.channel.file.CorruptEventException: Could not parse event from data file.
        at org.apache.flume.channel.file.TransactionEventRecord.fromByteArray(TransactionEventRecord.java:212)
        at org.apache.flume.channel.file.LogFileV3$RandomReader.doGet(LogFileV3.java:303)
        at org.apache.flume.channel.file.LogFile$RandomReader.get(LogFile.java:501)
        at org.apache.flume.channel.file.Log.get(Log.java:612)
        ... 7 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large.  May be malicious.  Use CodedInputStream.setSizeLimit() to increase the size limit.
 at com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
        at com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
        at com.google.protobuf.CodedInputStream.isAtEnd(CodedInputStream.java:701)
        at com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:99)
        at org.apache.flume.channel.file.proto.ProtosFactory$Put.<init>(ProtosFactory.java:3979)
        at org.apache.flume.channel.file.proto.ProtosFactory$Put.<init>(ProtosFactory.java:3943)
        at org.apache.flume.channel.file.proto.ProtosFactory$Put$1.parsePartialFrom(ProtosFactory.java:4039)
        at org.apache.flume.channel.file.proto.ProtosFactory$Put$1.parsePartialFrom(ProtosFactory.java:4034)
        at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
        at com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParser.java:241)
        at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:253)
        at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:259)
        at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:49)
        at org.apache.flume.channel.file.proto.ProtosFactory$Put.parseDelimitedFrom(ProtosFactory.java:4179)
        at org.apache.flume.channel.file.Put.readProtos(Put.java:97)
        at org.apache.flume.channel.file.TransactionEventRecord.fromByteArray(TransactionEventRecord.java:206)
        ... 10 more
08 Oct 2017 15:46:14,866 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:158)  - Unable to deliver event. Exception follows.
java.lang.IllegalStateException: Log is closed
        at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
        at org.apache.flume.channel.file.Log.getFlumeEventQueue(Log.java:591)
        at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.<init>(FileChannel.java:442)
        at org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:359)
        at org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
        at com.macys.daas.flume.sink.elasticsearch.CreateIndex.process(CreateIndex.java:136)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
        at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)