You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Marimuthu Ponnambalam (JIRA)" <ji...@apache.org> on 2014/06/18 10:22:05 UTC
[jira] [Created] (FLUME-2405) Flume stops working on arbitrary
Frame Size more than 16 MB on ScribeSource
Marimuthu Ponnambalam created FLUME-2405:
--------------------------------------------
Summary: Flume stops working on arbitrary Frame Size more than 16 MB on ScribeSource
Key: FLUME-2405
URL: https://issues.apache.org/jira/browse/FLUME-2405
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.5.0, v1.4.0
Reporter: Marimuthu Ponnambalam
Priority: Critical
Fix For: v1.5.0.1, v1.6.0, v1.5.0, v1.4.0
Flume stops receiving data on scribe source if there is an arbitrary size of data send from scribe, beyond 16MB.
TFrameTransport throws exception, due to frame size is more than 16 MB, which is default size on TFrameTransport.
This can be resolved by adding one more property "thriftFrameSize", which will allow custom frame size be configured, which will be used to create TFrameTransport.
This happens while scribe lag is being cleared, scribe sends Frame size larger than 16MB.
http://stackoverflow.com/questions/24150439/flume-scribesource-how-to-set-thrift-frame-size
2014-06-10 19:40:40,405 WARN org.apache.thrift.server.THsHaServer: Exception while invoking!
org.apache.thrift.transport.TTransportException: Frame size (23757404) larger than max length (16384000)!
at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:137)
at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478)
at org.apache.thrift.server.Invocation.run(Invocation.java:18)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
--
This message was sent by Atlassian JIRA
(v6.2#6252)