[jira] [Created] (FLUME-2748) ThriftLegacySource produces exception due to wrongly compiled thrift definitions

["Tobias Heintz (JIRA)" <ji...@apache.org>]

Tobias Heintz created FLUME-2748:
------------------------------------

             Summary: ThriftLegacySource produces exception due to wrongly compiled thrift definitions
                 Key: FLUME-2748
                 URL: https://issues.apache.org/jira/browse/FLUME-2748
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
    Affects Versions: v1.6.0
            Reporter: Tobias Heintz


We are in the process of upgrading our Flume installation from 0.9.2 to 1.6.0. Currently we are using the ThriftLegacySource to allow the Flume server to receive messages without having to update all components at the same time. For every received message, we are seeing this exception:
{code}
2015-07-24 17:15:28,892 (pool-3-thread-5) [ERROR - org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:215)] Error occurred during processing of message.

java.lang.NullPointerException
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:53)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
{code}

I've done some digging in the code and it appears that there is an error in the Java classes that were compiled from the legacy thrift definitions: the method [{{append}} is defined as {{oneway}}|https://github.com/apache/flume/blob/344e0accae5675fd3d14b8414531528607865aae/flume-ng-legacy-sources/flume-thrift-source/src/main/thrift/flumeCompatibility.thrift#L61], however in the compiled class, the method [{{isOneway()}} returns {{false}}|https://github.com/apache/flume/blob/344e0accae5675fd3d14b8414531528607865aae/flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java#L223]. This then leads to the NullPointerException, when the [ProcessFunction tries to write the result|https://github.com/apache/thrift/blob/master/lib/java/src/org/apache/thrift/ProcessFunction.java#L53] back to the producer.

I'm not sure how this happened, maybe the very old version (0.7) of the thrift compiler is at fault here. The fix however would be to simply make the {{isOneway()}} method return {{true}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)