You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Eloise Carlton <ec...@google.com> on 2011/06/27 04:06:19 UTC
java.nio.BufferUnderflowException
>
> The following java.nio.BufferUnderflowException happens at the final
> write to output file stage of a flume pipeline:
>
> Jun 25, 2011 12:19:26 PM com.google.mapreduce.ReduceDispatcher
> reduceCall
> SEVERE: java.nio.BufferUnderflowException
> at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:145)
> at
> com.google.io.file.jni.GoogleChannel.write(GoogleChannel.java:230)
> at
> com.google.pipeline.flume.rt.TextSink.writeToOutputChannel(TextSink.java:
> 135)
> at com.google.pipeline.flume.rt.TextSink.emit(TextSink.java:112)
> at com.google.pipeline.flume.rt.TextSink.emit(TextSink.java:23)
> at com.google.pipeline.flume.rt.JailableMRSinks
> $3.emit(JailableMRSinks.java:126)
> at com.google.pipeline.flume.rt.JailableMRSinks
> $3.emit(JailableMRSinks.java:101)
> at com.google.pipeline.flume.rt.JailableMSCRReducers
> $2.reduce(JailableMSCRReducers.java:190)
> at com.google.pipeline.flume.rt.MSCRReducer.reduce(MSCRReducer.java:
> 435)
> at
>
> com.google.mapreduce.ReduceDispatcher.reduceCallPropagatingExns(ReduceDispatcher.java:
> 138)
> at
> com.google.mapreduce.ReduceDispatcher.reduceCall(ReduceDispatcher.java:
> 109)
> at
>
> com.google.mapreduce.NativeReduceDispatcher.reduceCall(NativeReduceDispatcher.java:
> 107)
>
> F0625 12:19:36.007751 20648 reducerproxyjni.cc:121] Unexpected
> exception from invocation of Reducer
> *** Check failure stack trace: ***
> @ 0x2b66edc41bc6 LogMessage::SendToLog()
> @ 0x2b66edc3e4b0 LogMessage::Flush()
> @ 0x2b66edc3e589 LogMessageFatal::~LogMessageFatal()
> @ 0x2b66eb7ab622 ReducerProxyJNI::Reduce()
> @ 0x2b66ebae2092 mapreduce::ReduceOutputState::Reduce()
> @ 0x2b66ebaedfe2 mapreduce::ReduceImpl::DoReduce()
> @ 0x2b66ebb2ed4a mapreduce::MR_Worker::Reduce()
> @ 0x2b66ebb1edb9 _MemberResultCallback_5_0<>::Run()
> @ 0x2b66ebcff64d
> mapreduce::EventDispatcher::RunAndDecrQcount()
> @ 0x2b66ebcfe49d _MemberResultCallback_2_0<>::Run()
> @ 0x2b66edb87706 ThreadPoolWorker::Run()
> @ 0x2b66edb833c2 Thread::ThreadBody()
> @ 0x2b657596ab9a start_thread
> *** SIGABRT received by PID 20445 (TID 20648) from PID 20445; ***
> F0625 12:19:36.007751 20648 reducerproxyjni.cc:121] Unexpected
> exception from invocation of Reducer
>
>
>
> What would cause java.nio.BufferUnderflowException? The output file is a
> text file on cns.
>
> Thanks,
> Eloise
>
Re: java.nio.BufferUnderflowException
Posted by Jonathan Hsieh <jo...@cloudera.com>.
Hi Eloise,
I'm not familiar with the code your stack trace is showing us. Based on
some of the namespaces in the code, I think this may be an error in the
Google-internal FlumeJava System. (There was an publication about this
system a few years back).
This is a mailing list for Apache Flume (an open source log collection
system initially created by Cloudera) for Apache Hadoop (an open source
MapReduce implementation that Yahoo! has contributed to significantly).
Unfortunately, I didn't read the message very carefully when I asked you to
forward it to this list. Sorry to forward you again, but my guess is that
you may want to send this message to a Google internal mailing list.
Jon
On Sun, Jun 26, 2011 at 7:06 PM, Eloise Carlton <ec...@google.com> wrote:
> The following java.nio.BufferUnderflowException happens at the final
>> write to output file stage of a flume pipeline:
>>
>> Jun 25, 2011 12:19:26 PM com.google.mapreduce.ReduceDispatcher
>> reduceCall
>> SEVERE: java.nio.BufferUnderflowException
>> at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:145)
>> at
>> com.google.io.file.jni.GoogleChannel.write(GoogleChannel.java:230)
>> at
>> com.google.pipeline.flume.rt.TextSink.writeToOutputChannel(TextSink.java:
>> 135)
>> at com.google.pipeline.flume.rt.TextSink.emit(TextSink.java:112)
>> at com.google.pipeline.flume.rt.TextSink.emit(TextSink.java:23)
>> at com.google.pipeline.flume.rt.JailableMRSinks
>> $3.emit(JailableMRSinks.java:126)
>> at com.google.pipeline.flume.rt.JailableMRSinks
>> $3.emit(JailableMRSinks.java:101)
>> at com.google.pipeline.flume.rt.JailableMSCRReducers
>> $2.reduce(JailableMSCRReducers.java:190)
>> at
>> com.google.pipeline.flume.rt.MSCRReducer.reduce(MSCRReducer.java:
>> 435)
>> at
>>
>> com.google.mapreduce.ReduceDispatcher.reduceCallPropagatingExns(ReduceDispatcher.java:
>> 138)
>> at
>> com.google.mapreduce.ReduceDispatcher.reduceCall(ReduceDispatcher.java:
>> 109)
>> at
>>
>> com.google.mapreduce.NativeReduceDispatcher.reduceCall(NativeReduceDispatcher.java:
>> 107)
>>
>> F0625 12:19:36.007751 20648 reducerproxyjni.cc:121] Unexpected
>> exception from invocation of Reducer
>> *** Check failure stack trace: ***
>> @ 0x2b66edc41bc6 LogMessage::SendToLog()
>> @ 0x2b66edc3e4b0 LogMessage::Flush()
>> @ 0x2b66edc3e589 LogMessageFatal::~LogMessageFatal()
>> @ 0x2b66eb7ab622 ReducerProxyJNI::Reduce()
>> @ 0x2b66ebae2092 mapreduce::ReduceOutputState::Reduce()
>> @ 0x2b66ebaedfe2 mapreduce::ReduceImpl::DoReduce()
>> @ 0x2b66ebb2ed4a mapreduce::MR_Worker::Reduce()
>> @ 0x2b66ebb1edb9 _MemberResultCallback_5_0<>::Run()
>> @ 0x2b66ebcff64d
>> mapreduce::EventDispatcher::RunAndDecrQcount()
>> @ 0x2b66ebcfe49d _MemberResultCallback_2_0<>::Run()
>> @ 0x2b66edb87706 ThreadPoolWorker::Run()
>> @ 0x2b66edb833c2 Thread::ThreadBody()
>> @ 0x2b657596ab9a start_thread
>> *** SIGABRT received by PID 20445 (TID 20648) from PID 20445; ***
>> F0625 12:19:36.007751 20648 reducerproxyjni.cc:121] Unexpected
>> exception from invocation of Reducer
>>
>>
>>
>> What would cause java.nio.BufferUnderflowException? The output file is a
>> text file on cns.
>>
>> Thanks,
>> Eloise
>>
>
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com