You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by terreyshih <te...@gmail.com> on 2014/10/20 08:21:26 UTC

org.apache.avro.AvroRuntimeException: Excessively large list allocation request detected: 

Hi,

I am seeing this exception while sending avro packets from one agent to another.  It does not seem to affect my business entities though.

I looked at the avro source code and it seems like the exception is handled by closing the channel and new data is created.

Are the packets lost ?  what does this exception really mean ?

thx,
-terrey

Flume stats

Posted by Ed Judge <ej...@gmail.com>.
Does anyone know if there are byte level statistics available with Flume?

$ curl --fail --silent --show-error http://localhost:14001/metrics
{"SOURCE.r1":{"OpenConnectionCount":"0","Type":"SOURCE","AppendBatchAcceptedCount":"0","AppendBatchReceivedCount":"0","EventAcceptedCount":"4480","StopTime":"0","AppendReceivedCount":"0","StartTime":"1414115468065","EventReceivedCount":"4491","AppendAcceptedCount":"0"},"CHANNEL.c1":{"EventPutSuccessCount":"4480","ChannelFillPercentage":"0.4479","Type":"CHANNEL","StopTime":"0","EventPutAttemptCount":"4362","ChannelSize":"4359","StartTime":"1414115467779","EventTakeSuccessCount":"0","ChannelCapacity":"1000000","EventTakeAttemptCount":"1"},"SINK.k1":{"BatchCompleteCount":"0","ConnectionFailedCount":"0","EventDrainAttemptCount":"0","ConnectionCreatedCount":"0","Type":"SINK","BatchEmptyCount":"0","ConnectionClosedCount":"0","EventDrainSuccessCount":"0","StopTime":"0","StartTime":"1414115468032","BatchUnderflowCount":"0”}}

Thanks,
Ed

Re: org.apache.avro.AvroRuntimeException: Excessively large list allocation request detected:

Posted by Hari Shreedharan <hs...@cloudera.com>.
No, even if it throws an exception it is fine, Avro Sink will retry the entire batch.


Thanks,
Hari

On Sun, Oct 19, 2014 at 11:21 PM, terreyshih <te...@gmail.com> wrote:

> Hi,
> I am seeing this exception while sending avro packets from one agent to another.  It does not seem to affect my business entities though.
> I looked at the avro source code and it seems like the exception is handled by closing the channel and new data is created.
> Are the packets lost ?  what does this exception really mean ?
> thx,
> -terrey