You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by centerqi hu <ce...@gmail.com> on 2014/09/18 03:34:42 UTC
Flume Avro sink failure does not retry
Hi all
My flume version is 1.5.0
Flume Avro sink failure does not retry.
When the network timeout, Flume Avro sink failed, but he will not try again.
The failure of the sink will not send data.
How to let him to resend the data?
thanks
oyConnection(AbstractRpcSink.java:249)] Rpc sink k1 closing Rpc
client: NettyAvroRpcClient { host: 10.1.1.1, port: 1463 }
2014-08-28 22:55:36,264
(SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -
org.apache.flume.SinkRunner$PollingRunner.r
un(SinkRunner.java:160)] Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to send events
at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient
{ host: 10.1.1.1, port: 146
3 }: Failed to send batch
at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:311)
at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376)
... 3 more10.1.1.1, port: 146
3 }: Handshake timed out after 30000ms
at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:355)
at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:299)
... 4 more
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:353)
... 5 more
--
centerqi@gmail.com|齐忠
Re: Flume Avro sink failure does not retry
Posted by Anandkumar Lakshmanan <an...@orzota.com>.
Hi,
I have not seen the following error after using the two properties. And
I am not sure that, is there any other options.
Anand.
On 09/18/2014 02:20 PM, centerqi hu wrote:
> Thank you very much, but is there a better way? I prepared to code
> this function
>
> 2014-09-18 14:37 GMT+08:00 Anandkumar Lakshmanan <anand@orzota.com
> <ma...@orzota.com>>:
>
> Hi,
>
> Try increasing connect-timeout for avro sink. set it to 100000ms
> and also add request-timeout=100000.
>
> Anand.
>
>
> On 09/18/2014 07:04 AM, centerqi hu wrote:
>
> Hi all
>
> My flume version is 1.5.0
> Flume Avro sink failure does not retry.
> When the network timeout, Flume Avro sink failed, but he will
> not try again.
> The failure of the sink will not send data.
> How to let him to resend the data?
>
> thanks
>
> oyConnection(AbstractRpcSink.java:249)] Rpc sink k1 closing Rpc
> client: NettyAvroRpcClient { host: 10.1.1.1, port: 1463 }
> 2014-08-28 22:55:36,264
> (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -
> org.apache.flume.SinkRunner$PollingRunner.r
> un(SinkRunner.java:160)] Unable to deliver event. Exception
> follows.
> org.apache.flume.EventDeliveryException: Failed to send events
> at
> org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.flume.EventDeliveryException:
> NettyAvroRpcClient
> { host: 10.1.1.1, port: 146
> 3 }: Failed to send batch
> at
> org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:311)
> at
> org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376)
> ... 3 more10.1.1.1, port: 146
> 3 }: Handshake timed out after 30000ms
> at
> org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:355)
> at
> org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:299)
> ... 4 more
> Caused by: java.util.concurrent.TimeoutException
> at
> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258)
> at
> java.util.concurrent.FutureTask.get(FutureTask.java:119)
> at
> org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:353)
> ... 5 more
>
>
>
>
>
> --
> centerqi@gmail.com <ma...@gmail.com>|齐忠
Re: Flume Avro sink failure does not retry
Posted by centerqi hu <ce...@gmail.com>.
Thank you very much, but is there a better way? I prepared to code this
function
2014-09-18 14:37 GMT+08:00 Anandkumar Lakshmanan <an...@orzota.com>:
> Hi,
>
> Try increasing connect-timeout for avro sink. set it to 100000ms and also
> add request-timeout=100000.
>
> Anand.
>
>
> On 09/18/2014 07:04 AM, centerqi hu wrote:
>
>> Hi all
>>
>> My flume version is 1.5.0
>> Flume Avro sink failure does not retry.
>> When the network timeout, Flume Avro sink failed, but he will not try
>> again.
>> The failure of the sink will not send data.
>> How to let him to resend the data?
>>
>> thanks
>>
>> oyConnection(AbstractRpcSink.java:249)] Rpc sink k1 closing Rpc
>> client: NettyAvroRpcClient { host: 10.1.1.1, port: 1463 }
>> 2014-08-28 22:55:36,264
>> (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -
>> org.apache.flume.SinkRunner$PollingRunner.r
>> un(SinkRunner.java:160)] Unable to deliver event. Exception follows.
>> org.apache.flume.EventDeliveryException: Failed to send events
>> at org.apache.flume.sink.AbstractRpcSink.process(
>> AbstractRpcSink.java:392)
>> at org.apache.flume.sink.DefaultSinkProcessor.process(
>> DefaultSinkProcessor.java:68)
>> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.
>> java:147)
>> at java.lang.Thread.run(Thread.java:722)
>> Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient
>> { host: 10.1.1.1, port: 146
>> 3 }: Failed to send batch
>> at org.apache.flume.api.NettyAvroRpcClient.appendBatch(
>> NettyAvroRpcClient.java:311)
>> at org.apache.flume.sink.AbstractRpcSink.process(
>> AbstractRpcSink.java:376)
>> ... 3 more10.1.1.1, port: 146
>> 3 }: Handshake timed out after 30000ms
>> at org.apache.flume.api.NettyAvroRpcClient.appendBatch(
>> NettyAvroRpcClient.java:355)
>> at org.apache.flume.api.NettyAvroRpcClient.appendBatch(
>> NettyAvroRpcClient.java:299)
>> ... 4 more
>> Caused by: java.util.concurrent.TimeoutException
>> at java.util.concurrent.FutureTask$Sync.innerGet(
>> FutureTask.java:258)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:119)
>> at org.apache.flume.api.NettyAvroRpcClient.appendBatch(
>> NettyAvroRpcClient.java:353)
>> ... 5 more
>>
>>
>
--
centerqi@gmail.com|齐忠
Re: Flume Avro sink failure does not retry
Posted by Anandkumar Lakshmanan <an...@orzota.com>.
Hi,
Try increasing connect-timeout for avro sink. set it to 100000ms and
also add request-timeout=100000.
Anand.
On 09/18/2014 07:04 AM, centerqi hu wrote:
> Hi all
>
> My flume version is 1.5.0
> Flume Avro sink failure does not retry.
> When the network timeout, Flume Avro sink failed, but he will not try again.
> The failure of the sink will not send data.
> How to let him to resend the data?
>
> thanks
>
> oyConnection(AbstractRpcSink.java:249)] Rpc sink k1 closing Rpc
> client: NettyAvroRpcClient { host: 10.1.1.1, port: 1463 }
> 2014-08-28 22:55:36,264
> (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -
> org.apache.flume.SinkRunner$PollingRunner.r
> un(SinkRunner.java:160)] Unable to deliver event. Exception follows.
> org.apache.flume.EventDeliveryException: Failed to send events
> at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392)
> at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient
> { host: 10.1.1.1, port: 146
> 3 }: Failed to send batch
> at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:311)
> at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376)
> ... 3 more10.1.1.1, port: 146
> 3 }: Handshake timed out after 30000ms
> at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:355)
> at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:299)
> ... 4 more
> Caused by: java.util.concurrent.TimeoutException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258)
> at java.util.concurrent.FutureTask.get(FutureTask.java:119)
> at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:353)
> ... 5 more
>