答复: Flume sink exeption:org.apache.flume.EventDeliveryException: Failed to send events

[罗辉 <lu...@ifeng.com>]

Hi Iain,
     telnet is ok,  if I switch the sink to logger. Please check below log:
source:
[hadoop@master60 flume]$ telnet localhost 44444
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
abc
OK
abc
OK
gggd
OK


Sink:
16/08/29 09:57:16 INFO node.Application: Starting Sink k1
16/08/29 09:57:16 INFO node.Application: Starting Source r1
16/08/29 09:57:16 INFO source.NetcatSource: Source starting
16/08/29 09:57:16 INFO source.NetcatSource: Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:44444]
16/08/29 09:58:10 INFO sink.LoggerSink: Event: { headers:{} body: 61 62 63 0D                                     abc. }
16/08/29 09:58:14 INFO sink.LoggerSink: Event: { headers:{} body: 61 62 63 0D                                     abc. }
16/08/29 09:58:16 INFO sink.LoggerSink: Event: { headers:{} body: 67 67 67 64 0D                                  gggd. }

Seems it is ok when the sink is logger , but when I change the sink type from logger to avro. It failed with previous exception.



发件人: iain wright [mailto:iainwrig@gmail.com]
发送时间: 2016年8月27日 0:13
收件人: user@flume.apache.org
主题: Re: Flume sink exeption:org.apache.flume.EventDeliveryException: Failed to send events

Can you telnet master60 31002 from the server the flume agent is running on?

Sent from my iPhone

On Aug 26, 2016, at 1:00 AM, 罗辉 <lu...@ifeng.com>> wrote:
31002

Re: Flume sink exeption:org.apache.flume.EventDeliveryException: Failed to send events

[Denes Arvay <de...@cloudera.com>]

Hi,

Isn't it possible that your spark streaming application is not running when
flume tries to send events to it? That might be the reason of flume not
being able to connect to the port 31002.
According to the Spark streaming + Flume integration guide (
http://spark.apache.org/docs/latest/streaming-flume-integration.html) when
you use the push-based approach "the streaming application needs to be up,
with the receiver scheduled and listening on the chosen port, for Flume to
be able push data."

On the other hand you might want to consider using the more reliable
pull-based approach.

Regards,
Denes

On Mon, Aug 29, 2016 at 4:02 AM 罗辉 <lu...@ifeng.com> wrote:

> Hi Iain,
>
>      telnet is ok,  if I switch the sink to logger. Please check below log:
>
> source:
>
> [hadoop@master60 flume]$ telnet localhost 44444
>
> Trying ::1...
>
> telnet: connect to address ::1: Connection refused
>
> Trying 127.0.0.1...
>
> Connected to localhost.
>
> Escape character is '^]'.
>
> abc
>
> OK
>
> abc
>
> OK
>
> gggd
>
> OK
>
>
>
>
>
> Sink:
>
> 16/08/29 09:57:16 INFO node.Application: Starting Sink k1
>
> 16/08/29 09:57:16 INFO node.Application: Starting Source r1
>
> 16/08/29 09:57:16 INFO source.NetcatSource: Source starting
>
> 16/08/29 09:57:16 INFO source.NetcatSource: Created
> serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:44444]
>
> 16/08/29 09:58:10 INFO sink.LoggerSink: Event: { headers:{} body: 61 62 63
> 0D                                     abc. }
>
> 16/08/29 09:58:14 INFO sink.LoggerSink: Event: { headers:{} body: 61 62 63
> 0D                                     abc. }
>
> 16/08/29 09:58:16 INFO sink.LoggerSink: Event: { headers:{} body: 67 67 67
> 64 0D                                  gggd. }
>
>
>
> Seems it is ok when the sink is logger , but when I change the sink type
> from logger to avro. It failed with previous exception.
>
>
>
>
>
>
>
> *发件人:* iain wright [mailto:iainwrig@gmail.com]
> *发送时间:* 2016年8月27日 0:13
> *收件人:* user@flume.apache.org
> *主题:* Re: Flume sink exeption:org.apache.flume.EventDeliveryException:
> Failed to send events
>
>
>
> Can you telnet master60 31002 from the server the flume agent is running
> on?
>
>
> Sent from my iPhone
>
>
> On Aug 26, 2016, at 1:00 AM, 罗辉 <lu...@ifeng.com> wrote:
>
> 31002
>
>