You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Sai Sai <sa...@yahoo.in> on 2013/05/20 14:09:45 UTC

Re: Flume port issue

Not sure if this is the right group to ask questions about flume:

I am getting an exception about unable to open a port in flume when trying to create a remote agent, more details below:
---------------------------
13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC connection error
---------------------------


Here r the steps i have followed:

Step 1: Here is my agent3.conf created in the flume/conf dir:

******
agent3.sources = avrosource
agent3.sinks = filesink
agent3.channels = jdbcchannel

agent3.sources.avrosource.type = avro
agent3.sources.avrosource.bind = localhost
agent3.sources.avrosource.port = 4000
agent3.sources.avrosource.threads = 5

agent3.sinks.filesink.type = FILE_ROLL
agent3.sinks.filesink.sink.directory = /home/satish/work/apache-flume-1.3.1-bin/files
agent3.sinks.filesink.sink.rollInterval = 0

agent3.channels.jdbcchannel.type = jdbc

agent3.sources.avrosource.channels = jdbcchannel
agent3.sinks.filesink.channel = jdbcchannel

******


Step 2: Then i have saved it successfully and created a new test file like this:

Step 3: echo "Hello World" > /home/satish/message3

Step 4: Tried executing this command:

./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3

I get this exception below, please help:

------------------------------

Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F /usr/logs/log.10
13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC connection error
at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
... 5 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)


Please help.
Thanks
Sai

Re: Flume port issue

Posted by Paul Wilkinson <pa...@gmail.com>.

Reattempting to post this to the flume ML after subscribing...


On 21 May 2013 16:29, Paul Wilkinson <pa...@gmail.com> wrote:

> This could be a myriad of things, but some points to note:
>
> 1. Flume has its own mailing list - I've moved your post there instead.
> 2. Your command line suggests connecting to localhost:4000, but the log
> shows ubuntu:41414 instead (different attempts maybe?)
> 3. The correct type for file roll sink is "file_roll", not "FILE_ROLL".
>
> I'd suggest taking a look at the logs for the agent itself, rather than
> the Avro Client - chances are the agent isn't starting up due to the config
> problems.
>
> Paul
>
>
> On 21 May 2013, at 08:47, Sai Sai <sa...@yahoo.in> wrote:
>
> Just a friendly follow up to see if anyone has any suggestions for the
> issue with port given below.
> Any help is appreciated.
> Thanks
> Sai
>
> On May 20, 2013 5:40 PM, "Sai Sai" <sa...@yahoo.in> wrote:
>
> Not sure if this is the right group to ask questions about flume:
>
> I am getting an exception about unable to open a port in flume when trying
> to create a remote agent, more details below:
> ---------------------------
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to
> Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port:
> 41414 }: RPC connection error
> ---------------------------
>
> Here r the steps i have followed:
>
> Step 1: Here is my agent3.conf created in the flume/conf dir:
>
> ******
> agent3.sources = avrosource
> agent3.sinks = filesink
> agent3.channels = jdbcchannel
>
> agent3.sources.avrosource.type = avro
> agent3.sources.avrosource.bind = localhost
> agent3.sources.avrosource.port = 4000
> agent3.sources.avrosource.threads = 5
>
> agent3.sinks.filesink.type = FILE_ROLL
> agent3.sinks.filesink.sink.directory =
> /home/satish/work/apache-flume-1.3.1-bin/files
> agent3.sinks.filesink.sink.rollInterval = 0
>
> agent3.channels.jdbcchannel.type = jdbc
>
> agent3.sources.avrosource.channels = jdbcchannel
> agent3.sinks.filesink.channel = jdbcchannel
>
> ******
>
> Step 2: Then i have saved it successfully and created a new test file like
> this:
>
> Step 3: echo "Hello World" > /home/satish/message3
>
> Step 4: Tried executing this command:
>
> ./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3
>
> I get this exception below, please help:
>
> ------------------------------
>
> Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32
> org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F
> /usr/logs/log.10
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to
> Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port:
> 41414 }: RPC connection error
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>  at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> at
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> at
> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>  at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
> at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
> Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
> at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
>  at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
>  at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused
>  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
>  at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
>  at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  at java.lang.Thread.run(Thread.java:679)
>
>
> Please help.
> Thanks
> Sai
>
>
>
>
>
>
>
>
>

Re: Flume port issue

Posted by Paul Wilkinson <pa...@gmail.com>.

Reattempting to post this to the flume ML after subscribing...


On 21 May 2013 16:29, Paul Wilkinson <pa...@gmail.com> wrote:

> This could be a myriad of things, but some points to note:
>
> 1. Flume has its own mailing list - I've moved your post there instead.
> 2. Your command line suggests connecting to localhost:4000, but the log
> shows ubuntu:41414 instead (different attempts maybe?)
> 3. The correct type for file roll sink is "file_roll", not "FILE_ROLL".
>
> I'd suggest taking a look at the logs for the agent itself, rather than
> the Avro Client - chances are the agent isn't starting up due to the config
> problems.
>
> Paul
>
>
> On 21 May 2013, at 08:47, Sai Sai <sa...@yahoo.in> wrote:
>
> Just a friendly follow up to see if anyone has any suggestions for the
> issue with port given below.
> Any help is appreciated.
> Thanks
> Sai
>
> On May 20, 2013 5:40 PM, "Sai Sai" <sa...@yahoo.in> wrote:
>
> Not sure if this is the right group to ask questions about flume:
>
> I am getting an exception about unable to open a port in flume when trying
> to create a remote agent, more details below:
> ---------------------------
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to
> Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port:
> 41414 }: RPC connection error
> ---------------------------
>
> Here r the steps i have followed:
>
> Step 1: Here is my agent3.conf created in the flume/conf dir:
>
> ******
> agent3.sources = avrosource
> agent3.sinks = filesink
> agent3.channels = jdbcchannel
>
> agent3.sources.avrosource.type = avro
> agent3.sources.avrosource.bind = localhost
> agent3.sources.avrosource.port = 4000
> agent3.sources.avrosource.threads = 5
>
> agent3.sinks.filesink.type = FILE_ROLL
> agent3.sinks.filesink.sink.directory =
> /home/satish/work/apache-flume-1.3.1-bin/files
> agent3.sinks.filesink.sink.rollInterval = 0
>
> agent3.channels.jdbcchannel.type = jdbc
>
> agent3.sources.avrosource.channels = jdbcchannel
> agent3.sinks.filesink.channel = jdbcchannel
>
> ******
>
> Step 2: Then i have saved it successfully and created a new test file like
> this:
>
> Step 3: echo "Hello World" > /home/satish/message3
>
> Step 4: Tried executing this command:
>
> ./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3
>
> I get this exception below, please help:
>
> ------------------------------
>
> Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32
> org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F
> /usr/logs/log.10
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to
> Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port:
> 41414 }: RPC connection error
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>  at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> at
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> at
> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>  at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
> at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
> Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
> at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
>  at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
>  at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused
>  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
>  at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
>  at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  at java.lang.Thread.run(Thread.java:679)
>
>
> Please help.
> Thanks
> Sai
>
>
>
>
>
>
>
>
>

Re: Flume port issue

Posted by Paul Wilkinson <pa...@gmail.com>.

Reattempting to post this to the flume ML after subscribing...


On 21 May 2013 16:29, Paul Wilkinson <pa...@gmail.com> wrote:

> This could be a myriad of things, but some points to note:
>
> 1. Flume has its own mailing list - I've moved your post there instead.
> 2. Your command line suggests connecting to localhost:4000, but the log
> shows ubuntu:41414 instead (different attempts maybe?)
> 3. The correct type for file roll sink is "file_roll", not "FILE_ROLL".
>
> I'd suggest taking a look at the logs for the agent itself, rather than
> the Avro Client - chances are the agent isn't starting up due to the config
> problems.
>
> Paul
>
>
> On 21 May 2013, at 08:47, Sai Sai <sa...@yahoo.in> wrote:
>
> Just a friendly follow up to see if anyone has any suggestions for the
> issue with port given below.
> Any help is appreciated.
> Thanks
> Sai
>
> On May 20, 2013 5:40 PM, "Sai Sai" <sa...@yahoo.in> wrote:
>
> Not sure if this is the right group to ask questions about flume:
>
> I am getting an exception about unable to open a port in flume when trying
> to create a remote agent, more details below:
> ---------------------------
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to
> Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port:
> 41414 }: RPC connection error
> ---------------------------
>
> Here r the steps i have followed:
>
> Step 1: Here is my agent3.conf created in the flume/conf dir:
>
> ******
> agent3.sources = avrosource
> agent3.sinks = filesink
> agent3.channels = jdbcchannel
>
> agent3.sources.avrosource.type = avro
> agent3.sources.avrosource.bind = localhost
> agent3.sources.avrosource.port = 4000
> agent3.sources.avrosource.threads = 5
>
> agent3.sinks.filesink.type = FILE_ROLL
> agent3.sinks.filesink.sink.directory =
> /home/satish/work/apache-flume-1.3.1-bin/files
> agent3.sinks.filesink.sink.rollInterval = 0
>
> agent3.channels.jdbcchannel.type = jdbc
>
> agent3.sources.avrosource.channels = jdbcchannel
> agent3.sinks.filesink.channel = jdbcchannel
>
> ******
>
> Step 2: Then i have saved it successfully and created a new test file like
> this:
>
> Step 3: echo "Hello World" > /home/satish/message3
>
> Step 4: Tried executing this command:
>
> ./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3
>
> I get this exception below, please help:
>
> ------------------------------
>
> Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32
> org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F
> /usr/logs/log.10
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to
> Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port:
> 41414 }: RPC connection error
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>  at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> at
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> at
> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>  at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
> at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
> Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
> at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
>  at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
>  at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused
>  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
>  at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
>  at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  at java.lang.Thread.run(Thread.java:679)
>
>
> Please help.
> Thanks
> Sai
>
>
>
>
>
>
>
>
>

Re: Flume port issue

Posted by Paul Wilkinson <pa...@gmail.com>.

Reattempting to post this to the flume ML after subscribing...


On 21 May 2013 16:29, Paul Wilkinson <pa...@gmail.com> wrote:

> This could be a myriad of things, but some points to note:
>
> 1. Flume has its own mailing list - I've moved your post there instead.
> 2. Your command line suggests connecting to localhost:4000, but the log
> shows ubuntu:41414 instead (different attempts maybe?)
> 3. The correct type for file roll sink is "file_roll", not "FILE_ROLL".
>
> I'd suggest taking a look at the logs for the agent itself, rather than
> the Avro Client - chances are the agent isn't starting up due to the config
> problems.
>
> Paul
>
>
> On 21 May 2013, at 08:47, Sai Sai <sa...@yahoo.in> wrote:
>
> Just a friendly follow up to see if anyone has any suggestions for the
> issue with port given below.
> Any help is appreciated.
> Thanks
> Sai
>
> On May 20, 2013 5:40 PM, "Sai Sai" <sa...@yahoo.in> wrote:
>
> Not sure if this is the right group to ask questions about flume:
>
> I am getting an exception about unable to open a port in flume when trying
> to create a remote agent, more details below:
> ---------------------------
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to
> Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port:
> 41414 }: RPC connection error
> ---------------------------
>
> Here r the steps i have followed:
>
> Step 1: Here is my agent3.conf created in the flume/conf dir:
>
> ******
> agent3.sources = avrosource
> agent3.sinks = filesink
> agent3.channels = jdbcchannel
>
> agent3.sources.avrosource.type = avro
> agent3.sources.avrosource.bind = localhost
> agent3.sources.avrosource.port = 4000
> agent3.sources.avrosource.threads = 5
>
> agent3.sinks.filesink.type = FILE_ROLL
> agent3.sinks.filesink.sink.directory =
> /home/satish/work/apache-flume-1.3.1-bin/files
> agent3.sinks.filesink.sink.rollInterval = 0
>
> agent3.channels.jdbcchannel.type = jdbc
>
> agent3.sources.avrosource.channels = jdbcchannel
> agent3.sinks.filesink.channel = jdbcchannel
>
> ******
>
> Step 2: Then i have saved it successfully and created a new test file like
> this:
>
> Step 3: echo "Hello World" > /home/satish/message3
>
> Step 4: Tried executing this command:
>
> ./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3
>
> I get this exception below, please help:
>
> ------------------------------
>
> Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32
> org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F
> /usr/logs/log.10
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to
> Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port:
> 41414 }: RPC connection error
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>  at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> at
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> at
> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>  at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
> at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
> Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
> at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
>  at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
>  at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused
>  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
>  at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
>  at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  at java.lang.Thread.run(Thread.java:679)
>
>
> Please help.
> Thanks
> Sai
>
>
>
>
>
>
>
>
>

Re: Flume port issue

Posted by Paul Wilkinson <pa...@gmail.com>.

This could be a myriad of things, but some points to note:

1. Flume has its own mailing list - I've moved your post there instead.
2. Your command line suggests connecting to localhost:4000, but the log shows ubuntu:41414 instead (different attempts maybe?)
3. The correct type for file roll sink is "file_roll", not "FILE_ROLL".

I'd suggest taking a look at the logs for the agent itself, rather than the Avro Client - chances are the agent isn't starting up due to the config problems.

Paul


On 21 May 2013, at 08:47, Sai Sai <sa...@yahoo.in> wrote:

> Just a friendly follow up to see if anyone has any suggestions for the issue with port given below.
> Any help is appreciated.
> Thanks
> Sai
> 
> On May 20, 2013 5:40 PM, "Sai Sai" <sa...@yahoo.in> wrote:
> Not sure if this is the right group to ask questions about flume:
> 
> I am getting an exception about unable to open a port in flume when trying to create a remote agent, more details below:
> ---------------------------
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC connection error
> ---------------------------
> 
> Here r the steps i have followed:
> 
> Step 1: Here is my agent3.conf created in the flume/conf dir:
> 
> ******
> agent3.sources = avrosource
> agent3.sinks = filesink
> agent3.channels = jdbcchannel
> 
> agent3.sources.avrosource.type = avro
> agent3.sources.avrosource.bind = localhost
> agent3.sources.avrosource.port = 4000
> agent3.sources.avrosource.threads = 5
> 
> agent3.sinks.filesink.type = FILE_ROLL
> agent3.sinks.filesink.sink.directory = /home/satish/work/apache-flume-1.3.1-bin/files
> agent3.sinks.filesink.sink.rollInterval = 0
> 
> agent3.channels.jdbcchannel.type = jdbc
> 
> agent3.sources.avrosource.channels = jdbcchannel
> agent3.sinks.filesink.channel = jdbcchannel
> 
> ******
> 
> Step 2: Then i have saved it successfully and created a new test file like this:
> 
> Step 3: echo "Hello World" > /home/satish/message3
> 
> Step 4: Tried executing this command:
> 
> ./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3
> 
> I get this exception below, please help:
> 
> ------------------------------
> 
> Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F /usr/logs/log.10
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC connection error
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> 
> 	at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> 	at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
> 	at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
> 	at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
> Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
> 	at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
> 	at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> 	at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> 	... 5 more
> Caused by: java.net.ConnectException: Connection refused
> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:679)
> 
> 
> Please help.
> Thanks
> Sai
> 
> 
> 
> 
> 
> 
> 
>

Re: Flume port issue

Posted by Paul Wilkinson <pa...@gmail.com>.

This could be a myriad of things, but some points to note:

1. Flume has its own mailing list - I've moved your post there instead.
2. Your command line suggests connecting to localhost:4000, but the log shows ubuntu:41414 instead (different attempts maybe?)
3. The correct type for file roll sink is "file_roll", not "FILE_ROLL".

I'd suggest taking a look at the logs for the agent itself, rather than the Avro Client - chances are the agent isn't starting up due to the config problems.

Paul


On 21 May 2013, at 08:47, Sai Sai <sa...@yahoo.in> wrote:

> Just a friendly follow up to see if anyone has any suggestions for the issue with port given below.
> Any help is appreciated.
> Thanks
> Sai
> 
> On May 20, 2013 5:40 PM, "Sai Sai" <sa...@yahoo.in> wrote:
> Not sure if this is the right group to ask questions about flume:
> 
> I am getting an exception about unable to open a port in flume when trying to create a remote agent, more details below:
> ---------------------------
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC connection error
> ---------------------------
> 
> Here r the steps i have followed:
> 
> Step 1: Here is my agent3.conf created in the flume/conf dir:
> 
> ******
> agent3.sources = avrosource
> agent3.sinks = filesink
> agent3.channels = jdbcchannel
> 
> agent3.sources.avrosource.type = avro
> agent3.sources.avrosource.bind = localhost
> agent3.sources.avrosource.port = 4000
> agent3.sources.avrosource.threads = 5
> 
> agent3.sinks.filesink.type = FILE_ROLL
> agent3.sinks.filesink.sink.directory = /home/satish/work/apache-flume-1.3.1-bin/files
> agent3.sinks.filesink.sink.rollInterval = 0
> 
> agent3.channels.jdbcchannel.type = jdbc
> 
> agent3.sources.avrosource.channels = jdbcchannel
> agent3.sinks.filesink.channel = jdbcchannel
> 
> ******
> 
> Step 2: Then i have saved it successfully and created a new test file like this:
> 
> Step 3: echo "Hello World" > /home/satish/message3
> 
> Step 4: Tried executing this command:
> 
> ./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3
> 
> I get this exception below, please help:
> 
> ------------------------------
> 
> Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F /usr/logs/log.10
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC connection error
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> 
> 	at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> 	at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
> 	at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
> 	at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
> Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
> 	at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
> 	at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> 	at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> 	... 5 more
> Caused by: java.net.ConnectException: Connection refused
> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:679)
> 
> 
> Please help.
> Thanks
> Sai
> 
> 
> 
> 
> 
> 
> 
>

Re: Flume port issue

Posted by Paul Wilkinson <pa...@gmail.com>.

This could be a myriad of things, but some points to note:

1. Flume has its own mailing list - I've moved your post there instead.
2. Your command line suggests connecting to localhost:4000, but the log shows ubuntu:41414 instead (different attempts maybe?)
3. The correct type for file roll sink is "file_roll", not "FILE_ROLL".

I'd suggest taking a look at the logs for the agent itself, rather than the Avro Client - chances are the agent isn't starting up due to the config problems.

Paul


On 21 May 2013, at 08:47, Sai Sai <sa...@yahoo.in> wrote:

> Just a friendly follow up to see if anyone has any suggestions for the issue with port given below.
> Any help is appreciated.
> Thanks
> Sai
> 
> On May 20, 2013 5:40 PM, "Sai Sai" <sa...@yahoo.in> wrote:
> Not sure if this is the right group to ask questions about flume:
> 
> I am getting an exception about unable to open a port in flume when trying to create a remote agent, more details below:
> ---------------------------
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC connection error
> ---------------------------
> 
> Here r the steps i have followed:
> 
> Step 1: Here is my agent3.conf created in the flume/conf dir:
> 
> ******
> agent3.sources = avrosource
> agent3.sinks = filesink
> agent3.channels = jdbcchannel
> 
> agent3.sources.avrosource.type = avro
> agent3.sources.avrosource.bind = localhost
> agent3.sources.avrosource.port = 4000
> agent3.sources.avrosource.threads = 5
> 
> agent3.sinks.filesink.type = FILE_ROLL
> agent3.sinks.filesink.sink.directory = /home/satish/work/apache-flume-1.3.1-bin/files
> agent3.sinks.filesink.sink.rollInterval = 0
> 
> agent3.channels.jdbcchannel.type = jdbc
> 
> agent3.sources.avrosource.channels = jdbcchannel
> agent3.sinks.filesink.channel = jdbcchannel
> 
> ******
> 
> Step 2: Then i have saved it successfully and created a new test file like this:
> 
> Step 3: echo "Hello World" > /home/satish/message3
> 
> Step 4: Tried executing this command:
> 
> ./flume-ng avro-client -H localhost -p 4000 -F /home/satish/message3
> 
> I get this exception below, please help:
> 
> ------------------------------
> 
> Djava.library.path=:/home/satish/work/hadoop-1.0.4/libexec/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient -H ubuntu -p 41414 -F /usr/logs/log.10
> 13/05/20 04:55:30 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: ubuntu, port: 41414 }: RPC connection error
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> 
> 	at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> 	at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
> 	at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:180)
> 	at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:71)
> Caused by: java.io.IOException: Error connecting to ubuntu/127.0.0.1:41414
> 	at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
> 	at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> 	at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
> 	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> 	... 5 more
> Caused by: java.net.ConnectException: Connection refused
> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
> 	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:679)
> 
> 
> Please help.
> Thanks
> Sai
> 
> 
> 
> 
> 
> 
> 
>