You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Mostafa T <mt...@gmail.com> on 2015/03/01 11:20:49 UTC
Source not listening on defined port
Hello,
I have flume 1.5.1.2.2 as part of Hortonworks data platform (HDP) 2.2 on
Windows server 2012 R2
I have configured flumeagent source as avro, sink as hdfs. When I tried to
connect through a client via the api I got connection refused.
I tried to check if the defined port is active via netstat, I got nothing,
which indicate that the port is not active.
I turned the firewall off but with no avail.
*Here's the flume.conf*:
a1.channels = c1
a1.sources = r1
a1.sinks = k1
a1.channels.c1.type = memory
a1.channels.c1.capacity = 10000
a1.channels.c1.transactionCapacity = 10000
a1.sources.r1.channels = c1
a1.sources.r1.type = avro
a1.sources.r1.bind = 10.10.206.18
a1.sources.r1.port = 41414
a1.sinks.k1.channel = c1
a1.sinks.k1.type = hdfs
a1.sinks.k1.hdfs.path = hdfs://10.10.206.18:8020/user/flume/flume_test
*here's the flume.log*:
01 Mar 2015 11:59:30,916 INFO [lifecycleSupervisor-1-0]
(org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start:61)
- Configuration provider starting
01 Mar 2015 11:59:30,930 INFO [conf-file-poller-0]
(org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:133)
- Reloading configuration
file:C:\hdp\flume-1.5.1.2.2.0.0-2041\conf\flume.conf
01 Mar 2015 11:59:30,940 INFO [conf-file-poller-0]
(org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016)
- Processing:k1
01 Mar 2015 11:59:30,941 INFO [conf-file-poller-0]
(org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:930)
- Added sinks: k1 Agent: a1
01 Mar 2015 11:59:30,941 INFO [conf-file-poller-0]
(org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016)
- Processing:k1
01 Mar 2015 11:59:30,942 INFO [conf-file-poller-0]
(org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016)
- Processing:k1
01 Mar 2015 11:59:30,975 INFO [conf-file-poller-0]
(org.apache.flume.conf.FlumeConfiguration.validateConfiguration:140) -
Post-validation flume configuration contains configuration for agents: [a1]
01 Mar 2015 11:59:30,976 WARN [conf-file-poller-0]
(org.apache.flume.node.AbstractConfigurationProvider.getConfiguration:138)
- No configuration found for this host:agent
01 Mar 2015 11:59:30,991 INFO [conf-file-poller-0]
(org.apache.flume.node.Application.startAllComponents:138) - Starting new
configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
Any idea what could be the problem?
Re: Source not listening on defined port
Posted by Hari Shreedharan <hs...@cloudera.com>.
I don't know how the agent is being started, but changing the agent name in the conf from a1 to agent should fix this. The log does say that it can't find the agent's config.
Thanks, Hari
On Sun, Mar 1, 2015 at 2:21 AM, Mostafa T <mt...@gmail.com> wrote:
> Hello,
> I have flume 1.5.1.2.2 as part of Hortonworks data platform (HDP) 2.2 on
> Windows server 2012 R2
> I have configured flumeagent source as avro, sink as hdfs. When I tried to
> connect through a client via the api I got connection refused.
> I tried to check if the defined port is active via netstat, I got nothing,
> which indicate that the port is not active.
> I turned the firewall off but with no avail.
> *Here's the flume.conf*:
> a1.channels = c1
> a1.sources = r1
> a1.sinks = k1
> a1.channels.c1.type = memory
> a1.channels.c1.capacity = 10000
> a1.channels.c1.transactionCapacity = 10000
> a1.sources.r1.channels = c1
> a1.sources.r1.type = avro
> a1.sources.r1.bind = 10.10.206.18
> a1.sources.r1.port = 41414
> a1.sinks.k1.channel = c1
> a1.sinks.k1.type = hdfs
> a1.sinks.k1.hdfs.path = hdfs://10.10.206.18:8020/user/flume/flume_test
> *here's the flume.log*:
> 01 Mar 2015 11:59:30,916 INFO [lifecycleSupervisor-1-0]
> (org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start:61)
> - Configuration provider starting
> 01 Mar 2015 11:59:30,930 INFO [conf-file-poller-0]
> (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:133)
> - Reloading configuration
> file:C:\hdp\flume-1.5.1.2.2.0.0-2041\conf\flume.conf
> 01 Mar 2015 11:59:30,940 INFO [conf-file-poller-0]
> (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016)
> - Processing:k1
> 01 Mar 2015 11:59:30,941 INFO [conf-file-poller-0]
> (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:930)
> - Added sinks: k1 Agent: a1
> 01 Mar 2015 11:59:30,941 INFO [conf-file-poller-0]
> (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016)
> - Processing:k1
> 01 Mar 2015 11:59:30,942 INFO [conf-file-poller-0]
> (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:1016)
> - Processing:k1
> 01 Mar 2015 11:59:30,975 INFO [conf-file-poller-0]
> (org.apache.flume.conf.FlumeConfiguration.validateConfiguration:140) -
> Post-validation flume configuration contains configuration for agents: [a1]
> 01 Mar 2015 11:59:30,976 WARN [conf-file-poller-0]
> (org.apache.flume.node.AbstractConfigurationProvider.getConfiguration:138)
> - No configuration found for this host:agent
> 01 Mar 2015 11:59:30,991 INFO [conf-file-poller-0]
> (org.apache.flume.node.Application.startAllComponents:138) - Starting new
> configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
> Any idea what could be the problem?