You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by nagarjuna sarpuru <na...@gmail.com> on 2014/02/10 08:25:54 UTC

How too use Flume to generate logs from another machine

Hi,
I am new to this component flume .
I have one POC that I need to capture the logs from one machine where my
actual application is running ,I have IP address and port for that
particular machine..
For this I have declared one flume as follows:

 # http://flume.apache.org/FlumeUserGuide.html#exec-source

agent.sources = apache

agent.sources.apache.type = exec

agent.sources.apache.command = tail -F /var/log/auth.log

agent.sources.apache.batchSize = 1

agent.sources.apache.channels = memoryChannel

agent.sources.apache.interceptors = itime ihost itype

# http://flume.apache.org/FlumeUserGuide.html#timestamp-interceptor

agent.sources.apache.interceptors.itime.type = timestamp

# http://flume.apache.org/FlumeUserGuide.html#host-interceptor

agent.sources.apache.interceptors.ihost.type = host

agent.sources.apache.interceptors.ihost.useIP = false

agent.sources.apache.interceptors.ihost.hostHeader = host

# http://flume.apache.org/FlumeUserGuide.html#static-interceptor

agent.sources.apache.interceptors.itype.type = static

agent.sources.apache.interceptors.itype.key = log_type

agent.sources.apache.interceptors.itype.value = apache_access_combined

 # http://flume.apache.org/FlumeUserGuide.html#memory-channel

agent.channels = memoryChannel

agent.channels.memoryChannel.type = memory

agent.channels.memoryChannel.capacity = 100

 ## Send to Flume Collector on 1.2.3.4 (Hadoop Slave Node)

# http://flume.apache.org/FlumeUserGuide.html#avro-sink

agent.sinks = AvroSink

agent.sinks.AvroSink.type = avro

agent.sinks.AvroSink.channel = memoryChannel

agent.sinks.AvroSink.hostname = 0.0.0.0

agent.sinks.AvroSink.port = 1521

 ## Debugging Sink, Comment out AvroSink if you use this one

# http://flume.apache.org/FlumeUserGuide.html#file-roll-sink

agent.sinks = localout

agent.sinks.localout.type = hdfs

agent.sinks.localout.hdfs.path = hdfs://localhost:54310/flumerealtime

agent.sinks.localout.sink.rollInterval = 0

agent.sinks.localout.channel = memoryChannel

Now I have run the below command
*flume-ng agent -c /home/bruhati/Downlds/apache-flume-1.4.0-bin/conf -f
/home/bruhati/Downloads/apache-flume-1.4.0-bin/conf/realtime.conf -n agent*

Apart from that I Didnot do any thing ,So could you please explain the
necessary
 things to declare ,Which Include Where and how to declare agent at client
etc.,,
*I will be so Thankful,If I would receive reply ASAP!!!*
-- 
thanks&regards
Nagarjuna.S