You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by David Novogrodsky <da...@gmail.com> on 2014/11/25 00:25:42 UTC
Newbie question on using the Syslog agent in Flume
All,
I am new to the Hadoop Ecosystem. I have a question about Syslog and the
Flume agent for Syslog.
I am working to ingest network data from an agent. The agent is sending
data in Syslog format, or is creating data in syslog format. Here are the
required parameters for a syslog Flume source:
a1.sources = r1a1.channels = c1a1.sources.r1.type =
syslogudpa1.sources.r1.port = 5140a1.sources.r1.host = localhost
a1.sources.r1.channels = c1
I asked the developer for the IP address of the syslog source. I assume
that the Flume agent sends an agent program to the syslog server, defined
by a1.source.r1.host. Is this correct? Or is the a1.sources.c1.host IP
address, the address of the machine that is running the Flume instance?
David Novogrodsky
david.novogrodsky@gmail.com
http://www.linkedin.com/in/davidnovogrodsky
Re: Newbie question on using the Syslog agent in Flume
Posted by David Novogrodsky <da...@gmail.com>.
Thank you for the information.
David Novogrodsky
david.novogrodsky@gmail.com
http://www.linkedin.com/in/davidnovogrodsky
On Mon, Nov 24, 2014 at 5:49 PM, Joey Echeverria <jo...@cloudera.com> wrote:
> The host in your source config should be the hostname that will be
> listening for syslog events. It's typically either 0.0.0.0 to listen
> on the wildcard interface or the fully qualified hostname/IP address
> of the public network interface on the Flume agent.
>
> You then configure the host that will send syslog events to Flume with
> the Flume agent's hostname and the port you've configured. Here's an
> article that shows how to configure rsyslog and syslog-ng to send to a
> server:
>
> http://help.papertrailapp.com/kb/configuration/configuring-remote-syslog-from-unixlinux-and-bsdos-x/
>
> -Joey
>
> On Mon, Nov 24, 2014 at 3:25 PM, David Novogrodsky
> <da...@gmail.com> wrote:
>> All,
>>
>> I am new to the Hadoop Ecosystem. I have a question about Syslog and the
>> Flume agent for Syslog.
>>
>> I am working to ingest network data from an agent. The agent is sending
>> data in Syslog format, or is creating data in syslog format. Here are the
>> required parameters for a syslog Flume source:
>>
>> a1.sources = r1
>> a1.channels = c1
>> a1.sources.r1.type = syslogudp
>> a1.sources.r1.port = 5140
>> a1.sources.r1.host = localhost
>>
>> a1.sources.r1.channels = c1
>>
>>
>>
>> I asked the developer for the IP address of the syslog source. I assume
>> that the Flume agent sends an agent program to the syslog server, defined by
>> a1.source.r1.host. Is this correct? Or is the a1.sources.c1.host IP
>> address, the address of the machine that is running the Flume instance?
>>
>> David Novogrodsky
>> david.novogrodsky@gmail.com
>> http://www.linkedin.com/in/davidnovogrodsky
>
>
>
> --
> Joey Echeverria
Re: Newbie question on using the Syslog agent in Flume
Posted by Joey Echeverria <jo...@cloudera.com>.
The host in your source config should be the hostname that will be
listening for syslog events. It's typically either 0.0.0.0 to listen
on the wildcard interface or the fully qualified hostname/IP address
of the public network interface on the Flume agent.
You then configure the host that will send syslog events to Flume with
the Flume agent's hostname and the port you've configured. Here's an
article that shows how to configure rsyslog and syslog-ng to send to a
server:
http://help.papertrailapp.com/kb/configuration/configuring-remote-syslog-from-unixlinux-and-bsdos-x/
-Joey
On Mon, Nov 24, 2014 at 3:25 PM, David Novogrodsky
<da...@gmail.com> wrote:
> All,
>
> I am new to the Hadoop Ecosystem. I have a question about Syslog and the
> Flume agent for Syslog.
>
> I am working to ingest network data from an agent. The agent is sending
> data in Syslog format, or is creating data in syslog format. Here are the
> required parameters for a syslog Flume source:
>
> a1.sources = r1
> a1.channels = c1
> a1.sources.r1.type = syslogudp
> a1.sources.r1.port = 5140
> a1.sources.r1.host = localhost
>
> a1.sources.r1.channels = c1
>
>
>
> I asked the developer for the IP address of the syslog source. I assume
> that the Flume agent sends an agent program to the syslog server, defined by
> a1.source.r1.host. Is this correct? Or is the a1.sources.c1.host IP
> address, the address of the machine that is running the Flume instance?
>
> David Novogrodsky
> david.novogrodsky@gmail.com
> http://www.linkedin.com/in/davidnovogrodsky
--
Joey Echeverria
Re: Newbie question on using the Syslog agent in Flume
Posted by Arvind Prabhakar <ar...@apache.org>.
Hi David,
The configuration you shared sets up a Flume Agent to listen in for Syslog
messages using UDP transport on port 5140. Hence Flume itself acts as the
receiver of the messages and not the sender.
Regards,
Arvind Prabhakar
On Mon, Nov 24, 2014 at 3:25 PM, David Novogrodsky <
david.novogrodsky@gmail.com> wrote:
> All,
>
> I am new to the Hadoop Ecosystem. I have a question about Syslog and the
> Flume agent for Syslog.
>
> I am working to ingest network data from an agent. The agent is sending
> data in Syslog format, or is creating data in syslog format. Here are the
> required parameters for a syslog Flume source:
>
> a1.sources = r1a1.channels = c1a1.sources.r1.type = syslogudpa1.sources.r1.port = 5140a1.sources.r1.host = localhost
>
> a1.sources.r1.channels = c1
>
>
>
>
> I asked the developer for the IP address of the syslog source. I assume
> that the Flume agent sends an agent program to the syslog server, defined
> by a1.source.r1.host. Is this correct? Or is the a1.sources.c1.host IP
> address, the address of the machine that is running the Flume instance?
>
> David Novogrodsky
> david.novogrodsky@gmail.com
> http://www.linkedin.com/in/davidnovogrodsky
>