You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by chandra sekar <th...@hotmail.com> on 2015/11/19 10:07:26 UTC
IIS Web server Logs
Dear all,
I am new to Apache Flume. I have 5 Cluster Hadoop (YARN) node running on Linux machine. I would like to stream IIS webserver logs located at same subnet network to HDFS via Flume.
Do I need to run Flume-ng on the Windows Server machines which is running web server. Can you please let me know detailed guideline and configuration file.
I have archived Linux Realtime syslogs streaming to HDFS in the same cluster environment. But no idea in remote server environment.
Regards Shekar.
Re: IIS Web server Logs
Posted by Gonzalo Herreros <gh...@gmail.com>.
I see two options, either run a Flume agent on windows that spools the
local dir and either has access to hdfs or talks to other Flume agents
which do
Or you can have a small script scheduled on a regular basis to get logs and
post them to Flume.
Regards,
Gonzalo
On 19 November 2015 at 09:07, chandra sekar <th...@hotmail.com>
wrote:
> Dear all,
>
> I am new to Apache Flume. I have 5 Cluster Hadoop (YARN) node running on
> Linux machine. I would like to stream IIS webserver logs located at same
> subnet network to HDFS via Flume.
>
> Do I need to run Flume-ng on the Windows Server machines which is running
> web server. Can you please let me know detailed guideline and
> configuration file.
>
> I have archived Linux Realtime syslogs streaming to HDFS in the same
> cluster environment. But no idea in remote server environment.
>
> Regards
> Shekar.
>
>
>
>
>
Re: IIS Web server Logs
Posted by Ryan Suarez <ry...@sheridancollege.ca>.
nxlog is open source
On Nov 19, 2015, at 8:19 PM, chandra sekar <th...@hotmail.com>> wrote:
Dear Ryan,
Thanks for your info. Anyhow i am looking for open source technology only with Hadoop environment.
Regards
Sekar
________________________________
From: ryan.suarez@sheridancollege.ca<ma...@sheridancollege.ca>
To: user@flume.apache.org<ma...@flume.apache.org>
Subject: Re: IIS Web server Logs
Date: Thu, 19 Nov 2015 14:47:34 +0000
Use nxlog to forward the IIS logs to a remote flume-ng. Here’s a sample config:
https://www.loggly.com/docs/iis-web-server-logs/
On Nov 19, 2015, at 4:07 AM, chandra sekar <th...@hotmail.com>> wrote:
Dear all,
I am new to Apache Flume. I have 5 Cluster Hadoop (YARN) node running on Linux machine. I would like to stream IIS webserver logs located at same subnet network to HDFS via Flume.
Do I need to run Flume-ng on the Windows Server machines which is running web server. Can you please let me know detailed guideline and configuration file.
I have archived Linux Realtime syslogs streaming to HDFS in the same cluster environment. But no idea in remote server environment.
Regards
Shekar.
RE: IIS Web server Logs
Posted by chandra sekar <th...@hotmail.com>.
Dear Ryan,
Thanks for your info. Anyhow i am looking for open source technology only with Hadoop environment.
Regards Sekar
From: ryan.suarez@sheridancollege.ca
To: user@flume.apache.org
Subject: Re: IIS Web server Logs
Date: Thu, 19 Nov 2015 14:47:34 +0000
Use nxlog to forward the IIS logs to a remote flume-ng. Here’s a sample config:
https://www.loggly.com/docs/iis-web-server-logs/
On Nov 19, 2015, at 4:07 AM, chandra sekar <th...@hotmail.com> wrote:
Dear all,
I am new to Apache Flume. I have 5 Cluster Hadoop (YARN) node running on Linux machine. I would like to stream IIS webserver logs located at same subnet network to HDFS via Flume.
Do I need to run Flume-ng on the Windows Server machines which is running web server. Can you please let me know detailed guideline and configuration file.
I have archived Linux Realtime syslogs streaming to HDFS in the same cluster environment. But no idea in remote server environment.
Regards
Shekar.
Re: IIS Web server Logs
Posted by Ryan Suarez <ry...@sheridancollege.ca>.
Use nxlog to forward the IIS logs to a remote flume-ng. Here’s a sample config:
https://www.loggly.com/docs/iis-web-server-logs/
On Nov 19, 2015, at 4:07 AM, chandra sekar <th...@hotmail.com>> wrote:
Dear all,
I am new to Apache Flume. I have 5 Cluster Hadoop (YARN) node running on Linux machine. I would like to stream IIS webserver logs located at same subnet network to HDFS via Flume.
Do I need to run Flume-ng on the Windows Server machines which is running web server. Can you please let me know detailed guideline and configuration file.
I have archived Linux Realtime syslogs streaming to HDFS in the same cluster environment. But no idea in remote server environment.
Regards
Shekar.