stream logs from remote servers.

["Saurabh Agarwal (BLOOMBERG/ 731 LEX -)" <sa...@bloomberg.net>]

What is the best way to stream the log lines from a number of remote servers to the Kafka broker? Simple tailing does not work as these log files are rotated over the time. And if the tail process fail, the program need to start reading from the point of failure. It would be good if the tool has been used in real production. 

Thanks,
Saurabh. 

Re: stream logs from remote servers.

[Otis Gospodnetic <ot...@gmail.com>]

Hi,

Try:
https://github.com/joekiller/logstash-kafka
https://github.com/rngadam/sendkafka
https://issues.apache.org/jira/browse/FLUME-2242
https://github.com/baniuyao/flume-kafka

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/



On Tue, May 20, 2014 at 10:46 AM, Saurabh Agarwal (BLOOMBERG/ 731 LEX -) <
sagarwal144@bloomberg.net> wrote:

>
> What is the best way to stream the log lines from a number of remote
> servers to the Kafka broker? Simple tailing does not work as these log
> files are rotated over the time. And if the tail process fail, the program
> need to start reading from the point of failure. It would be good if the
> tool has been used in real production.
>
> Thanks,
> Saurabh.
>
>
>