You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Paul Libbrecht <pa...@hoplahup.net> on 2011/01/13 13:54:08 UTC
lucene-based log searcher?
Hello list,
has anyone built a log-analyzer based on Lucene?
Our logs are so big that grep takes more hours to do what I want it to do.
I'm sure Lucene would solve it.
Thanks in advance
paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: lucene-based log searcher?
Posted by Umesh Prasad <um...@gmail.com>.
Checkout http://code.google.com/p/lucene-log4j/
On Thu, Jan 13, 2011 at 7:09 PM, Steven A Rowe <sa...@syr.edu> wrote:
> Hi Paul,
>
> I saw this yesterday, but haven't tried it myself:
>
> http://karussell.wordpress.com/2010/10/27/feeding-solr-with-its-own-logs/
>
> The author has a project called "Sogger" - Solr + Logger? - that can read
> various forms of logs.
>
> Steve
>
> > -----Original Message-----
> > From: Paul Libbrecht [mailto:paul@hoplahup.net]
> > Sent: Thursday, January 13, 2011 7:54 AM
> > To: java-user@lucene.apache.org
> > Subject: lucene-based log searcher?
> >
> >
> > Hello list,
> >
> > has anyone built a log-analyzer based on Lucene?
> > Our logs are so big that grep takes more hours to do what I want it to
> do.
> > I'm sure Lucene would solve it.
> >
> > Thanks in advance
> > paul
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
---
Thanks & Regards
Umesh Prasad
RE: lucene-based log searcher?
Posted by Steven A Rowe <sa...@syr.edu>.
Hi Paul,
I saw this yesterday, but haven't tried it myself:
http://karussell.wordpress.com/2010/10/27/feeding-solr-with-its-own-logs/
The author has a project called "Sogger" - Solr + Logger? - that can read various forms of logs.
Steve
> -----Original Message-----
> From: Paul Libbrecht [mailto:paul@hoplahup.net]
> Sent: Thursday, January 13, 2011 7:54 AM
> To: java-user@lucene.apache.org
> Subject: lucene-based log searcher?
>
>
> Hello list,
>
> has anyone built a log-analyzer based on Lucene?
> Our logs are so big that grep takes more hours to do what I want it to do.
> I'm sure Lucene would solve it.
>
> Thanks in advance
> paul
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
Re: lucene-based log searcher?
Posted by Benzion G <be...@yahoo.com>.
Hi,
I almost finished these days to write such a tool.
It reads via FTPClient the log files and adds to Lucene index.
Via small Web application you can see/filter/sort the log messages.
--
View this message in context: http://lucene.472066.n3.nabble.com/lucene-based-log-searcher-tp2247969p2249215.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: lucene-based log searcher?
Posted by Lukáš Vlček <lu...@gmail.com>.
Hi,
I haven't tried that myself so far but may be you can check logstash as well
(http://www.loggly.com/2010/11/do-you-have-a-logstash/).
Regards,
Lukas
On Fri, Jan 14, 2011 at 7:36 AM, Benzion G <be...@yahoo.com> wrote:
>
> If you can define one searchable word for the whole log message and index
> this word only maybe you can use DB instead.
> I apply Analyzer to every and whole log message text - and then I can
> quickly find all messages, that have e.g. "NullPointerException" or "user
> not found" text or anything else.
>
> In DB using select with "LIKE '%something%'" will do full table scan.
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/lucene-based-log-searcher-tp2247969p2253738.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: lucene-based log searcher?
Posted by Benzion G <be...@yahoo.com>.
If you can define one searchable word for the whole log message and index
this word only maybe you can use DB instead.
I apply Analyzer to every and whole log message text - and then I can
quickly find all messages, that have e.g. "NullPointerException" or "user
not found" text or anything else.
In DB using select with "LIKE '%something%'" will do full table scan.
--
View this message in context: http://lucene.472066.n3.nabble.com/lucene-based-log-searcher-tp2247969p2253738.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org