You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by gudiseashok <gu...@gmail.com> on 2013/09/19 19:41:06 UTC
Lucene for Log file indexing and search
Hi
I am learning lucene, I am developing an application do do a search in log
files in multi-environment boxes, I have googled for the deeper
understanding, but all examples were just referring for just field "File
Name" & "Modification (i.e. fieldtypes associated with text search) and they
are returning results.
I would like to add more capabilities for text search , like providing more
inforamtion for requested search string. Could somebody help me , if anyone
added/explored more features (Field types/Query capabilities) for log file
search?
Your response is appreciated, thanks for taking time to read this.
Regards
Ashok Gudise
--
View this message in context: http://lucene.472066.n3.nabble.com/Lucene-for-Log-file-indexing-and-search-tp4091075.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene for Log file indexing and search
Posted by gudiseashok <gu...@gmail.com>.
Hi Paul,Otis & Ivan
I thank each one of you for taking time and pouring multiple options for my
requirement.
I saw the video in elastic search home page and I felt I got what I am
searching for, and I need to check Solr and this Logscene from Sema text.
I really liked lucene, and this is too powerful that it is taking very less
time for indexing in my case but since my requirement with log files I felt
like I am re inventing the wheel when I consider my requirement.
Thanks everyone again sir,
Regards
Ashok Gudise
--
View this message in context: http://lucene.472066.n3.nabble.com/Lucene-for-Log-file-indexing-and-search-tp4091075p4091359.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene for Log file indexing and search
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi,
Logstash is the piece that first touches your logs, filters them, and then outputs them somewhere.
People often use it with ElasticSearch. Once logs are in ES, they look at them with Kibana.
Note: somebody should write a Logstash output for Solr!
In Solr world there is Flume, which has a Solr sink.
Flume has file tailing capability and Cloudera's Morphlines should allow one to process the log much like Logstash filters let you process them.
At Sematext we've built something called Logsene - http://sematext.com/logsene/ , which uses some of the above technologies or plays nice with them.
Otis
----
Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm
>________________________________
> From: Ivan Krišto <iv...@gmail.com>
>To: java-user@lucene.apache.org
>Cc: gudiseashok <gu...@gmail.com>
>Sent: Friday, September 20, 2013 1:59 AM
>Subject: Re: Lucene for Log file indexing and search
>
>
>On 09/19/2013 07:41 PM, gudiseashok wrote:
>> I am learning lucene, I am developing an application do do a search in log
>> files in multi-environment boxes, I have googled for the deeper
>> understanding, but all examples were just referring for just field "File
>> Name" & "Modification (i.e. fieldtypes associated with text search) and they
>> are returning results.
>
>Hello!
>
>If you don't have some extremly specific needs checkout Logstash --
>http://logstash.net/ & http://www.elasticsearch.org/overview/logstash/
>It is powered by ElasticSearch (product similar to Solr, also based on
>Lucene).
>
>
> Regards,
> Ivan Krišto
>
>
>
Re: Lucene for Log file indexing and search
Posted by Ivan Krišto <iv...@gmail.com>.
On 09/19/2013 07:41 PM, gudiseashok wrote:
> I am learning lucene, I am developing an application do do a search in log
> files in multi-environment boxes, I have googled for the deeper
> understanding, but all examples were just referring for just field "File
> Name" & "Modification (i.e. fieldtypes associated with text search) and they
> are returning results.
Hello!
If you don't have some extremly specific needs checkout Logstash --
http://logstash.net/ & http://www.elasticsearch.org/overview/logstash/
It is powered by ElasticSearch (product similar to Solr, also based on
Lucene).
Regards,
Ivan Krišto
Re: Lucene for Log file indexing and search
Posted by Paul Libbrecht <pa...@hoplahup.net>.
Ashok,
I would look at solr which has an amount more field types to support more queries.
E.g. there you have a nice query syntax for times-spans and fantastic caching.
I think there's very few initiatives for indexing logs and I would be interested to see the results of your entreprise.
paul
Le 19 sept. 2013 à 19:41, gudiseashok a écrit :
>
> Hi
>
> I am learning lucene, I am developing an application do do a search in log
> files in multi-environment boxes, I have googled for the deeper
> understanding, but all examples were just referring for just field "File
> Name" & "Modification (i.e. fieldtypes associated with text search) and they
> are returning results.
>
> I would like to add more capabilities for text search , like providing more
> inforamtion for requested search string. Could somebody help me , if anyone
> added/explored more features (Field types/Query capabilities) for log file
> search?
>
> Your response is appreciated, thanks for taking time to read this.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org