You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Becky Bonner <bb...@teleflora.com> on 2018/01/25 16:59:31 UTC
solr usage reporting
Hi all,
We are in the process of replacing our Google Search Appliance with SOLR 7.1 and are needing one last piece of our requirements. We provide a monthly report to our business that shows the top 1000 query terms requested during the date range as well as the query terms requested that contained no results. Is there a way to log the requests and later query solr for these results? Or is there a plugin to add this functionality?
Your help appreciated.
Bcubed
RE: solr usage reporting
Posted by Rick Leir <rl...@leirtech.com>.
Becky,
There are excellent log analysis systems. Logstash? Awstats? I do not think Solr should do this. Some people index their logs into a separate Solr core for analysis, but it might be a challenge to do this in a useful way.
Cheers -- Rick
On January 25, 2018 2:56:01 PM EST, Becky Bonner <bb...@teleflora.com> wrote:
>That would work for a single server but collecting the logs from the
>farm would be a problematic since we would have logs from all nodes and
>replicas from all the members of the farm. We would then need weed out
>what we are interested in and combine. It would be better if there were
>a way to query it within Solr. I think something in Solr would be best
>... a separate collection that can be queried and reports generated
>from it. The log does have the basic info we need though.
>
>
>-----Original Message-----
>From: Marco Reis [mailto:ma@marcoreis.net]
>Sent: Thursday, January 25, 2018 11:14 AM
>To: solr-user@lucene.apache.org
>Subject: Re: solr usage reporting
>
>One way is to collect the log from your server and, then, use other
>tool to generate your report.
>
>
>On Thu, Jan 25, 2018 at 2:59 PM Becky Bonner <bb...@teleflora.com>
>wrote:
>
>> Hi all,
>> We are in the process of replacing our Google Search Appliance with
>> SOLR
>> 7.1 and are needing one last piece of our requirements. We provide a
>
>> monthly report to our business that shows the top 1000 query terms
>> requested during the date range as well as the query terms requested
>> that contained no results. Is there a way to log the requests and
>> later query solr for these results? Or is there a plugin to add this
>functionality?
>>
>> Your help appreciated.
>> Bcubed
>>
>>
>> --
>Marco Reis
>Software Engineer
>http://marcoreis.net
>https://github.com/masreis
>+55 61 9 81194620
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
RE: solr usage reporting
Posted by Becky Bonner <bb...@teleflora.com>.
That would work for a single server but collecting the logs from the farm would be a problematic since we would have logs from all nodes and replicas from all the members of the farm. We would then need weed out what we are interested in and combine. It would be better if there were a way to query it within Solr. I think something in Solr would be best ... a separate collection that can be queried and reports generated from it. The log does have the basic info we need though.
-----Original Message-----
From: Marco Reis [mailto:ma@marcoreis.net]
Sent: Thursday, January 25, 2018 11:14 AM
To: solr-user@lucene.apache.org
Subject: Re: solr usage reporting
One way is to collect the log from your server and, then, use other tool to generate your report.
On Thu, Jan 25, 2018 at 2:59 PM Becky Bonner <bb...@teleflora.com> wrote:
> Hi all,
> We are in the process of replacing our Google Search Appliance with
> SOLR
> 7.1 and are needing one last piece of our requirements. We provide a
> monthly report to our business that shows the top 1000 query terms
> requested during the date range as well as the query terms requested
> that contained no results. Is there a way to log the requests and
> later query solr for these results? Or is there a plugin to add this functionality?
>
> Your help appreciated.
> Bcubed
>
>
> --
Marco Reis
Software Engineer
http://marcoreis.net
https://github.com/masreis
+55 61 9 81194620
Re: solr usage reporting
Posted by Marco Reis <ma...@marcoreis.net>.
One way is to collect the log from your server and, then, use other tool to
generate your report.
On Thu, Jan 25, 2018 at 2:59 PM Becky Bonner <bb...@teleflora.com> wrote:
> Hi all,
> We are in the process of replacing our Google Search Appliance with SOLR
> 7.1 and are needing one last piece of our requirements. We provide a
> monthly report to our business that shows the top 1000 query terms
> requested during the date range as well as the query terms requested that
> contained no results. Is there a way to log the requests and later query
> solr for these results? Or is there a plugin to add this functionality?
>
> Your help appreciated.
> Bcubed
>
>
> --
Marco Reis
Software Engineer
http://marcoreis.net
https://github.com/masreis
+55 61 9 81194620
Re: solr usage reporting
Posted by Shawn Heisey <ap...@elyograg.org>.
On 1/25/2018 9:59 AM, Becky Bonner wrote:
> We are in the process of replacing our Google Search Appliance with SOLR 7.1 and are needing one last piece of our requirements. We provide a monthly report to our business that shows the top 1000 query terms requested during the date range as well as the query terms requested that contained no results. Is there a way to log the requests and later query solr for these results? Or is there a plugin to add this functionality?
Solr currently does not track information like this.
When you leave settings at default, Solr does log every query -- the
logfile simply contains all of the parameters, both those included on
the URL itself and those added by the handler definition in
solrconfig.xml. It is possible to write a program that goes through
Solr's logs and compiles query statistics.
Solr doesn't log the IP address making the request. If you want that,
the container that is running Solr can log it separately. The jetty.xml
file included with Solr has had a commented out request log
configuration for a VERY long time.
Exactly which information from the query would you want Solr to track?
I suspect that the answer to that question would vary widely, which
means the feature not only has to be developed, but must also be
configurable. Adding configuration options makes the development take
longer and the code is more likely to have bugs.
Yes, query tracking would be a good feature. But unless you want to
write the code yourself, expect development to take a considerable
amount of time. There are a lot of already existing issues that are
more important in the eyes of Solr's development team.
Thanks,
Shawn