You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by jfeist <jf...@llminc.com> on 2013/08/22 23:09:59 UTC
Storing query results
I am in the process of setting up a search application that allows the user
to view paginated query results. The documents are highly dynamic but I
want the search results to be static, i.e. I don't want the user to click
the next page button, the query reruns, and now he has a different set of
search results because the data changed while he was looking through it. I
want the results stored somewhere else and the successive page queries to
draw from that. I know Solr has query result caching, but I want to store
it entirely. Does Solr provide any functionality like this? I imagine it
doesn't, because then you'd need to specify how long to store it, etc. I'm
using Solr 4.4.0. I found someone asking something similar here
<http://lucene.472066.n3.nabble.com/storing-results-td476351.html> but
that was 6 years ago.
--
View this message in context: http://lucene.472066.n3.nabble.com/Storing-query-results-tp4086182.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing query results
Posted by Erick Erickson <er...@gmail.com>.
bq: Also, my boss told me it unequivocally has to be this way :p
Pesky bosses <G>.
But how often is the index changing? If you're not doing any updates
to it, then the problem is moot.... the other way to approach this problem
is to just control when the index changes. Would it suffice to only have the
data (possibly) change once every hour? Day? whatever?
FWIW,
Erick
On Fri, Aug 23, 2013 at 11:57 AM, jfeist <jf...@llminc.com> wrote:
> I completely agree. I would prefer to just rerun the search each time.
> However, we are going to be replacing our rdb based search with something
> like Solr, and the application currently behaves this way. Our users
> understand that the search is essentially a snapshot (and I would guess
> many
> prefer this over changing results) and we don't want to change existing
> behavior and confuse anyone. Also, my boss told me it unequivocally has to
> be this way :p
>
> Thanks for your input though, looks like I'm going to have to do something
> like you've suggested within our application.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Storing-query-results-tp4086182p4086349.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Storing query results
Posted by Dan Davis <da...@gmail.com>.
You could copy the existing core to a new core every once in awhile, and
then do your delta indexing into a new core once the copy is complete. If
a Persistent URL for the search results included the name of the original
core, the results you would get from a bookmark would be stable. However,
if you went to the site, and did a new site, you would be searching the
newest core.
This I think applies whether the site is "Intranet" or not.
Older cores could be aged out gracefully, and the search handler for an old
core could be replaced by a search on the new core via sharding.
On Fri, Aug 23, 2013 at 11:57 AM, jfeist <jf...@llminc.com> wrote:
> I completely agree. I would prefer to just rerun the search each time.
> However, we are going to be replacing our rdb based search with something
> like Solr, and the application currently behaves this way. Our users
> understand that the search is essentially a snapshot (and I would guess
> many
> prefer this over changing results) and we don't want to change existing
> behavior and confuse anyone. Also, my boss told me it unequivocally has to
> be this way :p
>
> Thanks for your input though, looks like I'm going to have to do something
> like you've suggested within our application.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Storing-query-results-tp4086182p4086349.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Storing query results
Posted by jfeist <jf...@llminc.com>.
I completely agree. I would prefer to just rerun the search each time.
However, we are going to be replacing our rdb based search with something
like Solr, and the application currently behaves this way. Our users
understand that the search is essentially a snapshot (and I would guess many
prefer this over changing results) and we don't want to change existing
behavior and confuse anyone. Also, my boss told me it unequivocally has to
be this way :p
Thanks for your input though, looks like I'm going to have to do something
like you've suggested within our application.
--
View this message in context: http://lucene.472066.n3.nabble.com/Storing-query-results-tp4086182p4086349.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Storing query results
Posted by Erick Erickson <er...@gmail.com>.
No, there's nothing like that in Solr. The closest you
could come would be to not do a hard commit (openSearcher=true)
or a soft commit for a very long time. As long as neither
of these things happen, the search results won't
change. But that's a hackish solution.
In fact I question your basic assumption. You say you don't
want the search results to change. But....
1> the user probably wouldn't notice
2> this can mislead in completely different ways. What if
most of the search results were deleted after the
first query? What if the _exact_ document she was
looking for got indexed after the first query?
This is one of those features that at first blush sounds
somewhat reasonable, but I don't think stands up under
inspection. It'd be some amount of work for, IMO, dubious
utility.
If you _must_ do something like this, the app layer could
do something like request a &rows=1000&fl=id
and essentially re-implement the queryResultCache at the
app layer. Subsequent pages would cause you to issue
queries like id=(1 or 54 or 90 ....).
Best
Erick
On Thu, Aug 22, 2013 at 6:00 PM, Ahmet Arslan <io...@yahoo.com> wrote:
> Hi jfeist,
>
> Your mail reminds me this blog, not sure about solr though.
>
>
> http://blog.mikemccandless.com/2011/11/searcherlifetimemanager-prevents-broken.html
>
>
>
> ________________________________
> From: jfeist <jf...@llminc.com>
> To: solr-user@lucene.apache.org
> Sent: Friday, August 23, 2013 12:09 AM
> Subject: Storing query results
>
>
> I am in the process of setting up a search application that allows the user
> to view paginated query results. The documents are highly dynamic but I
> want the search results to be static, i.e. I don't want the user to click
> the next page button, the query reruns, and now he has a different set of
> search results because the data changed while he was looking through it. I
> want the results stored somewhere else and the successive page queries to
> draw from that. I know Solr has query result caching, but I want to store
> it entirely. Does Solr provide any functionality like this? I imagine it
> doesn't, because then you'd need to specify how long to store it, etc. I'm
> using Solr 4.4.0. I found someone asking something similar here
> <http://lucene.472066.n3.nabble.com/storing-results-td476351.html> but
> that was 6 years ago.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Storing-query-results-tp4086182.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Storing query results
Posted by Ahmet Arslan <io...@yahoo.com>.
Hi jfeist,
Your mail reminds me this blog, not sure about solr though.
http://blog.mikemccandless.com/2011/11/searcherlifetimemanager-prevents-broken.html
________________________________
From: jfeist <jf...@llminc.com>
To: solr-user@lucene.apache.org
Sent: Friday, August 23, 2013 12:09 AM
Subject: Storing query results
I am in the process of setting up a search application that allows the user
to view paginated query results. The documents are highly dynamic but I
want the search results to be static, i.e. I don't want the user to click
the next page button, the query reruns, and now he has a different set of
search results because the data changed while he was looking through it. I
want the results stored somewhere else and the successive page queries to
draw from that. I know Solr has query result caching, but I want to store
it entirely. Does Solr provide any functionality like this? I imagine it
doesn't, because then you'd need to specify how long to store it, etc. I'm
using Solr 4.4.0. I found someone asking something similar here
<http://lucene.472066.n3.nabble.com/storing-results-td476351.html> but
that was 6 years ago.
--
View this message in context: http://lucene.472066.n3.nabble.com/Storing-query-results-tp4086182.html
Sent from the Solr - User mailing list archive at Nabble.com.