You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by vrparekh <vr...@gmail.com> on 2013/06/13 17:24:48 UTC
Solr - Get DocID of search result
Hello All,
How can I get docID of result from solr?
What I am doing currently is,
I do search request in solr.
I get certain records (Say 10).
solrurl/start=0&rows=10
Now, again I do search request with below
solrurl/start=10&rows=10
So i get next 10 records.
Now new records are inserted in solr (Say 10 records).
and Now If I do request again by
solrurl/start=20&rows=10
So I might get repeated records.
So if I have docID of than I can query by less than that docID.
So is it possible to get docID?
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Get-DocID-of-search-result-tp4070253.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr - Get DocID of search result
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
I think the problem is the desire for the idempotent search results
across paging calls. Not sure if that explains it any better than the
original poster though. :-)
Basically, if the repeated search gets a different documents returned,
the offsets become somewhat problematic. Specifically, a page 2 may
skip some of the documents if there were some deletions that affected
page 1 or repeat documents if there were some additions.
I swore there was an article explaining this whole problem and
possible solution, but I can't find it now.
I did find the pageDoc reference that seems related, but I am not sure
if was actually implemented (JIRA is confusing on this one):
http://wiki.apache.org/solr/CommonQueryParameters#pageDoc_and_pageScore
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all
at once. Lately, it doesn't seem to be working. (Anonymous - via GTD
book)
On Thu, Jun 13, 2013 at 11:34 AM, Jack Krupansky
<ja...@basetechnology.com> wrote:
> You can use a Solr transformer to get the Lucene docID in the fl parameter:
>
> &fl=id,[docid],score,my-field,...
>
> But... you can't use the Lucene docId in a query.
>
> Relevancy and sorting, not to mention updating of existing documents, can
> change the order of results so that docId is not a good indicator of
> document "order".
>
> But, rather than focus prematurely on a solution, what exactly is the
> problem you are trying to solve? What exactly is duplicated?
>
> -- Jack Krupansky
>
> -----Original Message----- From: vrparekh
> Sent: Thursday, June 13, 2013 11:24 AM
> To: solr-user@lucene.apache.org
> Subject: Solr - Get DocID of search result
>
>
> Hello All,
>
> How can I get docID of result from solr?
>
> What I am doing currently is,
>
> I do search request in solr.
>
> I get certain records (Say 10).
>
> solrurl/start=0&rows=10
>
> Now, again I do search request with below
>
> solrurl/start=10&rows=10
>
> So i get next 10 records.
>
> Now new records are inserted in solr (Say 10 records).
> and Now If I do request again by
> solrurl/start=20&rows=10
>
> So I might get repeated records.
>
> So if I have docID of than I can query by less than that docID.
>
> So is it possible to get docID?
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Get-DocID-of-search-result-tp4070253.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr - Get DocID of search result
Posted by Chris Hostetter <ho...@fucit.org>.
Your phrasing of the question may be convoluting things -- you refered to
"DocID" but it's not clear if you mean...
* the low level internal lucene doc id
* the uniqueKey field of your schema.xml
* some identifier whose providence you don't care about.
In the first case, you can use the doctransformer jack mentioned, and you
can even sort on the internal lucene id using "_docid_" but you can't
really filter on it.
In the context of your specific problem the internal lucene doc id
wouldn't help you anyway, since internal lucene doc ids can change as
segments get merged and delets get flushed.
You can however use any of the later two cases, along with an fq to
implement" cursor" style logic to insure you never get the same document
more then once. instead of increasing the "start" param, you just
specify an fq param that filters on you id field using a range
query, and you constinually increase (or descrease) the boundary on the
range based on the last document fetched.
using your previous example...
: query return
: start=0&rows=1 A
: start=1&rows=1 B
: start=2&rows=1 C
you would instead do...
start=0&rows=1&sort=id+asc A
start=0&rows=1&sort=id+asc&fq=id:{A TO *] B
start=0&rows=1&sort=id+asc&fq=id:{B TO *] C
If you choose your id field such that the ids were always increasing (ie:
time based) then you could also be certain that you were always able to
fetch all documents (ie: you would never miss a doc because you were
already "past" it's place in the ordered list of docs)
-Hoss
Re: Solr - Get DocID of search result
Posted by vrparekh <vr...@gmail.com>.
Thanks Jack, below is the actual problem,
suppose currently 4 records are there in solr engine. A,B, C and D.
query return
start=0&rows=1 A
start=1&rows=1 B
start=2&rows=1 C
now at this time, 1 new record has been inserted in solr "AA". so documents
in solr are AA,A,B,C and D.
now if i query
start=3&rows=1, it will return "C" (2nd time so i have A, B, C, C),
while i am expecting "D".
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Get-DocID-of-search-result-tp4070253p4070271.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr - Get DocID of search result
Posted by Jack Krupansky <ja...@basetechnology.com>.
You can use a Solr transformer to get the Lucene docID in the fl parameter:
&fl=id,[docid],score,my-field,...
But... you can't use the Lucene docId in a query.
Relevancy and sorting, not to mention updating of existing documents, can
change the order of results so that docId is not a good indicator of
document "order".
But, rather than focus prematurely on a solution, what exactly is the
problem you are trying to solve? What exactly is duplicated?
-- Jack Krupansky
-----Original Message-----
From: vrparekh
Sent: Thursday, June 13, 2013 11:24 AM
To: solr-user@lucene.apache.org
Subject: Solr - Get DocID of search result
Hello All,
How can I get docID of result from solr?
What I am doing currently is,
I do search request in solr.
I get certain records (Say 10).
solrurl/start=0&rows=10
Now, again I do search request with below
solrurl/start=10&rows=10
So i get next 10 records.
Now new records are inserted in solr (Say 10 records).
and Now If I do request again by
solrurl/start=20&rows=10
So I might get repeated records.
So if I have docID of than I can query by less than that docID.
So is it possible to get docID?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Get-DocID-of-search-result-tp4070253.html
Sent from the Solr - User mailing list archive at Nabble.com.