You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Papp Richard <cc...@gmail.com> on 2010/10/30 18:22:03 UTC
solr 4.0 - pagination
Hi all,
I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I
can't solve the pagination.
Mainly there are two problems:
- the query fields "start" & "rows" doesn't work anymore -
beside of the values, it always returns the data as the start would be 0
(start = 0)
- the result contains just the total document number and not
the total groups number
Can anyone help me, how to solve this?
regards,
Rich
Re: solr 4.0 - pagination
Posted by Erick Erickson <er...@gmail.com>.
See: https://issues.apache.org/jira/browse/SOLR-2207
<https://issues.apache.org/jira/browse/SOLR-2207>You can track the progress
here, but it hasn't been committed to trunk yet.
Your options are:
1> get a recent trunk source tree and apply the patch and compile.
2> wait until it gets put in the trunk and get a nightly build.
3> volunteer to test it a lot <G>...
Best
Erick
On Sat, Oct 30, 2010 at 1:27 PM, Papp Richard <cc...@gmail.com> wrote:
> Can you estimate please when it will be done?
>
>
> thanks,
> Rich
>
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
> Seeley
> Sent: Saturday, October 30, 2010 19:29
> To: solr-user@lucene.apache.org
> Subject: Re: solr 4.0 - pagination
>
> On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard <cc...@gmail.com> wrote:
> > I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I
> > can't solve the pagination.
>
> It's not implemented yet, but I'm working on that right now.
>
> -Yonik
> http://www.lucidimagination.com
>
>
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature
> database 5576 (20101029) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature
> database 5576 (20101029) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
RE: solr 4.0 - pagination
Posted by Papp Richard <cc...@gmail.com>.
Can you estimate please when it will be done?
thanks,
Rich
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Saturday, October 30, 2010 19:29
To: solr-user@lucene.apache.org
Subject: Re: solr 4.0 - pagination
On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard <cc...@gmail.com> wrote:
> I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I
> can't solve the pagination.
It's not implemented yet, but I'm working on that right now.
-Yonik
http://www.lucidimagination.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5576 (20101029) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5576 (20101029) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
RE: solr 4.0 - pagination
Posted by Papp Richard <cc...@gmail.com>.
Hey Yonik,
Sorry, I think the matches is ok - because it probably returns always the
total document number - however I don't know how to compute the number of
pages.
thanks,
Rich
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Sunday, November 07, 2010 18:04
To: Papp Richard
Cc: solr-user@lucene.apache.org
Subject: Re: solr 4.0 - pagination
On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard <cc...@gmail.com> wrote:
> this is fantastic, but can you tell any time it will be ready ?
It already is ;-) Grab the latest trunk or the latest nightly build.
-Yonik
http://www.lucidimagination.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5598 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5599 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
RE: solr 4.0 - pagination
Posted by Papp Richard <cc...@gmail.com>.
thank you very much Yonik!
you are a magician!
regards,
Rich
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Sunday, November 07, 2010 18:04
To: Papp Richard
Cc: solr-user@lucene.apache.org
Subject: Re: solr 4.0 - pagination
On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard <cc...@gmail.com> wrote:
> this is fantastic, but can you tell any time it will be ready ?
It already is ;-) Grab the latest trunk or the latest nightly build.
-Yonik
http://www.lucidimagination.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5598 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5598 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
RE: solr 4.0 - pagination
Posted by "Grijesh.singh" <pi...@gmail.com>.
Then what will be when we filter out only some result and want to group ,how
your index time group count will help.
-----
Grijesh
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-4-0-pagination-tp1812384p2124747.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: solr 4.0 - pagination
Posted by phpcip <ph...@gmail.com>.
Well, right now, I'm using SOLR in a LOT of my projects.
I'm VERY fond of it, proud of it and VERY happy that such a team exists to
make it work.
Of course the pagination issue is a bit frustrating on the field
collapsing... But... heck... I'm currently de-normalizing my postgresql
database and... I'm just counting the total unique rows using SQL :D
And let SOLR do the rest of the job....
So... As a generic idea for the SOLR JAVA expert-people, I think the
counting of groups per each field should be something to be done at indexing
time rather than at query-time.
So basically when you index each document, you compute the grouping thingie
IF the user would so choose to have his field as eligible for grouping
inside his schema.xml... If you guys geather my meaning...
So I would have a field like this:
<field GROUPING="TRUE" name="unit_id" type="tint" indexed="true"
stored="true" required="true" />
Or something like that... and the indexer will know to compute the number of
groups that exist for this field, so that number would be available at
query-time without too much stress on the memory or CPU.
Hope this helps....
In the meanwhile... just count the total from SQL... And keep your index
in-sync often :D
Cip.
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-4-0-pagination-tp1812384p2123255.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: solr 4.0 - pagination
Posted by Papp Richard <cc...@gmail.com>.
I see. Let's assume that there are 1000 groups.
Can I use safely (with no negative impact on memory usage or slowness) the
start = 990, rows = 10 to get the latest page?
Or this will not work, due you will need to compute all the groups till
1000, in order to return the last 10, and because of this the whole will be
slow / memory usage will increase considerably.
regards,
Rich
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Sunday, November 07, 2010 21:54
To: Papp Richard
Cc: solr-user@lucene.apache.org
Subject: Re: solr 4.0 - pagination
On Sun, Nov 7, 2010 at 2:45 PM, Papp Richard <cc...@gmail.com> wrote:
> Hi Yonik,
>
> I've just tried the latest stable version from nightly build:
> apache-solr-4.0-2010-11-05_08-06-28.war
>
> I have some concerns however: I have 3 documents; 2 in the first group, 1
> in the 2nd group.
>
> 1. I got for matches 3 - which is good, but I still don't know how many
> groups I have. (using start = 0, rows = 10)
> 2. as far as I see the start / rows is working now, but the matches is
> returned incorrectly => it said matches = 3 instead of = 1, when I used
> start = 1, rows = 1
"matches" is the number of documents before grouping, so start/rows or
group.offset/group.limit will not affect this number.
> so can you help me, how to compute how many pages I'll have, because the
> matches can't use for this.
Solr doesn't even know given the current algorithm, hence it can't
return that info.
The issue is that to calculate the total number of groups, we would
need to keep each group in memory (which could cause a big blowup if
there are tons of groups). The current algorithm only keeps the top
10 groups (assuming rows=10) in memory at any one time, hence it has
no idea what the total number of groups is.
-Yonik
http://www.lucidimagination.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5599 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5599 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
Re: solr 4.0 - pagination
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Sun, Nov 7, 2010 at 2:45 PM, Papp Richard <cc...@gmail.com> wrote:
> Hi Yonik,
>
> I've just tried the latest stable version from nightly build:
> apache-solr-4.0-2010-11-05_08-06-28.war
>
> I have some concerns however: I have 3 documents; 2 in the first group, 1
> in the 2nd group.
>
> 1. I got for matches 3 - which is good, but I still don't know how many
> groups I have. (using start = 0, rows = 10)
> 2. as far as I see the start / rows is working now, but the matches is
> returned incorrectly => it said matches = 3 instead of = 1, when I used
> start = 1, rows = 1
"matches" is the number of documents before grouping, so start/rows or
group.offset/group.limit will not affect this number.
> so can you help me, how to compute how many pages I'll have, because the
> matches can't use for this.
Solr doesn't even know given the current algorithm, hence it can't
return that info.
The issue is that to calculate the total number of groups, we would
need to keep each group in memory (which could cause a big blowup if
there are tons of groups). The current algorithm only keeps the top
10 groups (assuming rows=10) in memory at any one time, hence it has
no idea what the total number of groups is.
-Yonik
http://www.lucidimagination.com
RE: solr 4.0 - pagination
Posted by Papp Richard <cc...@gmail.com>.
Hi Yonik,
I've just tried the latest stable version from nightly build:
apache-solr-4.0-2010-11-05_08-06-28.war
I have some concerns however: I have 3 documents; 2 in the first group, 1
in the 2nd group.
1. I got for matches 3 - which is good, but I still don't know how many
groups I have. (using start = 0, rows = 10)
2. as far as I see the start / rows is working now, but the matches is
returned incorrectly => it said matches = 3 instead of = 1, when I used
start = 1, rows = 1
so can you help me, how to compute how many pages I'll have, because the
matches can't use for this.
regards,
Rich
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Sunday, November 07, 2010 18:04
To: Papp Richard
Cc: solr-user@lucene.apache.org
Subject: Re: solr 4.0 - pagination
On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard <cc...@gmail.com> wrote:
> this is fantastic, but can you tell any time it will be ready ?
It already is ;-) Grab the latest trunk or the latest nightly build.
-Yonik
http://www.lucidimagination.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5598 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5599 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
Re: solr 4.0 - pagination
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard <cc...@gmail.com> wrote:
> this is fantastic, but can you tell any time it will be ready ?
It already is ;-) Grab the latest trunk or the latest nightly build.
-Yonik
http://www.lucidimagination.com
RE: solr 4.0 - pagination
Posted by Papp Richard <cc...@gmail.com>.
Dear Yonik,
this is fantastic, but can you tell any time it will be ready ?
I would need this feature in two weeks. Is it possible to finish and make
an update in this time or should I look for another solution cocerning the
pgaination (like implement just "more results" link instead of pagination) ?
best regards,
Rich
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Saturday, October 30, 2010 19:29
To: solr-user@lucene.apache.org
Subject: Re: solr 4.0 - pagination
On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard <cc...@gmail.com> wrote:
> I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I
> can't solve the pagination.
It's not implemented yet, but I'm working on that right now.
-Yonik
http://www.lucidimagination.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5576 (20101029) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5598 (20101107) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
Re: solr 4.0 - pagination
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard <cc...@gmail.com> wrote:
> I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I
> can't solve the pagination.
It's not implemented yet, but I'm working on that right now.
-Yonik
http://www.lucidimagination.com