Pagination with streaming expressions

[Pratik Patel <pr...@semandex.net>]

Hello Everyone,

Is there a way to paginate the results of Streaming Expression?

Let's say I have a simple gatherNodes function which has count operation at
the end of it. I can sort by the count fine but now I would like to be able
to select specific sub set of result based on pagination parameters. Is
there any way to do that?

Thanks!
Pratik

Re: Pagination with streaming expressions

[Joel Bernstein <jo...@gmail.com>]

There is an open ticket which deals with this:

https://issues.apache.org/jira/browse/SOLR-12209

I've been very focused though on anything that enhances the Solr Math
Expressions or has been needed for the Fusion SQL engine, which is what I
work on at LucidWorks. SOLR-12209 doesn't fall into that category.
Eventually though I will clear that ticket if someone else doesn't resolve
it first.




Joel Bernstein
http://joelsolr.blogspot.com/


On Wed, May 1, 2019 at 7:56 PM Erick Erickson <er...@gmail.com>
wrote:

> This sounds like an XY problem. You’re asking now to paginate, but not
> explaining the problem you want to solve with paginating.
>
> I don’t immediately see what purpose paginating serves here. What
> significance does a page have to do with the gatherNodes? How use would the
> _user_ have with these results? Especially for two unrelated queries. IOW
> if for query1 you count something for page 13, and for query2 you also
> count something for page 13 what information is the user getting in those
> two cases? Especially if the total result set for query1 is 1,000 docs but
> for query2 is 10,000,000 does?
>
> But in general no, streaming is orthogonal to most use-cases for
> pagination and isn’t really supported except if you read through the
> returns and throw away the first N pages, probably pretty inefficient.
>
> Erick
>
> > On May 1, 2019, at 1:28 PM, Pratik Patel <pr...@semandex.net> wrote:
> >
> > Hello Everyone,
> >
> > Is there a way to paginate the results of Streaming Expression?
> >
> > Let's say I have a simple gatherNodes function which has count operation
> at
> > the end of it. I can sort by the count fine but now I would like to be
> able
> > to select specific sub set of result based on pagination parameters. Is
> > there any way to do that?
> >
> > Thanks!
> > Pratik
>
>

Re: Pagination with streaming expressions

[Erick Erickson <er...@gmail.com>]

This sounds like an XY problem. You’re asking now to paginate, but not explaining the problem you want to solve with paginating.

I don’t immediately see what purpose paginating serves here. What significance does a page have to do with the gatherNodes? How use would the _user_ have with these results? Especially for two unrelated queries. IOW if for query1 you count something for page 13, and for query2 you also count something for page 13 what information is the user getting in those two cases? Especially if the total result set for query1 is 1,000 docs but for query2 is 10,000,000 does?

But in general no, streaming is orthogonal to most use-cases for pagination and isn’t really supported except if you read through the returns and throw away the first N pages, probably pretty inefficient.

Erick

> On May 1, 2019, at 1:28 PM, Pratik Patel <pr...@semandex.net> wrote:
> 
> Hello Everyone,
> 
> Is there a way to paginate the results of Streaming Expression?
> 
> Let's say I have a simple gatherNodes function which has count operation at
> the end of it. I can sort by the count fine but now I would like to be able
> to select specific sub set of result based on pagination parameters. Is
> there any way to do that?
> 
> Thanks!
> Pratik