You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Nicola Buso <nb...@ebi.ac.uk> on 2013/07/02 17:36:18 UTC

Facets ordering

Hi,

I was looking to change the order of the facet results; in this case I
would like to order by the facet label instead of the facet value
(count).

An example is a facet on dates; suppose the facet is saved as
YYYY/MM/dd, I would like obtain values for this date ordered by the
date; i.e. with depth 1:

2013 (2,403,222)
2012 (3,632.098)
2011 (1,213,990)
....

is this possible?


Nicola


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Facets ordering

Posted by Shai Erera <se...@gmail.com>.
RangeFacetRequest will be released in 4.4, I guess a couple of weeks away.

Shai
On Jul 4, 2013 12:02 PM, "Nicola Buso" <nb...@ebi.ac.uk> wrote:

> On Wed, 2013-07-03 at 21:58 +0300, Shai Erera wrote:
> > What's maxCount? What I mean is that if you create a FacetRequest with
> > numResults = 5*K (for example), then you get the top-5K categories and
> can
> > choose the best top-K of those, by their label. Yes, this will hurt top-K
> > computation the least, but is not guaranteed to return the correct top-K.
> maxCount = numResults
>
> >
> > The other alternative, which you should test, is to create a
> > FacetResultsHandler which labels every ordinal and compares by their
> label.
> > While it will be slower, perhaps it's acceptable for your app.
> I will try it!
>
> >
> > Regarding the ranges, I assume you're not talking about numeric ranges
> > (cause we have a RangeFacetRequest for that), but something else?
> > E.g. maybe show Year/1981-1990 and Year/1991-2000? Is that the case?
> when lucene 5.x will be released or these api ported to 4.x?
>
> >
> > If so, how would you decide which buckets to create? And can the buckets
> > pre-created at indexing already? E.g.
> > Year/1981-1990/1982/Jan-Mar/Feb/1-10/8?
> > This can definitely turn into an interesting usecase and feature to have
> in
> > Lucene, but I'd need to understand better what sort of ranges do u have
> in
> > mind.
> > Maybe we can discuss that on a separate thread?
> >
> > Shai
>
> Nicola
> >
> >
> > On Wed, Jul 3, 2013 at 7:34 PM, Nicola Buso <nb...@ebi.ac.uk> wrote:
> >
> > > Hi Shai,
> > >
> > > if I'm correctly understanding, you are suggesting to use maxCount in a
> > > new FacetResultHandler to subset the whole facet hierarchy so that the
> > > counting still has good performances; this is not guaranteeing it will
> > > obtain maxcCount values but some approximations to it.
> > >
> > > The second solution, get all children and than sort by label, is not
> > > feasible because the facet values are spread into a big range.
> > >
> > > Another thing that should be nice to introduce is grouping the result
> > > facet values; like for date, but for many other things that are
> > > measurable, the values should be spread into big ranges and giving a
> > > representation to the user is difficult and probably not useful. If you
> > > can group values you should give the user a good approximation.
> > > Do you think is feasible to create a new facet hierarchy at runtime
> > > composed by groups of values and fill the values in the original
> > > hierarchy in the new created one? Too expensive?
> > >
> > >
> > > Nicola.
> > >
> > >
> > >
> > > On Tue, 2013-07-02 at 20:49 +0300, Shai Erera wrote:
> > > > Well, in general it can be done, but it won't be cheap. You can
> > > > implement a FacetResultsHandler which instead of sorting by value
> will
> > > > sort by the category label. But that means you're going to label
> > > > *every single category* in order to sort by it.
> > > >
> > > >
> > > > Maybe if you can do away with approximations, you can ask for top-50
> > > > and return the "top by label" from that list.
> > > > Or, if the number of children is bounded, maybe ask to return all of
> > > > them, and then just sort by label. I think that sorting is cheaper
> > > > than updating a heap.
> > > >
> > > > Shai
> > > >
> > > >
> > > >
> > > > On Tue, Jul 2, 2013 at 6:54 PM, Nicola Buso <nb...@ebi.ac.uk> wrote:
> > > >         Hi,
> > > >
> > > >         I was thinking about it, what is needed is the 1st, than,
> > > >         supposing
> > > >         FacetRequest maxCount is setted to 10 I want the latest 10
> > > >         years with
> > > >         respective counts, also if on year 2000 there are more
> counts
> > > >         than in
> > > >         2013.
> > > >
> > > >
> > > >         Nicola.
> > > >
> > > >         On Tue, 2013-07-02 at 18:40 +0300, Shai Erera wrote:
> > > >         > Do you want your top-K to be computed by label too? Or
> first
> > > >         deduce
> > > >         > the top-K facets, then sort them otherwise?
> > > >         >
> > > >         > Shai
> > > >         >
> > > >         >
> > > >         >
> > > >         > On Tue, Jul 2, 2013 at 6:36 PM, Nicola Buso
> > > >         <nb...@ebi.ac.uk> wrote:
> > > >         >         Hi,
> > > >         >
> > > >         >         I was looking to change the order of the facet
> > > >         results; in
> > > >         >         this case I
> > > >         >         would like to order by the facet label instead of
> > > >         the facet
> > > >         >         value
> > > >         >         (count).
> > > >         >
> > > >         >         An example is a facet on dates; suppose the facet
> is
> > > >         saved as
> > > >         >         YYYY/MM/dd, I would like obtain values for this
> date
> > > >         ordered
> > > >         >         by the
> > > >         >         date; i.e. with depth 1:
> > > >         >
> > > >         >         2013 (2,403,222)
> > > >         >         2012 (3,632.098)
> > > >         >         2011 (1,213,990)
> > > >         >         ....
> > > >         >
> > > >         >         is this possible?
> > > >         >
> > > >         >
> > > >         >         Nicola
> > > >         >
> > > >         >
> > > >         >
> > > >
> > > ---------------------------------------------------------------------
> > > >         >         To unsubscribe, e-mail:
> > > >         >         java-user-unsubscribe@lucene.apache.org
> > > >         >         For additional commands, e-mail:
> > > >         >         java-user-help@lucene.apache.org
> > > >         >
> > > >         >
> > > >         >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Re: Facets ordering

Posted by Nicola Buso <nb...@ebi.ac.uk>.
On Wed, 2013-07-03 at 21:58 +0300, Shai Erera wrote:
> What's maxCount? What I mean is that if you create a FacetRequest with
> numResults = 5*K (for example), then you get the top-5K categories and can
> choose the best top-K of those, by their label. Yes, this will hurt top-K
> computation the least, but is not guaranteed to return the correct top-K.
maxCount = numResults

> 
> The other alternative, which you should test, is to create a
> FacetResultsHandler which labels every ordinal and compares by their label.
> While it will be slower, perhaps it's acceptable for your app.
I will try it!

> 
> Regarding the ranges, I assume you're not talking about numeric ranges
> (cause we have a RangeFacetRequest for that), but something else?
> E.g. maybe show Year/1981-1990 and Year/1991-2000? Is that the case?
when lucene 5.x will be released or these api ported to 4.x?

> 
> If so, how would you decide which buckets to create? And can the buckets
> pre-created at indexing already? E.g.
> Year/1981-1990/1982/Jan-Mar/Feb/1-10/8?
> This can definitely turn into an interesting usecase and feature to have in
> Lucene, but I'd need to understand better what sort of ranges do u have in
> mind.
> Maybe we can discuss that on a separate thread?
> 
> Shai

Nicola
> 
> 
> On Wed, Jul 3, 2013 at 7:34 PM, Nicola Buso <nb...@ebi.ac.uk> wrote:
> 
> > Hi Shai,
> >
> > if I'm correctly understanding, you are suggesting to use maxCount in a
> > new FacetResultHandler to subset the whole facet hierarchy so that the
> > counting still has good performances; this is not guaranteeing it will
> > obtain maxcCount values but some approximations to it.
> >
> > The second solution, get all children and than sort by label, is not
> > feasible because the facet values are spread into a big range.
> >
> > Another thing that should be nice to introduce is grouping the result
> > facet values; like for date, but for many other things that are
> > measurable, the values should be spread into big ranges and giving a
> > representation to the user is difficult and probably not useful. If you
> > can group values you should give the user a good approximation.
> > Do you think is feasible to create a new facet hierarchy at runtime
> > composed by groups of values and fill the values in the original
> > hierarchy in the new created one? Too expensive?
> >
> >
> > Nicola.
> >
> >
> >
> > On Tue, 2013-07-02 at 20:49 +0300, Shai Erera wrote:
> > > Well, in general it can be done, but it won't be cheap. You can
> > > implement a FacetResultsHandler which instead of sorting by value will
> > > sort by the category label. But that means you're going to label
> > > *every single category* in order to sort by it.
> > >
> > >
> > > Maybe if you can do away with approximations, you can ask for top-50
> > > and return the "top by label" from that list.
> > > Or, if the number of children is bounded, maybe ask to return all of
> > > them, and then just sort by label. I think that sorting is cheaper
> > > than updating a heap.
> > >
> > > Shai
> > >
> > >
> > >
> > > On Tue, Jul 2, 2013 at 6:54 PM, Nicola Buso <nb...@ebi.ac.uk> wrote:
> > >         Hi,
> > >
> > >         I was thinking about it, what is needed is the 1st, than,
> > >         supposing
> > >         FacetRequest maxCount is setted to 10 I want the latest 10
> > >         years with
> > >         respective counts, also if on year 2000 there are more counts
> > >         than in
> > >         2013.
> > >
> > >
> > >         Nicola.
> > >
> > >         On Tue, 2013-07-02 at 18:40 +0300, Shai Erera wrote:
> > >         > Do you want your top-K to be computed by label too? Or first
> > >         deduce
> > >         > the top-K facets, then sort them otherwise?
> > >         >
> > >         > Shai
> > >         >
> > >         >
> > >         >
> > >         > On Tue, Jul 2, 2013 at 6:36 PM, Nicola Buso
> > >         <nb...@ebi.ac.uk> wrote:
> > >         >         Hi,
> > >         >
> > >         >         I was looking to change the order of the facet
> > >         results; in
> > >         >         this case I
> > >         >         would like to order by the facet label instead of
> > >         the facet
> > >         >         value
> > >         >         (count).
> > >         >
> > >         >         An example is a facet on dates; suppose the facet is
> > >         saved as
> > >         >         YYYY/MM/dd, I would like obtain values for this date
> > >         ordered
> > >         >         by the
> > >         >         date; i.e. with depth 1:
> > >         >
> > >         >         2013 (2,403,222)
> > >         >         2012 (3,632.098)
> > >         >         2011 (1,213,990)
> > >         >         ....
> > >         >
> > >         >         is this possible?
> > >         >
> > >         >
> > >         >         Nicola
> > >         >
> > >         >
> > >         >
> > >
> > ---------------------------------------------------------------------
> > >         >         To unsubscribe, e-mail:
> > >         >         java-user-unsubscribe@lucene.apache.org
> > >         >         For additional commands, e-mail:
> > >         >         java-user-help@lucene.apache.org
> > >         >
> > >         >
> > >         >
> > >
> > >
> > >
> > >
> >
> >
> >



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Facets ordering

Posted by Shai Erera <se...@gmail.com>.
What's maxCount? What I mean is that if you create a FacetRequest with
numResults = 5*K (for example), then you get the top-5K categories and can
choose the best top-K of those, by their label. Yes, this will hurt top-K
computation the least, but is not guaranteed to return the correct top-K.

The other alternative, which you should test, is to create a
FacetResultsHandler which labels every ordinal and compares by their label.
While it will be slower, perhaps it's acceptable for your app.

Regarding the ranges, I assume you're not talking about numeric ranges
(cause we have a RangeFacetRequest for that), but something else?
E.g. maybe show Year/1981-1990 and Year/1991-2000? Is that the case?

If so, how would you decide which buckets to create? And can the buckets
pre-created at indexing already? E.g.
Year/1981-1990/1982/Jan-Mar/Feb/1-10/8?
This can definitely turn into an interesting usecase and feature to have in
Lucene, but I'd need to understand better what sort of ranges do u have in
mind.
Maybe we can discuss that on a separate thread?

Shai


On Wed, Jul 3, 2013 at 7:34 PM, Nicola Buso <nb...@ebi.ac.uk> wrote:

> Hi Shai,
>
> if I'm correctly understanding, you are suggesting to use maxCount in a
> new FacetResultHandler to subset the whole facet hierarchy so that the
> counting still has good performances; this is not guaranteeing it will
> obtain maxcCount values but some approximations to it.
>
> The second solution, get all children and than sort by label, is not
> feasible because the facet values are spread into a big range.
>
> Another thing that should be nice to introduce is grouping the result
> facet values; like for date, but for many other things that are
> measurable, the values should be spread into big ranges and giving a
> representation to the user is difficult and probably not useful. If you
> can group values you should give the user a good approximation.
> Do you think is feasible to create a new facet hierarchy at runtime
> composed by groups of values and fill the values in the original
> hierarchy in the new created one? Too expensive?
>
>
> Nicola.
>
>
>
> On Tue, 2013-07-02 at 20:49 +0300, Shai Erera wrote:
> > Well, in general it can be done, but it won't be cheap. You can
> > implement a FacetResultsHandler which instead of sorting by value will
> > sort by the category label. But that means you're going to label
> > *every single category* in order to sort by it.
> >
> >
> > Maybe if you can do away with approximations, you can ask for top-50
> > and return the "top by label" from that list.
> > Or, if the number of children is bounded, maybe ask to return all of
> > them, and then just sort by label. I think that sorting is cheaper
> > than updating a heap.
> >
> > Shai
> >
> >
> >
> > On Tue, Jul 2, 2013 at 6:54 PM, Nicola Buso <nb...@ebi.ac.uk> wrote:
> >         Hi,
> >
> >         I was thinking about it, what is needed is the 1st, than,
> >         supposing
> >         FacetRequest maxCount is setted to 10 I want the latest 10
> >         years with
> >         respective counts, also if on year 2000 there are more counts
> >         than in
> >         2013.
> >
> >
> >         Nicola.
> >
> >         On Tue, 2013-07-02 at 18:40 +0300, Shai Erera wrote:
> >         > Do you want your top-K to be computed by label too? Or first
> >         deduce
> >         > the top-K facets, then sort them otherwise?
> >         >
> >         > Shai
> >         >
> >         >
> >         >
> >         > On Tue, Jul 2, 2013 at 6:36 PM, Nicola Buso
> >         <nb...@ebi.ac.uk> wrote:
> >         >         Hi,
> >         >
> >         >         I was looking to change the order of the facet
> >         results; in
> >         >         this case I
> >         >         would like to order by the facet label instead of
> >         the facet
> >         >         value
> >         >         (count).
> >         >
> >         >         An example is a facet on dates; suppose the facet is
> >         saved as
> >         >         YYYY/MM/dd, I would like obtain values for this date
> >         ordered
> >         >         by the
> >         >         date; i.e. with depth 1:
> >         >
> >         >         2013 (2,403,222)
> >         >         2012 (3,632.098)
> >         >         2011 (1,213,990)
> >         >         ....
> >         >
> >         >         is this possible?
> >         >
> >         >
> >         >         Nicola
> >         >
> >         >
> >         >
> >
> ---------------------------------------------------------------------
> >         >         To unsubscribe, e-mail:
> >         >         java-user-unsubscribe@lucene.apache.org
> >         >         For additional commands, e-mail:
> >         >         java-user-help@lucene.apache.org
> >         >
> >         >
> >         >
> >
> >
> >
> >
>
>
>

Re: Facets ordering

Posted by Nicola Buso <nb...@ebi.ac.uk>.
Hi,

I was thinking about it, what is needed is the 1st, than, supposing
FacetRequest maxCount is setted to 10 I want the latest 10 years with
respective counts, also if on year 2000 there are more counts than in
2013.


Nicola.

On Tue, 2013-07-02 at 18:40 +0300, Shai Erera wrote:
> Do you want your top-K to be computed by label too? Or first deduce
> the top-K facets, then sort them otherwise?
> 
> Shai
> 
> 
> 
> On Tue, Jul 2, 2013 at 6:36 PM, Nicola Buso <nb...@ebi.ac.uk> wrote:
>         Hi,
>         
>         I was looking to change the order of the facet results; in
>         this case I
>         would like to order by the facet label instead of the facet
>         value
>         (count).
>         
>         An example is a facet on dates; suppose the facet is saved as
>         YYYY/MM/dd, I would like obtain values for this date ordered
>         by the
>         date; i.e. with depth 1:
>         
>         2013 (2,403,222)
>         2012 (3,632.098)
>         2011 (1,213,990)
>         ....
>         
>         is this possible?
>         
>         
>         Nicola
>         
>         
>         ---------------------------------------------------------------------
>         To unsubscribe, e-mail:
>         java-user-unsubscribe@lucene.apache.org
>         For additional commands, e-mail:
>         java-user-help@lucene.apache.org
>         
> 
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Facets ordering

Posted by Shai Erera <se...@gmail.com>.
Do you want your top-K to be computed by label too? Or first deduce the
top-K facets, then sort them otherwise?

Shai


On Tue, Jul 2, 2013 at 6:36 PM, Nicola Buso <nb...@ebi.ac.uk> wrote:

> Hi,
>
> I was looking to change the order of the facet results; in this case I
> would like to order by the facet label instead of the facet value
> (count).
>
> An example is a facet on dates; suppose the facet is saved as
> YYYY/MM/dd, I would like obtain values for this date ordered by the
> date; i.e. with depth 1:
>
> 2013 (2,403,222)
> 2012 (3,632.098)
> 2011 (1,213,990)
> ....
>
> is this possible?
>
>
> Nicola
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>