You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Grant Ingersoll (JIRA)" <ji...@apache.org> on 2011/02/16 14:33:57 UTC
[jira] Created: (SOLR-2366) Facet Range Gaps
Facet Range Gaps
----------------
Key: SOLR-2366
URL: https://issues.apache.org/jira/browse/SOLR-2366
Project: Solr
Issue Type: Improvement
Reporter: Grant Ingersoll
Priority: Minor
Fix For: 3.2, 4.0
There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
For instance,
facet.range.start=0
facet.range.end=400
facet.range.gap=5,25,50,100
would yield buckets of:
0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13055793#comment-13055793 ]
Hoss Man commented on SOLR-2366:
--------------------------------
bq. Guess my main point with the examples was to suggest that a facet.range.spec should not require facet.range.start and facet.range.end, but that the first and last values in the spec list should be taken as start and end, instead of requiring start and end in addition. ...
bq. Simply document that facet.range.spec is mutually exclusive to the parameters gap,start,end and other.
I respect your argument, but i think if this new "spec" param is going to be mutually exclusive of facet.range.other as well as all of the existing mandatory facet.range params (facet.range.gap, facet.range.start, and facet.range.end) then it seems like what you're describing really shouldn't be an extension of "facet.range" at all ... it sounds should be some completley distinct type of faceting ("sequence faceting" ?) with it's own params and section in the response. ie...
{noformat}
facet.seq=fieldName
f.fieldName.facet.seq.spec=0,5,25,50,100,200,400,*
f.fieldName.facet.seq.include=edge
{noformat}
(where facet.seq.include has same semantics as facet.range.include ... except i don't think "edge" makes sense at all w/o the "other" param concept ... need to think it through more)
Otherwise it could get really confusing for users trying to udnerstandwhat "facet.range.*" params do/don't make sense if they start using facet.range.gap and then switch to facet.range.spec (or vice-versa) ... ie: "how come i'm not getting the before/after ranges when i use 'facet.range.spec=0,5,25,50&facet.range.other=after' ?")
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.3
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056337#comment-13056337 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
I disagree that this is a fundamentally different feature requring its own plugin. It's simply an alternative way of specifying the gaps for range facet. I won't mind working through the documentation to describe clearly how facet.range.spec interacts with the other params, and also implement a parameter check which throws an exception if the user supplies incompatible params.
What do others think?
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.3
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996164#comment-12996164 ]
Grant Ingersoll commented on SOLR-2366:
---------------------------------------
Hoss, I can live with ranges. I had originally thought of doing that, but decided this syntax is simpler, especially for dates. Then again, we could just as well support both. The nice thing doing ranges gives you is you can have non-contiguous ranges, which might be interesting to some.
As for:
bq. would be hard pressed to explain why the resulting ranges were...
It really isn't that hard to explain:
{quote}start + gap[0], prevEnd + gap[1], ... prevEnd + gap[i], ... prevEnd + gap[n] (and repeating until end){quote}
In other words, it's a variable width gap starting at whatever the last end point was.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995838#comment-12995838 ]
Ryan McKinley commented on SOLR-2366:
-------------------------------------
I agree with grant that this syntax is more clear then using facet.query for each bucket.
Just throwing it out there... but is there a way to not specify the start/end, and have that based on the values in the index? start=* end=*? In this case, it would be nice to specify the gap as round numbers. Perhaps gap=%10? assuming you have the values: 22,28,35, that would give you gaps for 20-30 and 30-40
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015088#comment-13015088 ]
Hoss Man commented on SOLR-2366:
--------------------------------
In no particular order...
* I like Jan's {{facet.range.spec}} naming suggestion better then my {{facet.range.buckets}} suggestion ... but i think {{facet.range.series}}, {{facet.range.seq}}, or {{facet.range.sequence}} might be better still.
* I think Jan's point about {{N}} vs {{+N}} in the sequence list as a way to mix absolute values vs increments definitely makes sense, and would be consistent with the existing date match expression.
* the complexity with supporting *both* absolute values and increments would be the question of what solr should do with input like {{facet.range.seq=10,20,+50,+100,120,150}} ? what ranges would we return? (10-20, 20-70, 70-???....) would it be an error? would we give back ranges that overlapped? what about {{facet.range.seq=10,50,+50,100,150&facet.range.include=all}} .. would that result in one of the ranges being [100 TO 100] or would we throw that one out? (I think it would be wise to start out only implementing the absolute value approavh, since that seems (to me) the more useful option of the two, and then consider adding the incremental values as a separate issue later after hashing out hte semantics of these types of situations)
* A few of Jan's sample input suggestions used {{*}} at either the start or end of the sequence to denote "everything before" the second value or "everything after" the second to last value -- i don't think we need to support this syntax, I think the existing {{facet.range.other}} would still be the right way to support this with {{facet.range.sequence}}. if you want "everything before" and/or "everything after" use {{facet.range.include=before}} and/or {{facet.range.include=after}} .. otherwise it would be confusing to decide what things like {{facet.range.include=before&facet.range.seq=*,10,20}} and {{facet.range.include=none&facet.range.seq=*,10,20}} mean.
* I *REALLY* don't think we should try to implement something like Jan's {{facet.range.labels}} suggestion. I can't imagine any way of supporting it thta wouldn't prevent or radically complicate the "..." type continuation of series i suggested before, and that seems like a much more powerful feature then labels. if a user is going to provide a label for every range, then you must enumerate every range, and you might as well enumerate them (and label them) with {{facet.query}} where the label and the query can be side by side.
This...
{code}
facet.query={!label="One or more"}bedrooms:[1 TO *]
facet.query={!label="Two or more"}bedrooms:[2 TO *]
facet.query={!label="Three or more"}bedrooms:[3 TO *]
facet.query={!label="Four or more"}bedrooms:[4 TO *]
{code}
...seems way more readable, and less prone to user error in tweaking, then this...
{code}
f.bedrooms.facet.range.spec=1..*,2..*,3..*,4..*
f.bedrooms.facet.range.labels="One or more","Two or more","Three or more","Four or more"
{code}
* Herman commented...
bq. While using fact.query allows us to construct arbitrary ranges, we must then pick them out of the results separately. This becomes more difficult if we arbitrarily facet on two or more fields/expressions.
I don't see that as being particularly hard problem that we need to worry about helping users avoid, Especially since users can anotate those queries using localparams and set any arbitrary key=val pairs on them that you want to help organize them and identify them later when parsing the response...
{code}
facet.query={!group=bed label="One or more"}bedrooms:[1 TO *]
facet.query={!group=bed label="Two or more"}bedrooms:[2 TO *]
facet.query={!group=bed label="Three or more"}bedrooms:[3 TO *]
facet.query={!group=bed label="Four or more"}bedrooms:[4 TO *]
facet.query={!group=size label="Small"}sqft:[* TO 1000]
facet.query={!group=size label="Medium"}sqft:[1000 TO 2500]
facet.query={!group=size label="Large"}sqft:[2500 TO *]
{code}
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995585#comment-12995585 ]
Hoss Man commented on SOLR-2366:
--------------------------------
the use case of facet.range (and facet.date before it) was always about having ranges generated for you automatcly using a fixed gap size. if you want variable gap sizes, it's just as easy to specify them using facet.query.
i don't really understand how your proposal adds value over using facet.query for the ranges you want to have specific widths, and then using facet.range for the rest of the ranges you want generated automaticly with a specific gap.
it just seems like a more confusing way of expressing the same thing
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243611#comment-13243611 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
Note to self: catch up on this again :)
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets.
> (Original syntax proposal removed, see discussion for concrete syntax)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-2366) Facet Range Gaps
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015088#comment-13015088 ]
Hoss Man edited comment on SOLR-2366 at 4/2/11 11:48 PM:
---------------------------------------------------------
In no particular order...
* I like Jan's {{facet.range.spec}} naming suggestion better then my {{facet.range.buckets}} suggestion ... but i think {{facet.range.series}}, {{facet.range.seq}}, or {{facet.range.sequence}} might be better still.
* I think Jan's point about {{N}} vs {{+N}} in the sequence list as a way to mix absolute values vs increments definitely makes sense, and would be consistent with the existing date match expression.
* the complexity with supporting *both* absolute values and increments would be the question of what solr should do with input like {{facet.range.seq=10,20,+50,+100,120,150}} ? what ranges would we return? (10-20, 20-70, 70-???....) would it be an error? would we give back ranges that overlapped? what about {{facet.range.seq=10,50,+50,100,150&facet.range.include=all}} .. would that result in one of the ranges being [100 TO 100] or would we throw that one out? (I think it would be wise to start out only implementing the absolute value approavh, since that seems (to me) the more useful option of the two, and then consider adding the incremental values as a separate issue later after hashing out hte semantics of these types of situations)
* A few of Jan's sample input suggestions used {{ * }} at either the start or end of the sequence to denote "everything before" the second value or "everything after" the second to last value -- i don't think we need to support this syntax, I think the existing {{facet.range.other}} would still be the right way to support this with {{facet.range.sequence}}. if you want "everything before" and/or "everything after" use {{facet.range.include=before}} and/or {{facet.range.include=after}} .. otherwise it would be confusing to decide what things like {{facet.range.include=before&facet.range.seq=*,10,20}} and {{facet.range.include=none&facet.range.seq= * ,10,20}} mean.
* I *REALLY* don't think we should try to implement something like Jan's {{facet.range.labels}} suggestion. I can't imagine any way of supporting it thta wouldn't prevent or radically complicate the "..." type continuation of series i suggested before, and that seems like a much more powerful feature then labels. if a user is going to provide a label for every range, then you must enumerate every range, and you might as well enumerate them (and label them) with {{facet.query}} where the label and the query can be side by side.
This...
{code}
facet.query={!label="One or more"}bedrooms:[1 TO *]
facet.query={!label="Two or more"}bedrooms:[2 TO *]
facet.query={!label="Three or more"}bedrooms:[3 TO *]
facet.query={!label="Four or more"}bedrooms:[4 TO *]
{code}
...seems way more readable, and less prone to user error in tweaking, then this...
{code}
f.bedrooms.facet.range.spec=1..*,2..*,3..*,4..*
f.bedrooms.facet.range.labels="One or more","Two or more","Three or more","Four or more"
{code}
* Herman commented...
bq. While using fact.query allows us to construct arbitrary ranges, we must then pick them out of the results separately. This becomes more difficult if we arbitrarily facet on two or more fields/expressions.
I don't see that as being particularly hard problem that we need to worry about helping users avoid, Especially since users can anotate those queries using localparams and set any arbitrary key=val pairs on them that you want to help organize them and identify them later when parsing the response...
{code}
facet.query={!group=bed label="One or more"}bedrooms:[1 TO *]
facet.query={!group=bed label="Two or more"}bedrooms:[2 TO *]
facet.query={!group=bed label="Three or more"}bedrooms:[3 TO *]
facet.query={!group=bed label="Four or more"}bedrooms:[4 TO *]
facet.query={!group=size label="Small"}sqft:[* TO 1000]
facet.query={!group=size label="Medium"}sqft:[1000 TO 2500]
facet.query={!group=size label="Large"}sqft:[2500 TO *]
{code}
was (Author: hossman):
In no particular order...
* I like Jan's {{facet.range.spec}} naming suggestion better then my {{facet.range.buckets}} suggestion ... but i think {{facet.range.series}}, {{facet.range.seq}}, or {{facet.range.sequence}} might be better still.
* I think Jan's point about {{N}} vs {{+N}} in the sequence list as a way to mix absolute values vs increments definitely makes sense, and would be consistent with the existing date match expression.
* the complexity with supporting *both* absolute values and increments would be the question of what solr should do with input like {{facet.range.seq=10,20,+50,+100,120,150}} ? what ranges would we return? (10-20, 20-70, 70-???....) would it be an error? would we give back ranges that overlapped? what about {{facet.range.seq=10,50,+50,100,150&facet.range.include=all}} .. would that result in one of the ranges being [100 TO 100] or would we throw that one out? (I think it would be wise to start out only implementing the absolute value approavh, since that seems (to me) the more useful option of the two, and then consider adding the incremental values as a separate issue later after hashing out hte semantics of these types of situations)
* A few of Jan's sample input suggestions used {{*}} at either the start or end of the sequence to denote "everything before" the second value or "everything after" the second to last value -- i don't think we need to support this syntax, I think the existing {{facet.range.other}} would still be the right way to support this with {{facet.range.sequence}}. if you want "everything before" and/or "everything after" use {{facet.range.include=before}} and/or {{facet.range.include=after}} .. otherwise it would be confusing to decide what things like {{facet.range.include=before&facet.range.seq=*,10,20}} and {{facet.range.include=none&facet.range.seq=*,10,20}} mean.
* I *REALLY* don't think we should try to implement something like Jan's {{facet.range.labels}} suggestion. I can't imagine any way of supporting it thta wouldn't prevent or radically complicate the "..." type continuation of series i suggested before, and that seems like a much more powerful feature then labels. if a user is going to provide a label for every range, then you must enumerate every range, and you might as well enumerate them (and label them) with {{facet.query}} where the label and the query can be side by side.
This...
{code}
facet.query={!label="One or more"}bedrooms:[1 TO *]
facet.query={!label="Two or more"}bedrooms:[2 TO *]
facet.query={!label="Three or more"}bedrooms:[3 TO *]
facet.query={!label="Four or more"}bedrooms:[4 TO *]
{code}
...seems way more readable, and less prone to user error in tweaking, then this...
{code}
f.bedrooms.facet.range.spec=1..*,2..*,3..*,4..*
f.bedrooms.facet.range.labels="One or more","Two or more","Three or more","Four or more"
{code}
* Herman commented...
bq. While using fact.query allows us to construct arbitrary ranges, we must then pick them out of the results separately. This becomes more difficult if we arbitrarily facet on two or more fields/expressions.
I don't see that as being particularly hard problem that we need to worry about helping users avoid, Especially since users can anotate those queries using localparams and set any arbitrary key=val pairs on them that you want to help organize them and identify them later when parsing the response...
{code}
facet.query={!group=bed label="One or more"}bedrooms:[1 TO *]
facet.query={!group=bed label="Two or more"}bedrooms:[2 TO *]
facet.query={!group=bed label="Three or more"}bedrooms:[3 TO *]
facet.query={!group=bed label="Four or more"}bedrooms:[4 TO *]
facet.query={!group=size label="Small"}sqft:[* TO 1000]
facet.query={!group=size label="Medium"}sqft:[1000 TO 2500]
facet.query={!group=size label="Large"}sqft:[2500 TO *]
{code}
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101185#comment-13101185 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
I've given the Wiki page another take, with the new proposed start/end syntax and added an example or two. The "mutually exclusive" sentence now boils down to facet.range.gap/facet.range.spec being mutually exclusive (one the same field). Have a look at http://wiki.apache.org/solr/VariableRangeGaps#facet.range.spec
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.4, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996176#comment-12996176 ]
Yonik Seeley commented on SOLR-2366:
------------------------------------
bq. would be hard pressed to explain why the resulting ranges were...
I agree - it requires summing all previous deltas to figure out what the current range actually is.
I think we need to drive this from use-cases. The first use case that comes to mind is price ranges... and that would be a pain to insert a new price range if we were just dealing with a list of deltas. Anything I can think of where you would want variable sized buckets, it seems like you care more about the absolute values of those buckets, rather than their delta to the previous bucket.
I pretty much came up with what Hoss suggested I think (except I didn't think of the "..." syntax).
We could potentially support a mix of absolute starting points and ranges:
0,5,10,20,100-1000
Normally one would stick to one syntax or the other in a single request, but we could support both in a single parameter as a convenience.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Høydahl updated SOLR-2366:
------------------------------
Description:
There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets.
(Original syntax proposal removed, see discussion for concrete syntax)
was:
There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
For instance,
facet.range.start=0
facet.range.end=400
facet.range.gap=5,25,50,100
would yield buckets of:
0-5,5-30,30-80,80-180,180-280,280-380,380-400
Here's Grant's original syntax proposal which is removed from issue description to avoid confusion:
{quote}
I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
For instance,
facet.range.start=0
facet.range.end=400
facet.range.gap=5,25,50,100
would yield buckets of:
0-5,5-30,30-80,80-180,180-280,280-380,380-400
{quote}
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.4, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets.
> (Original syntax proposal removed, see discussion for concrete syntax)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101204#comment-13101204 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
One thing this improvement needs to tackle is how to return the range buckets in the Response. It will not be enough with the simple range_facet format
{code:xml}
<lst name="facet_ranges">
<lst name="url_length">
<lst name="counts">
<int name="42">1</int>
<int name="45">1</int>
<int name="51">1</int>
<int name="66">1</int>
</lst>
<int name="gap">3</int>
<int name="start">0</int>
<int name="end">102</int>
</lst>
</lst>
{code}
We need something which can return the explicit ranges, similar to what facet_queries has. This format can then be used for the old plain gap format as well.
{code:xml}
<lst name="facet_ranges">
<lst name="url_length">
<lst name="counts">
<int name="[42 TO 45}">1</int>
<int name="[45 TO 48}">1</int>
<int name="[51 TO 54}">1</int>
<int name="[66 TO 69}">1</int>
</lst>
<int name="gap">3</int>
<int name="start">0</int>
<int name="end">102</int>
</lst>
<lst name="bedrooms">
<lst name="counts">
<int name="[1 TO *]">12</int>
<int name="[2 TO *]">31</int>
<int name="[3 TO *]">26</int>
<int name="[4 TO *]">9</int>
</lst>
<int name="spec">1..*,2..*,3..*,4..*</int>
<int name="include">all</int>
</lst>
</lst>
{code}
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.4, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets.
> (Original syntax proposal removed, see discussion for concrete syntax)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Herman J Kiefus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000286#comment-13000286 ]
Herman J Kiefus commented on SOLR-2366:
---------------------------------------
With absolute ranges (no gap) couldn’t we also support alphabetic ranges? I would find this useful.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995870#comment-12995870 ]
Ryan McKinley commented on SOLR-2366:
-------------------------------------
bq. sometimes you want percentage buckets
This does not map easily since you need to know the min/max value before traversing -- it may actually take two passes.
The use case I am looking at is trying to show reasonable histograms for field values, without really knowing much about the field values as input. Currently I run the stats component, and then in a second query facet within that range -- gets the job done, but not ideal.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099315#comment-13099315 ]
Hoss Man commented on SOLR-2366:
--------------------------------
Jan: i took a look at r3 of your VariableRangeGaps wiki, here are the things I'm concerned about because they seem a bit confusing/ambiguious....
1) we need to decide what the behavior should be when the spec identifies values out of order (ie: {{10, 50, 30}}) ... it might be tempting to say "allow them, and swap the values" (ie: "10-50, 30-50") but the merit of that approach doesn't seem worth the potential risk of silently hiding errors (ie: if the user made a typo and ment "10-50, 50-130") not to mention it could be really hard to understand what's going on in the case where some values are specified absolutely and some are specified as incriments (see bullet #3 in my "02/Apr/11 23:43" comment above -- ie: what ranges would we produce for {{10,20,+50,+100,120,150}} ?).
I would suggest define any case where the spec contains absolute value N after (effective) value M where N < M as an error and fail fast.
Still not sure what (if anything) should be done about overlapping ranges that appear out of order (ie: {{0,100,50..90,150}} ... is that "0-100,50-90,90-150" ?)
2) Independent of my opinion on the {{*}} syntax, I'm a little concerned by the descrepency in these examples...
{noformat}
facet.range.spec=*,10,50,100,250,* - gives 5 ranges: MIN-10, 10-50, 50-100, 100-250, 250->MAX
facet.range.spec=*,10,+40,+50,250,* - gives exactly the same ranges, using relative gap size
facet.range.spec=0,+10,50,250,* - gives ranges: 0-10, 10-20, 20-30, 30-40, 40-50, 20-250, 250-MAX
facet.range.spec=0,10,50,+50,+100,* - gives ranges: 0-10, 10-50, 50-100, 100-200, 200-300 repeating until max
{noformat}
The first three examples suggest that {{*}} will be treated as "-Infinity" and "+Infinity" based on position (ie: the first and last ranges will be unbounded on one end) but in the last example the wording "...100-200, 200-300 repeating until max" seems inconsistent with that.
In general, i'm concerned about providing a feature that would attempt to produce an infinite number of range queries, but even if that is intentional/acceptible the discrepency in syntax bothers me -- I would suggest that that sequence should result in the ranges "0-10, 10-50, 50-100, 100-200, 200-Infinity"
If we want to support the idea of "repeat the last increment continuously" that should be with it's own "repeat" syntax such as the "..." (three dots) i suggested in comment "17/Feb/11 23:50" above. I would argue that this should only be legal after an increment and before a concrete value (ie: {{0,+10,...,100}}). Requiring it to follow an increment seems like a given (otherwise what exactly are you repeating?) requiring that it be followed by an absolute value is based on my concern that if it's the last item in the spec (or the last item before {{*}}) it results in an infinite number of ranges.
3) The final comment on the page says (in section about facet.range.spec) ...
{quote}
This parameter can be combined with facet.range.include, but is mutually exclusive to facet.range.gap, facet.range.begin, facet.range.end and facet.range.other, resulting in an exception if uncompatible mix is attempted.
{quote}
That seems like it isn't specific enough about what is/isn't going to be allowed -- particularly since all of the facet.range params can be specified on a per field basis.
Imagine an index of "historic people" docs that provides range faceting on a bunch of date fields for significant milestones using common facet.range.start, facet.range.end, facet.range.gap params - and the solr admin wants to add "facet.range=height" and a "f.height.facet.range.spec" param....
{code}
facet.range=birth_date
facet.range=first_notable_historic_event
facet.range=last_notable_historic_event
facet.range=death_date
facet.range.start=1500-01-01T00:00:00Z
facet.range.end=NOW/YEAR+1YEAR
facet.range.hardend=false
facet.range.gap=+10YEARS
facet.range=height
f.height.facet.range.spec=*,100,+10,...,300,*
{code}
...that should be a totally legal usecase right? to mix and match this way? but how will the code behave? Technically the "height" field has both a facet.range.spec and facet.range.start params specified and there is no way to "unset" the default facet.range.start/facet.range.end/facet.range.gap params in the context of the "height" field
4) Related to the same sentence as #3, it says that facet.range.include can be used with facet.range.spec, but it doesn't explain how it will be interpreted -- this is kind of important since values like "outer" define how the "before" and "after" ranges are affected, and values like "edge" affect the "first" and "last" "gap ranges".
Should all ranges produced by facet.range.spec be considered "gap" ranges? even the ones with no lower/upper bound?
What would the following combination mean...
{code}
facet.range.spec=100,150,200,250*
facet.range.include=outer
facet.range.include=edge
{code}
* Are "100" and "250" considered "edge" boundaries?
* Is "250" considered an "outer" boundery (on the equivilent of an "after" range) ?
What about when the spec includes overlapping ranges?
{code}
facet.range.spec=50..150,100..200,150,*
facet.range.include=outer
facet.range.include=edge
{code}
* Is "200" an "edge" boundary?
* Is "150" an "outer" boundary?
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.4, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2366) Facet Range Gaps
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated SOLR-2366:
----------------------------------
Attachment: SOLR-2366.patch
Adds variable width gap capabilities and some tests. Still needs some more tests for edge conditions, etc. but it is something that others can look at and comment on.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101152#comment-13101152 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
Hoss: Good comments, which need to be decided upon, including corner cases.
1)
bq.I would suggest define any case where the spec contains absolute value N after (effective) value M where N < M as an error and fail fast.
Agree
bq.Still not sure what (if anything) should be done about overlapping ranges that appear out of order (ie: 0,100,50..90,150 ... is that "0-100,50-90,90-150" ?)
If all gaps are specified as explicit ranges this is no ambiguity, so we could require all gaps to be explicit ranges if one wants to use it?
2)
bq.The first three examples suggest that * will be treated as "-Infinity" and "+Infinity" based on position (ie: the first and last ranges will be unbounded on one end) but in the last example the wording "...100-200, 200-300 repeating until max" seems inconsistent with that.
Agree. The 0,10,50,+50,+100,* example would create infinite gaps which would be less than desireable. But 0,10,50,+50,+100,500 would give repeating 100-gaps until upper bound 500, while 0,10,50,+50,+100,500,* would in addition give a last range 500-*. That was the intentional syntax.
bq.If we want to support the idea of "repeat the last increment continuously" that should be with it's own "repeat" syntax such as the "..." (three dots) i suggested in comment "17/Feb/11 23:50" above. I would argue that this should only be legal after an increment and before a concrete value (ie: 0,+10,...,100). Requiring it to follow an increment seems like a given (otherwise what exactly are you repeating?) requiring that it be followed by an absolute value is based on my concern that if it's the last item in the spec (or the last item before *) it results in an infinite number of ranges.
Agree. Alternatively, if Solr could compute myField.max(), the useful value of "*" could be computed a bit smarter, but that would probably be hard to scale in a multi-shard setting.
bq.That seems like it isn't specific enough about what is/isn't going to be allowed – particularly since all of the facet.range params can be specified on a per field basis.
Didn't really think much about the global params. Silently not caring about gap, begin, end, other would be one way to go, but then the error feedback is not explicit in case of misunderstanding; the user will see that he does not get back what he thought, and start reading the documentation :)
I have no good answer to this, other than inventing some syntax. The default could be that facet.range.spec respects the global values for start and end, but also allow explicitly overriding start and end values as part of spec with a special syntax.
The following params would result in ranges 0-1, 1-2, 2-3, 3-5, 5-10 :
{noformat}
facet.range.start=0
facet.range.end=10
facet.range.gap=2
f.bedrooms.facet.range.spec=1,2,3,5
{noformat}
But these params would result in the same ranges because we specify start and end with a special syntax N.. for start and ..M for end:
{noformat}
facet.range.start=100
facet.range.end=200
facet.range.gap=10
f.bedrooms.facet.range.spec=0..,1,2,3,5,..10
{noformat}
This would be equivalent with adding the two params f.bedrooms.facet.range.start=0&f.bedrooms.facet.range.end=10, which could then still be allowed as an alternative. If the first value of the spec is not an N.., we'll require a facet.range.start. If the last value of the spec is not ..M, we'll require facet.range.end.
Also, it must not be allowed to specify both a global facet.range.gap and a global facet.range.spec.
Would this be a good "compromise"? :-) My primary reason for suggesting this is to give users a terse, intuitive syntax for ranges.
4)
bq.Should all ranges produced by facet.range.spec be considered "gap" ranges? even the ones with no lower/upper bound?
Good question. I think the values facet.range.include=upper/lower is clear. Outer/edge would need some more work/definition.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.4, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995652#comment-12995652 ]
Grant Ingersoll commented on SOLR-2366:
---------------------------------------
bq. it just seems like a more confusing way of expressing the same thing
I think it's a lot less confusing. You only have to express start, end and the size of the buckets you want. With facet.query, you have to write out each expression for every bucket and do the math on all the boundaries. I don't think it is just as easy to specify using facet.query. Not too mention that facet.query also involves a lot more parsing.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996341#comment-12996341 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
+1 for using absolute values instead of gap values
+1 for keeping the bucket spec as a separate param, including start and end
+1 for letting the start/end in the spec automatically disable hardend
I wrote down some thoughts the other day which is almost exactly what Hoss suggests, only I called it facet.range.spec :) Was going to start another issue but now that the dicussion is rolling here, here we go.
The facet.range.spec must be intuitive and should include start, all absolute boundaries and end. Sample:
{code}
facet.range.spec=0,5,25,50,100,400 ==> 0-5, 5-25, 25-50, 50-100, 100-400.
{code}
To specify the gap size instead of next absolute threshold, we could have a +N syntax:
{code}
facet.range.spec=0,5,25,+25,+50,400
{code}
would be equivalent to the above absolute spec.
A +N value would repeat as many times as needed to reach the next absolute value:
{code}
facet.range.spec=0,5,+10,25,50,100,+100,400 ==> 0-5, 5-15, 15-25, 25-50, 50-100, 100-200, 200-300, 300-400
facet.range.spec=0,5,+10,25,50,100,+100,400 ==> 0-5, 5-15, 15-25, 25-50, 50-100, 100-200, 200-300, 300-400
{code}
Date example:
{code}
facet.range.spec=*,2000-01-01T00:00:00Z,+5YEARS,NOW/YEAR,+1MONTH,NOW
{code}
...gives a range before 2000, two 5-year ranges 2000-2005, 2005-2010, one range until start of this year 2010-2011, then monthly ranges for this year until now.
Now, having all this power of defining buckets available, it would be easy to introduce (i.e. feature creep :) a facet.range.labels param. Imagine:
{code}
facet.range.spec=NOW/MONTH-1MONTH,NOW/MONTH,NOW/DAY-1DAY,NOW/DAY,NOW/HOUR,NOW,*
facet.range.labels="Last month","This month","Yesterday","Today","This hour","Future"
{code}
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996154#comment-12996154 ]
Hoss Man commented on SOLR-2366:
--------------------------------
(FYI: i haven't looked at the patch, because i'm trying to focus on 3.1 bug fixes, but grant specifically called me out on this on irc, so i'm replying based purely on the comments)
bq. I think it's a lot less confusing. You only have to express start, end and the size of the buckets you want. With facet.query, you have to write out each expression for every bucket and do the math on all the boundaries.
ok ... fair enough, i can't deny the syntax you are proposing would be _easier_ then specifying individual facet.query params, i'm just not convinced it would be completely intuitive. If i told someone about this feature, and then showed them this request...
{code}facet.range.start=10&facet.range.end=100&facet.range.gap=10,20,50{code}
I would be hard pressed to explain why the resulting ranges were...
{code}10-20, 20-40, 40-90, and 90-190{code}
...instead of...
{code}10-20, 10-30, 10-60, and 60-110{code}
(bearing in mind: facet.range.hardend defaults to "false")
the existing start/end/gap params may not be 100% intuitive purely by name, but once you read about them once, they are fairly easy to grasp and not very confusing at all when you read examples later. likewise, a collection of facet.query objects is fairly intuitive and unambiguious. I just don't feel that way about what you are suggesting (then agian: i unleashed "mm" on the world, so i'm not really in a good position to throw stones)
I'm also not convinced that it really makes sense in use cases like this (where you want variable sized buckets) to specify the *gap sizes* as a list, instead of the specifying the *boundaries* on each bucket.
What you are describing almost feels like it should be a new category of faceting -- or a variation on range faceting that doesn't involve the start/end/gap params at all (but could still respects facet.range.include and facet.range.other)
Here's my counter-proposal/suggestion...
I'm imagining a facet.range.buckets param that (if present) would override facet.range.gap, facet.range.start, and facet.range.end (so using facet.range would require *either* bucket or start/end/gap). facet.range.buckets would take a comma separated list of value representing the specific values you wanted to see used to define adjoining range boundary points, with some syntax ("..." seems natural) indicating "repeat last range size until reach this next value"
so you could say...
{code}facet.range=price&facet.range.buckets=0,10,25,50,100,...,300{code}
...and the resulting ranges computed would be...
{code}0-10, 10-25, 25-50, 50-100, 100-150, 150-200, 200-250, 250-300{code}
...likewise you could say...
{code}facet.range=age&facet.range.buckets=0,1,...,18,25,40,60,...,100{code}
...and you would get ranges for each year from 0 to 18, followed by 18-25, 25-40, 40-60, 60-80, 80-100.
The tricky situations would be things like...
# {code}facet.range.buckets=0,2,3,...,10{code}
# {code}facet.range.buckets=0,7,...,10,20{code}
...the first _could_ be dealt with using facet.range.hardend like we do today (so the resulting buckets were "0-2,2-5,5-8,8-11") but i don't think it should. I think it should result in "0-2,2-5,5-8,8-10" ... it's hard to imaging letting a param like facet.range.hardend override the explicit "10" in the buckets list when we don't have programaticly generate buckets of precisesly the same size, particularly when you consider the implications that would carry over to the second case (i *really* can't imagine letting that produce any ranges other then "0-7,7-10,10-20")
So yeah ... that's what i think would make more sense then letting you specify a comma seperated list in the "gaps" param ... fundamentally i think it comes down to the point i alluded to earlier in this comment: is specifying a sequence of varying gap sizes more intuitive for this type of use case then specifying a sequence of boundary points? i don't think it is.
(PS: i think the discussion about dynamically generating range points based on stats in the index should really be tracked in a distinct issue ... it's got a lot of complexity to it that we've talked about on the mailing list a few times that i don't really want to try and get into now)
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015216#comment-13015216 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
bq. If you want "everything before" and/or "everything after" use facet.range.include=before and/or facet.range.include=after .. otherwise it would be confusing to decide what things like facet.range.include=before&facet.range.seq=*,10,20 and facet.range.include=none&facet.range.seq= * ,10,20 mean.
I think you meant facet.range.other=before/after, not facet.range.include=before/after - see, the syntax is confusing :)
Guess my main point with the examples was to suggest that a facet.range.spec should not require facet.range.start and facet.range.end, but that the first and last values in the spec list should be taken as start and end, instead of requiring start and end in addition. In my opinion
{code}
facet.range.spec=0,5,25,50,100,200,400
{code}
is more fluent and easy to read that the first and last buckets will be 0-5 and 200-400, than with
{code}
facet.range.spec=5,25,50,100,200
facet.range.start=0
facet.range.end=400
{code}
and when talking about before/after,
{code}
facet.range.spec=0,5,25,50,100,200,400,*
{code}
is in my mind better than
{code}
facet.range.spec=5,25,50,100,200
facet.range.start=0
facet.range.end=400
facet.range.other=after
{code}
Simply document that facet.range.spec is mutually exclusive to the parameters gap,start,end and other.
bq. I REALLY don't think we should try to implement something like Jan's facet.range.labels suggestion
Sure, this is not a priority since it's possible with facet.query
+1 on concentrating on a simple "spec" or "sequence" feature in some flavour
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092956#comment-13092956 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
I've attempted a possible documentation of the facet.range.spec param as I envision it, at http://wiki.apache.org/solr/VariableRangeGaps
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.4, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Jan Høydahl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996358#comment-12996358 ]
Jan Høydahl commented on SOLR-2366:
-----------------------------------
Both the date ranges above and other typical use cases call for overlapping buckets. This would be a generalization of Yonik's range suggestion. Imagine a real estate site with a bedrooms facet:
{code}
f.bedrooms.facet.range.spec=1..*,2..*,3..*,4..*
f.bedrooms.facet.range.labels="One or more","Two or more","Three or more","Four or more"
{code}
I've chosen ".." as range delimiter since "-" would be confused with Date Math.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995843#comment-12995843 ]
Grant Ingersoll commented on SOLR-2366:
---------------------------------------
{quote}Just throwing it out there... but is there a way to not specify the start/end, and have that based on the values in the index? start=* end=*? In this case, it would be nice to specify the gap as round numbers. Perhaps gap=%10? assuming you have the values: 22,28,35, that would give you gaps for 20-30 and 30-40{quote}
Ryan, I think that's also an excellent variation. Sometimes you want hard start/ends, sometimes you want percentage buckets, especially for the thing I'm working on now, which is facet by function
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2366) Facet Range Gaps
Posted by "Hoss Man (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124591#comment-13124591 ]
Hoss Man commented on SOLR-2366:
--------------------------------
Jan: I've got to be completely honest here -- catching up on this issue, I got really confused and lost by some of your comments and the updated docs.
This sequence of comments really stands out at me...
{quote}
I have no good answer to this, other than inventing some syntax.
...
I think the values facet.range.include=upper/lower is clear. Outer/edge would need some more work/definition.
...
*My primary reason for suggesting this is to give users a terse, intuitive syntax for ranges.*
...
One thing this improvement needs to tackle is how to return the range buckets in the Response. It will not be enough with the simple range_facet format ... We need something which can return the explicit ranges,
{quote}
(emphasis added by me)
I really liked the simplicity of your earlier proposal, and I agree that it would be really powerful/helpful to give users a terse, intuitive syntax for specifying sequential ranges of variable sizes -- but it seems like we're really moving away from the syntax being "intuitive" because of the hoops you're having to jump through to treat this as an extension of the existing "facet.range" param in your design.
I think we really ought to revisit my earlier suggestion to approach this as an entirely new "type" of faceting - not a new plugin or a contrib, but a new first-class type of faceting that FacetComponent would support, right along side facet.field, facet.query, and facet.range. Let's ignore everything about the existing facet.range.* param syntax, and the facet_range response format, and think about what makes the most sense for this feature on it's own. If there are ideas from facet.range that make sense to carry over (like facet.range.include) then great -- but let's approach it from the "something new that can borrow from facet.range" standpoint instead of the "extension to facet.range that has a bunch of caveats with how facet.range already works"
I mean: if it looks like a duck, walks like a duck, and quacks like a duck, then i'm happy to call it a duck -- but in this case:
* doesn't make sense with facet.range.other
* needs special start/end syntax to play nice with facet.range.start/end
* needs to change the response format
...ie: it doesn't look the same, it doesn't walk the same, and it doesn't quack.
---
Regardless of whether this functionality becomes part of facet.range or not, I wanted to comment specifically on this idea...
bq. If all gaps are specified as explicit ranges this is no ambiguity, so we could require all gaps to be explicit ranges if one wants to use it?
This seems like a really harsh limitation to impose. If the only way to use an explicit range is in use cases where you *only* use explicit ranges, then what value add does this feature give you over just using multiple facet.query params? (it might be marginally fewer characters, but multiple facet.query params seem more intuitive and easier to read). I mean: I don't have a solution to propose, it just seems like there's not much point in supporting explicit ranges in that case.
---
Having not thought about this issue in almost a month, and revisiting it with (fairly) fresh eyes, and thinking about all the use cases that have been discussed, it seems like the main goals we should address are really:
* an intuitive syntax for specifying end points for ranges of varying sizes
* ability to specify range end points using either fixed values or increments
* ability to specify that ranges should be either use sequential end points, or be overlapping relative some fixed min/max value
In other words: the only reason (that i know of) why overlapping ranges even came up in this issue was use cases like...
{noformat}
Price: $0-10, $0-20, $0-50, $0-100
Date: NOW-1DAY TO NOW, NOW-1MONTH TO NOW, NOW-1YEAR TO NOW
{noformat}
...there doesn't seem to be a lot of motivations for using overlapping ranges in the "middle" of a sequence, and these types of use cases where *all* the ranges overlap seem just as important as use cases where the ranges don't overlap at all...
{noformat}
Price: $0-10, $10-20, $20-50, $50-100
Date: NOW-1DAY TO NOW, NOW-1MONTH TO NOW-1DAY, NOW-1YEAR TO NOW-1MONTH
{noformat}
...so let's try to focus on a syntax that makes both easy, using both fixed and relative values, w/o worrying about supporting arbitrary overlapping ranges (since I can't think of a use case for it, and it could always be achieved using facet.query)
So how about something like...
{noformat}
facet.sequence=<fieldname>
facet.sequence.spec=[<wild>,]?<val>,<relval>[,<relval>]*[,<wild>]?
facet.sequence.type=[before|after|between]
facet.sequence.include=(same as facet.range.include)
{noformat}
Where "relval" would either be a concrete value, or a relative value; the effective sequence has to either increase or decrease consistently or it's an error; and "facet.sequence.type" determines whether the ranges are overlapping ("before" and "after") or not ("between")
So if you had a spec like this...
{noformat}
facet.sequence.spec=0,10,+10,50,+50
{noformat}
Then depending on facet.sequence.type you could either get...
{noformat}
facet.sequence.type=after
Price: $0-10, $0-20, $0-50, $0-100
facet.sequence.type=between
Price: $0-10, $10-20, $20-50, $50-100
facet.sequence.type=before
Price: $0-100, $10-100, $20-100, $50-100
{noformat}
"*" could be used at the start or end to indicate that you wanted an unbounded range, but it wouldn't be a factor in determining the "fixed point" used if type was "after" or "before", ie...
{noformat}
f.price.facet.sequence.spec=*,0,10,+10,50,+50,*
f.created.facet.sequence.spec=NOW,-1DAY,-1MONTH,-1YEAR
facet.sequence.type=after
Price: below $0, $0-10, $0-20, $0-50, $0-100, $100 and up
Created: NOW-1YEAR TO NOW, NOW-1YEAR TO NOW-1DAY, NOW-1YEAR TO NOW-1MONTH
facet.sequence.type=between
Price: below $0, $0-10, $10-20, $20-50, $50-100, $100 and up
Created: NOW-1DAY TO NOW, NOW-1MONTH TO NOW-1DAY, NOW-1YEAR TO NOW-1MONTH
facet.sequence.type=before
Price: below $0, $0-100, $10-100, $20-100, $50-100, $100 and up
Created: NOW-1DAY TO NOW, NOW-1MONTH TO NOW, NOW-1YEAR TO NOW
{noformat}
...if we defined things that way, i *think* that would simplify a lot of the complexity we've been talking about, and simplify some of the use cases.
the only remaining issues that have been brought up (that i can think of) that would still need to be work out would be:
1) what the response format needs to look like - I'd vote to punt on this until we figure out the semantics.
2) when exactly ranges are inclusive/exclusive of their endpoints - i *think* we should be able reuse the semantics from facet.range.include here, including "edge", if we define ranges involving "*" as "outer" ranges, but we'd need to work through more scenarios to be sure.
3) what happens if an increment overlaps with an absolute value, ie: my original example of "10,20,+50,+100,120,150". The three possible solutions I can think of are:
* fail loudly
* implement "precedence" rules, ie: that absolute values trump relative values (10-20,20-70,70-120,120-150) or vice-versa (10-20,20-70,70-170)
* implement precedence rules but let them be controlled via a request param (similar to how "facet.range.hardend" works)
---
What do you think? Are there any key use cases / features we've talked about that you think this approach overlooks? Do you still think it should really be an extension to "facet.range" ?
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.5, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets.
> (Original syntax proposal removed, see discussion for concrete syntax)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Herman J Kiefus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000293#comment-13000293 ]
Herman J Kiefus commented on SOLR-2366:
---------------------------------------
Also regarding arbitrary ranges:
While using fact.query allows us to construct arbitrary ranges, we must then pick them out of the results separately. This becomes more difficult if we arbitrarily facet on two or more fields/expressions. Essentially we have to parse the results, grouping by expression and then picking out each range in the order we want to illustrate it. This would seem to be unnecessary, if we had the ability to add n absolute ranges to a facet.range.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (SOLR-2366) Facet Range Gaps
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995290#comment-12995290 ]
Grant Ingersoll commented on SOLR-2366:
---------------------------------------
Note, also, the proposed syntax is just a draft, I'm definitely open to other syntax
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (SOLR-2366) Facet Range Gaps
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated SOLR-2366:
----------------------------------
Attachment: SOLR-2366.patch
Added more tests, cleaned up the patch, all tests pass. I think it is ready to commit and will do so in a day or two or maybe this weekend.
> Facet Range Gaps
> ----------------
>
> Key: SOLR-2366
> URL: https://issues.apache.org/jira/browse/SOLR-2366
> Project: Solr
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2366.patch, SOLR-2366.patch
>
>
> There really is no reason why the range gap for date and numeric faceting needs to be evenly spaced. For instance, if and when SOLR-1581 is completed and one were doing spatial distance calculations, one could facet by function into 3 different sized buckets: walking distance (0-5KM), driving distance (5KM-150KM) and everything else (150KM+), for instance. We should be able to quantize the results into arbitrarily sized buckets. I'd propose the syntax to be a comma separated list of sizes for each bucket. If only one value is specified, then it behaves as it currently does. Otherwise, it creates the different size buckets. If the number of buckets doesn't evenly divide up the space, then the size of the last bucket specified is used to fill out the remaining space (not sure on this)
> For instance,
> facet.range.start=0
> facet.range.end=400
> facet.range.gap=5,25,50,100
> would yield buckets of:
> 0-5,5-30,30-80,80-180,180-280,280-380,380-400
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org