You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by levanDev <le...@gmail.com> on 2015/04/16 13:46:31 UTC
5.1 'unique' facet function / calcDistinct
Hello,
We are looking at a couple of options for using solr to dynamically calulate
unique values per field. In testing out Solr 5.1, I've been using the
unique() facet function:
http://yonik.com/solr-facet-functions/
Overall, loving the JSON Facet API, especially the sub-faceting thus far.
Here's my two part question:
I. When I use the unique aggregation function on a "string" field
(uniqueValues:'unique(myStringField)'), it works as expected, returns the
number of unique fields. However when I pass in an int -- or date -- field
(uniqueValues:'unique(myIntField)') the resulting count is 0. The cause
might be something else, but if it can be replicated by another user, would
be great to discuss the unique function further -- in our current use-case,
we have a field where under 20 unique values are present but the values are
ints.
II. Is there a way to use the stats.calcdistinct functionality and only
return the countDistinct portion of the response and not the full list of
distinct values -- as provided in the distinctValues portion of the
response. In a field with high cardinality the response size becomes too
large.
If there is no such option, could someone point me in the right direction
for implementing a custom solution?
Thank you for your time,
Levan
--
View this message in context: http://lucene.472066.n3.nabble.com/5-1-unique-facet-function-calcDistinct-tp4200110.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: 5.1 'unique' facet function / calcDistinct
Posted by levanDev <le...@gmail.com>.
Perfect, thank you for the information -- will have a look through those
classes.
Thank you,
Levan
--
View this message in context: http://lucene.472066.n3.nabble.com/5-1-unique-facet-function-calcDistinct-tp4200110p4200535.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: 5.1 'unique' facet function / calcDistinct
Posted by Tomás Fernández Löbbe <to...@gmail.com>.
>
> II. Is there a way to use the stats.calcdistinct functionality and only
> return the countDistinct portion of the response and not the full list of
> distinct values -- as provided in the distinctValues portion of the
> response. In a field with high cardinality the response size becomes too
> large.
>
I don't think this is currently supported.
>
> If there is no such option, could someone point me in the right direction
> for implementing a custom solution?
>
The problem is how to calculate this in distributed requests. Even if the
final response doesn't include the distinct values, the shard responses
will probably have to.
Look at StatsComponent.java and AbstractStatsValues in
StatsValuesFactory.java
Tomás
>
> Thank you for your time,
> Levan
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/5-1-unique-facet-function-calcDistinct-tp4200110.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: 5.1 'unique' facet function / calcDistinct
Posted by levanDev <le...@gmail.com>.
I've posted the issue here, please let me know if any additional information
needs to be provided.
https://issues.apache.org/jira/browse/SOLR-7417
Happy to provide the feedback, using the sub-facets has been a lot of fun,
the nested facet query is especially useful.
--
View this message in context: http://lucene.472066.n3.nabble.com/5-1-unique-facet-function-calcDistinct-tp4200110p4200534.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: 5.1 'unique' facet function / calcDistinct
Posted by Yonik Seeley <ys...@gmail.com>.
Thanks for the feedback Levan!
Could you open a JIRA issue for unique() on numeric/date fields?
We don't yet have explicit numeric support for unique() and I think
some changes in Lucene 5 broke treating these fields as strings (i.e.
the ability to retrieve ords).
-Yonik
On Thu, Apr 16, 2015 at 7:46 AM, levanDev <le...@gmail.com> wrote:
> Hello,
>
> We are looking at a couple of options for using solr to dynamically calulate
> unique values per field. In testing out Solr 5.1, I've been using the
> unique() facet function:
>
> http://yonik.com/solr-facet-functions/
>
> Overall, loving the JSON Facet API, especially the sub-faceting thus far.
>
> Here's my two part question:
>
> I. When I use the unique aggregation function on a "string" field
> (uniqueValues:'unique(myStringField)'), it works as expected, returns the
> number of unique fields. However when I pass in an int -- or date -- field
> (uniqueValues:'unique(myIntField)') the resulting count is 0. The cause
> might be something else, but if it can be replicated by another user, would
> be great to discuss the unique function further -- in our current use-case,
> we have a field where under 20 unique values are present but the values are
> ints.
>
> II. Is there a way to use the stats.calcdistinct functionality and only
> return the countDistinct portion of the response and not the full list of
> distinct values -- as provided in the distinctValues portion of the
> response. In a field with high cardinality the response size becomes too
> large.
>
> If there is no such option, could someone point me in the right direction
> for implementing a custom solution?
>
> Thank you for your time,
> Levan