You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Scott Smith <ss...@mainstreamdata.com> on 2017/03/14 01:08:01 UTC
Facet? Search problem
I'm trying to solve a search problem and wondering if facets (or something else) might solve the problem.
Let's assume I have a bunch of documents (100 million+). Each document has a category (keyword) assigned to it. A single document my only have one category, but there may be multiple documents with the same category (1 to a few hundred documents may be in any one category). There are several million categories.
Supposed I'm doing a search with a page size of 50. What I want to do is do a search (e.g., "dog") and get back the top 50 documents that match the contain the word "dog" and are all in different categories. So, there needs to be one document from 50 different categories.
If that's not possible, then is it possible to do it if I know the 50 categories up-front and hand that off as part of the search (so "find 50 documents that match the term 'dog' and there is one document from each of 50 specified categories").
Is there a way to do this?
I'm not extremely knowledgeable about facets, but thought that might be a solution. But, it doesn't have to be facets.
Thanks for any help
Scott
Re: Facet? Search problem
Posted by Dave <ha...@gmail.com>.
https://wiki.apache.org/solr/FieldCollapsing
> On Mar 13, 2017, at 9:59 PM, Dave <ha...@gmail.com> wrote:
>
> Perhaps look into grouping on that field.
>
>> On Mar 13, 2017, at 9:08 PM, Scott Smith <ss...@mainstreamdata.com> wrote:
>>
>> I'm trying to solve a search problem and wondering if facets (or something else) might solve the problem.
>>
>> Let's assume I have a bunch of documents (100 million+). Each document has a category (keyword) assigned to it. A single document my only have one category, but there may be multiple documents with the same category (1 to a few hundred documents may be in any one category). There are several million categories.
>>
>> Supposed I'm doing a search with a page size of 50. What I want to do is do a search (e.g., "dog") and get back the top 50 documents that match the contain the word "dog" and are all in different categories. So, there needs to be one document from 50 different categories.
>>
>> If that's not possible, then is it possible to do it if I know the 50 categories up-front and hand that off as part of the search (so "find 50 documents that match the term 'dog' and there is one document from each of 50 specified categories").
>>
>> Is there a way to do this?
>>
>> I'm not extremely knowledgeable about facets, but thought that might be a solution. But, it doesn't have to be facets.
>>
>> Thanks for any help
>>
>> Scott
>>
>>
RE: Facet? Search problem
Posted by Scott Smith <ss...@mainstreamdata.com>.
Thanks. I'll look at that as well.
-----Original Message-----
From: Stefan Matheis [mailto:matheis.stefan@gmail.com]
Sent: Tuesday, March 14, 2017 1:20 PM
To: solr-user@lucene.apache.org
Subject: RE: Facet? Search problem
Scott
Depending on what you're looking for
https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results
might be worth a look as well.
-Stefan
On Mar 14, 2017 7:25 PM, "Scott Smith" <ss...@mainstreamdata.com> wrote:
> Grouping appears to be exactly what I'm looking for. I added
> "group=true&group.field=category" to my search and It appears that I
> get a list of groups, one document in each group that matches the
> search along with (bonus) the number of documents in the category that
> match that search. Perfect. Thank you very much.
>
> -----Original Message-----
> From: Dave [mailto:hastings.recursive@gmail.com]
> Sent: Monday, March 13, 2017 7:59 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Facet? Search problem
>
> Perhaps look into grouping on that field.
>
> > On Mar 13, 2017, at 9:08 PM, Scott Smith <ss...@mainstreamdata.com>
> wrote:
> >
> > I'm trying to solve a search problem and wondering if facets (or
> something else) might solve the problem.
> >
> > Let's assume I have a bunch of documents (100 million+). Each
> > document
> has a category (keyword) assigned to it. A single document my only
> have one category, but there may be multiple documents with the same
> category (1 to a few hundred documents may be in any one category).
> There are several million categories.
> >
> > Supposed I'm doing a search with a page size of 50. What I want to
> > do
> is do a search (e.g., "dog") and get back the top 50 documents that
> match the contain the word "dog" and are all in different categories.
> So, there needs to be one document from 50 different categories.
> >
> > If that's not possible, then is it possible to do it if I know the
> > 50
> categories up-front and hand that off as part of the search (so "find
> 50 documents that match the term 'dog' and there is one document from
> each of
> 50 specified categories").
> >
> > Is there a way to do this?
> >
> > I'm not extremely knowledgeable about facets, but thought that might
> > be
> a solution. But, it doesn't have to be facets.
> >
> > Thanks for any help
> >
> > Scott
> >
> >
>
RE: Facet? Search problem
Posted by Stefan Matheis <ma...@gmail.com>.
Scott
Depending on what you're looking for
https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results
might be worth a look as well.
-Stefan
On Mar 14, 2017 7:25 PM, "Scott Smith" <ss...@mainstreamdata.com> wrote:
> Grouping appears to be exactly what I'm looking for. I added
> "group=true&group.field=category" to my search and It appears that I get
> a list of groups, one document in each group that matches the search along
> with (bonus) the number of documents in the category that match that
> search. Perfect. Thank you very much.
>
> -----Original Message-----
> From: Dave [mailto:hastings.recursive@gmail.com]
> Sent: Monday, March 13, 2017 7:59 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Facet? Search problem
>
> Perhaps look into grouping on that field.
>
> > On Mar 13, 2017, at 9:08 PM, Scott Smith <ss...@mainstreamdata.com>
> wrote:
> >
> > I'm trying to solve a search problem and wondering if facets (or
> something else) might solve the problem.
> >
> > Let's assume I have a bunch of documents (100 million+). Each document
> has a category (keyword) assigned to it. A single document my only have
> one category, but there may be multiple documents with the same category (1
> to a few hundred documents may be in any one category). There are several
> million categories.
> >
> > Supposed I'm doing a search with a page size of 50. What I want to do
> is do a search (e.g., "dog") and get back the top 50 documents that match
> the contain the word "dog" and are all in different categories. So, there
> needs to be one document from 50 different categories.
> >
> > If that's not possible, then is it possible to do it if I know the 50
> categories up-front and hand that off as part of the search (so "find 50
> documents that match the term 'dog' and there is one document from each of
> 50 specified categories").
> >
> > Is there a way to do this?
> >
> > I'm not extremely knowledgeable about facets, but thought that might be
> a solution. But, it doesn't have to be facets.
> >
> > Thanks for any help
> >
> > Scott
> >
> >
>
Re: Facet? Search problem
Posted by David Hastings <ha...@gmail.com>.
glad it worked for you. im planning on some experimentation using that
feature, could contribute to an interface nicely if thought through well.
On Tue, Mar 14, 2017 at 2:25 PM, Scott Smith <ss...@mainstreamdata.com>
wrote:
> Grouping appears to be exactly what I'm looking for. I added
> "group=true&group.field=category" to my search and It appears that I get
> a list of groups, one document in each group that matches the search along
> with (bonus) the number of documents in the category that match that
> search. Perfect. Thank you very much.
>
> -----Original Message-----
> From: Dave [mailto:hastings.recursive@gmail.com]
> Sent: Monday, March 13, 2017 7:59 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Facet? Search problem
>
> Perhaps look into grouping on that field.
>
> > On Mar 13, 2017, at 9:08 PM, Scott Smith <ss...@mainstreamdata.com>
> wrote:
> >
> > I'm trying to solve a search problem and wondering if facets (or
> something else) might solve the problem.
> >
> > Let's assume I have a bunch of documents (100 million+). Each document
> has a category (keyword) assigned to it. A single document my only have
> one category, but there may be multiple documents with the same category (1
> to a few hundred documents may be in any one category). There are several
> million categories.
> >
> > Supposed I'm doing a search with a page size of 50. What I want to do
> is do a search (e.g., "dog") and get back the top 50 documents that match
> the contain the word "dog" and are all in different categories. So, there
> needs to be one document from 50 different categories.
> >
> > If that's not possible, then is it possible to do it if I know the 50
> categories up-front and hand that off as part of the search (so "find 50
> documents that match the term 'dog' and there is one document from each of
> 50 specified categories").
> >
> > Is there a way to do this?
> >
> > I'm not extremely knowledgeable about facets, but thought that might be
> a solution. But, it doesn't have to be facets.
> >
> > Thanks for any help
> >
> > Scott
> >
> >
>
RE: Facet? Search problem
Posted by Scott Smith <ss...@mainstreamdata.com>.
Grouping appears to be exactly what I'm looking for. I added "group=true&group.field=category" to my search and It appears that I get a list of groups, one document in each group that matches the search along with (bonus) the number of documents in the category that match that search. Perfect. Thank you very much.
-----Original Message-----
From: Dave [mailto:hastings.recursive@gmail.com]
Sent: Monday, March 13, 2017 7:59 PM
To: solr-user@lucene.apache.org
Subject: Re: Facet? Search problem
Perhaps look into grouping on that field.
> On Mar 13, 2017, at 9:08 PM, Scott Smith <ss...@mainstreamdata.com> wrote:
>
> I'm trying to solve a search problem and wondering if facets (or something else) might solve the problem.
>
> Let's assume I have a bunch of documents (100 million+). Each document has a category (keyword) assigned to it. A single document my only have one category, but there may be multiple documents with the same category (1 to a few hundred documents may be in any one category). There are several million categories.
>
> Supposed I'm doing a search with a page size of 50. What I want to do is do a search (e.g., "dog") and get back the top 50 documents that match the contain the word "dog" and are all in different categories. So, there needs to be one document from 50 different categories.
>
> If that's not possible, then is it possible to do it if I know the 50 categories up-front and hand that off as part of the search (so "find 50 documents that match the term 'dog' and there is one document from each of 50 specified categories").
>
> Is there a way to do this?
>
> I'm not extremely knowledgeable about facets, but thought that might be a solution. But, it doesn't have to be facets.
>
> Thanks for any help
>
> Scott
>
>
Re: Facet? Search problem
Posted by Dave <ha...@gmail.com>.
Perhaps look into grouping on that field.
> On Mar 13, 2017, at 9:08 PM, Scott Smith <ss...@mainstreamdata.com> wrote:
>
> I'm trying to solve a search problem and wondering if facets (or something else) might solve the problem.
>
> Let's assume I have a bunch of documents (100 million+). Each document has a category (keyword) assigned to it. A single document my only have one category, but there may be multiple documents with the same category (1 to a few hundred documents may be in any one category). There are several million categories.
>
> Supposed I'm doing a search with a page size of 50. What I want to do is do a search (e.g., "dog") and get back the top 50 documents that match the contain the word "dog" and are all in different categories. So, there needs to be one document from 50 different categories.
>
> If that's not possible, then is it possible to do it if I know the 50 categories up-front and hand that off as part of the search (so "find 50 documents that match the term 'dog' and there is one document from each of 50 specified categories").
>
> Is there a way to do this?
>
> I'm not extremely knowledgeable about facets, but thought that might be a solution. But, it doesn't have to be facets.
>
> Thanks for any help
>
> Scott
>
>