You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Tony Mullins <to...@gmail.com> on 2013/07/02 12:24:19 UTC
Aggregate TermFrequency on Result Grouping / Field Collapsing
Hi,
Is it possible to perform aggregated termfreq(field,term) on Result
Grouping ?
I am trying to get total count of term's appearance in a document and then
want to aggregate that count by grouping the document on one of my field.
Like this
http://localhost:8080/solr/collection1/select?q=iphone&wt=json&indent=true&group=true&group.field=url&fl=freq%3Atermfreq%28CommentX%2C%27iphone%27%29
Problem is it returning only top level result (doc) in each group and thus
the term frequency of that result (doc).
How can I make it to sum the termfred() of all the documents per group ?
Thanks,
Tony
Re: Aggregate TermFrequency on Result Grouping / Field Collapsing
Posted by Erick Erickson <er...@gmail.com>.
Well, you've just restated the problem. I'm asking
what use-case this is supporting? You've said:
"he/she wants to know the occurrence of a
specific term in the result set of that 'X1' search criteria"
OK, _why_? Idle curiosity? Ranking the docs?
Choosing the most relevant? I don't think you can get
what you want OOB, so I'm wondering if you can
take another approach to support the top-level use-case.
Erick
On Wed, Jul 3, 2013 at 7:59 AM, Tony Mullins <to...@gmail.com>wrote:
> Ok let me explain the problem.
>
> I have lots of crawled data, indexed in my Solr (4.3.0) and lets say user
> creates a search criteria 'X1' and he/she wants to know the occurrence of a
> specific term in the result set of that 'X1' search criteria.
> And then again he/she creates another search criteria 'X2' and he/she wants
> to know the occurrence of that same term in the result set of that 'X2'
> search criteria.
>
> At the moment if I give termfreq(field,term) then it gives me the term
> frequency per document and if I use totaltermfreq(field,term), it gives me
> the total term frequency in entire index not in the result set of my search
> criteria.
>
> So what I need is your help to find how to how to get total occurrence of a
> term in query's result set.
>
> If this is my result set
>
> <doc>
> <str name="type">Movies</str>
> <str name="format">dvd</str>
> <str name="product">The Hunger Games</str></doc>
>
> <doc>
> <str name="type">Books</str>
> <str name="format">paperback</str>
> <str name="product">The Hunger Book</str></doc>
>
> And I am looking for term 'hunger' in product field then I want to get
> value = '2' , and if I am searching for term 'games' in product field I
> want to get value = '1' .
>
> Thanks,
> Tony
> **
>
>
> On Wed, Jul 3, 2013 at 4:41 PM, Erick Erickson <erickerickson@gmail.com
> >wrote:
>
> > I don't think you can, thus the silence. But why do you want
> > to do this thing? Smells like an XY problem, you've asked
> > how to do a specific thing without stating the problem. Perhaps
> > there's a better approach that _is_ do-able.
> >
> > Best
> > Erick
> >
> >
> > On Wed, Jul 3, 2013 at 2:14 AM, Tony Mullins <tonymullins.tm@gmail.com
> > >wrote:
> >
> > > Any suggestions please !
> > >
> > >
> > > On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins <tonymullins.tm@gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > Is it possible to perform aggregated termfreq(field,term) on Result
> > > > Grouping ?
> > > >
> > > > I am trying to get total count of term's appearance in a document and
> > > then
> > > > want to aggregate that count by grouping the document on one of my
> > field.
> > > >
> > > > Like this
> > > >
> > > >
> > > >
> > >
> >
> http://localhost:8080/solr/collection1/select?q=iphone&wt=json&indent=true&group=true&group.field=url&fl=freq%3Atermfreq%28CommentX%2C%27iphone%27%29
> > > >
> > > > Problem is it returning only top level result (doc) in each group and
> > > thus
> > > > the term frequency of that result (doc).
> > > >
> > > > How can I make it to sum the termfred() of all the documents per
> group
> > ?
> > > >
> > > > Thanks,
> > > > Tony
> > > >
> > >
> >
>
Re: Aggregate TermFrequency on Result Grouping / Field Collapsing
Posted by Tony Mullins <to...@gmail.com>.
Ok let me explain the problem.
I have lots of crawled data, indexed in my Solr (4.3.0) and lets say user
creates a search criteria 'X1' and he/she wants to know the occurrence of a
specific term in the result set of that 'X1' search criteria.
And then again he/she creates another search criteria 'X2' and he/she wants
to know the occurrence of that same term in the result set of that 'X2'
search criteria.
At the moment if I give termfreq(field,term) then it gives me the term
frequency per document and if I use totaltermfreq(field,term), it gives me
the total term frequency in entire index not in the result set of my search
criteria.
So what I need is your help to find how to how to get total occurrence of a
term in query's result set.
If this is my result set
<doc>
<str name="type">Movies</str>
<str name="format">dvd</str>
<str name="product">The Hunger Games</str></doc>
<doc>
<str name="type">Books</str>
<str name="format">paperback</str>
<str name="product">The Hunger Book</str></doc>
And I am looking for term 'hunger' in product field then I want to get
value = '2' , and if I am searching for term 'games' in product field I
want to get value = '1' .
Thanks,
Tony
**
On Wed, Jul 3, 2013 at 4:41 PM, Erick Erickson <er...@gmail.com>wrote:
> I don't think you can, thus the silence. But why do you want
> to do this thing? Smells like an XY problem, you've asked
> how to do a specific thing without stating the problem. Perhaps
> there's a better approach that _is_ do-able.
>
> Best
> Erick
>
>
> On Wed, Jul 3, 2013 at 2:14 AM, Tony Mullins <tonymullins.tm@gmail.com
> >wrote:
>
> > Any suggestions please !
> >
> >
> > On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins <tonymullins.tm@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > Is it possible to perform aggregated termfreq(field,term) on Result
> > > Grouping ?
> > >
> > > I am trying to get total count of term's appearance in a document and
> > then
> > > want to aggregate that count by grouping the document on one of my
> field.
> > >
> > > Like this
> > >
> > >
> > >
> >
> http://localhost:8080/solr/collection1/select?q=iphone&wt=json&indent=true&group=true&group.field=url&fl=freq%3Atermfreq%28CommentX%2C%27iphone%27%29
> > >
> > > Problem is it returning only top level result (doc) in each group and
> > thus
> > > the term frequency of that result (doc).
> > >
> > > How can I make it to sum the termfred() of all the documents per group
> ?
> > >
> > > Thanks,
> > > Tony
> > >
> >
>
Re: Aggregate TermFrequency on Result Grouping / Field Collapsing
Posted by Erick Erickson <er...@gmail.com>.
I don't think you can, thus the silence. But why do you want
to do this thing? Smells like an XY problem, you've asked
how to do a specific thing without stating the problem. Perhaps
there's a better approach that _is_ do-able.
Best
Erick
On Wed, Jul 3, 2013 at 2:14 AM, Tony Mullins <to...@gmail.com>wrote:
> Any suggestions please !
>
>
> On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins <tonymullins.tm@gmail.com
> >wrote:
>
> > Hi,
> >
> > Is it possible to perform aggregated termfreq(field,term) on Result
> > Grouping ?
> >
> > I am trying to get total count of term's appearance in a document and
> then
> > want to aggregate that count by grouping the document on one of my field.
> >
> > Like this
> >
> >
> >
> http://localhost:8080/solr/collection1/select?q=iphone&wt=json&indent=true&group=true&group.field=url&fl=freq%3Atermfreq%28CommentX%2C%27iphone%27%29
> >
> > Problem is it returning only top level result (doc) in each group and
> thus
> > the term frequency of that result (doc).
> >
> > How can I make it to sum the termfred() of all the documents per group ?
> >
> > Thanks,
> > Tony
> >
>
Re: Aggregate TermFrequency on Result Grouping / Field Collapsing
Posted by Tony Mullins <to...@gmail.com>.
Any suggestions please !
On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins <to...@gmail.com>wrote:
> Hi,
>
> Is it possible to perform aggregated termfreq(field,term) on Result
> Grouping ?
>
> I am trying to get total count of term's appearance in a document and then
> want to aggregate that count by grouping the document on one of my field.
>
> Like this
>
>
> http://localhost:8080/solr/collection1/select?q=iphone&wt=json&indent=true&group=true&group.field=url&fl=freq%3Atermfreq%28CommentX%2C%27iphone%27%29
>
> Problem is it returning only top level result (doc) in each group and thus
> the term frequency of that result (doc).
>
> How can I make it to sum the termfred() of all the documents per group ?
>
> Thanks,
> Tony
>