You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ravish Bhagdev <ra...@gmail.com> on 2011/02/27 13:57:41 UTC
Getting sum of all terms count in dataset instead of document count
using TermsComponent....(and TermsComponent vs Facets)
Hi Guys,
I need a bit of help.
I want to produce frequency analysis of all tokens inside my solr Index from
a specific (content) field.
When I use TermsComponent or FacetCounts, what I get is how many records or
documents each term appears in (which again confuses me as to what the
difference is, is it facets are restricted to terms in result set and
termscomponent is not restricted by the query?). Is there yet a way to get
total terms count (not per document but across the whole index)? I have
tried searching in archieves and across web but closest match I found is
this: http://search-lucene.com/m/of5Fn1PUOHU/
It is suggested in this post that I can post the mentioned lines of code
into TermsComponent.java and it should work. However, the code seems to
have changed since and when I try this, the Class TermDocs is not even
recognized.
I was wondering if there is any other way by using Lucene or Solr to do
this. I will be very grateful for any reply. If it helps, below is the
code I am running right now which gives me document count and not Terms
count.
String queryString = "document:*";
SolrQuery solrQuery = new SolrQuery();
solrQuery.setQuery(queryString);
solrQuery.setQueryType("/terms");
solrQuery.setTerms(true);
solrQuery.setTermsLimit(20);
solrQuery.setParam("terms.fl", "document");
solrQuery.setTermsSortString("count");
QueryResponse solrResp = conf._solr.executeQuery(solrQuery, 0, 10);
TermsResponse termsResp = solrResp.getTermsResponse();
List<TermsResponse.Term> terms = termsResp.getTerms("document");
Ignore the conf object and _solr variable thats just my internal singleton
object.
Thanks,
Ravish Bhagdev
Re: Getting sum of all terms count in dataset instead of document count using TermsComponent....(and TermsComponent vs Facets)
Posted by Ahmet Arslan <io...@yahoo.com>.
> from head, but also tried on
> > 1.4.1.
>
>
> The instructions i posted works with 1.4.1. You need to
> import TermDocs in TermsComponent.java :
>
> import org.apache.lucene.index.TermDocs;
Ravish, did you get it working? Can you give us feedback?
Re: Getting sum of all terms count in dataset instead of document count using TermsComponent....(and TermsComponent vs Facets)
Posted by Ahmet Arslan <io...@yahoo.com>.
> I've compiled current source from head, but also tried on
> 1.4.1.
The instructions i posted works with 1.4.1. You need to import TermDocs in TermsComponent.java :
import org.apache.lucene.index.TermDocs;
Re: Getting sum of all terms count in dataset instead of document
count using TermsComponent....(and TermsComponent vs Facets)
Posted by Ravish Bhagdev <ra...@gmail.com>.
Yes, you are right. Ignore the query (document:*), it wont matter if i have
it for termscomponent i guess.
I've compiled current source from head, but also tried on 1.4.1.
Any idea how to go about finding a solution to this?
Thanks,
Ravish
On Sun, Feb 27, 2011 at 1:56 PM, Ahmet Arslan <io...@yahoo.com> wrote:
> > I want to produce frequency analysis of all tokens inside
> > my solr Index from
> > a specific (content) field.
> >
> > When I use TermsComponent or FacetCounts, what I get is how
> > many records or
> > documents each term appears in (which again confuses me as
> > to what the
> > difference is, is it facets are restricted to terms in
> > result set and
> > termscomponent is not restricted by the query?). Is
> > there yet a way to get
> > total terms count (not per document but across the whole
> > index)?
>
> Terms Component does not respect &q= parameter. In other words, it is not
> restricted by the query.
>
> > I have
> > tried searching in archieves and across web but closest
> > match I found is
> > this: http://search-lucene.com/m/of5Fn1PUOHU/
> >
> > It is suggested in this post that I can post the mentioned
> > lines of code
> > into TermsComponent.java and it should work. However,
> > the code seems to
> > have changed since and when I try this, the Class TermDocs
> > is not even
> > recognized.
>
> What version of solr are you using?
>
>
>
>
>
Re: Getting sum of all terms count in dataset instead of document count using TermsComponent....(and TermsComponent vs Facets)
Posted by Ahmet Arslan <io...@yahoo.com>.
> I want to produce frequency analysis of all tokens inside
> my solr Index from
> a specific (content) field.
>
> When I use TermsComponent or FacetCounts, what I get is how
> many records or
> documents each term appears in (which again confuses me as
> to what the
> difference is, is it facets are restricted to terms in
> result set and
> termscomponent is not restricted by the query?). Is
> there yet a way to get
> total terms count (not per document but across the whole
> index)?
Terms Component does not respect &q= parameter. In other words, it is not restricted by the query.
> I have
> tried searching in archieves and across web but closest
> match I found is
> this: http://search-lucene.com/m/of5Fn1PUOHU/
>
> It is suggested in this post that I can post the mentioned
> lines of code
> into TermsComponent.java and it should work. However,
> the code seems to
> have changed since and when I try this, the Class TermDocs
> is not even
> recognized.
What version of solr are you using?