You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Aaron Bycoffe <ab...@sunlightfoundation.com> on 2011/01/12 15:08:23 UTC

Term frequency across multiple documents

I'm attempting to calculate term frequency across multiple documents
in Solr. I've been able to use TermVectorComponent to get this data on
a per-document basis but have been unable to find a way to do it for
multiple documents -- that is, get a list of terms appearing in the
documents and how many times each one appears. I'd also like to be
able to filter the list of terms to be able to see how many times a
specific term appears, though this is less important.

Is there a way to do this in Solr?


Aaron

Re: Term frequency across multiple documents

Posted by Ahmet Arslan <io...@yahoo.com>.
So you are interested in collection frequency of words.

TermsComponent gives you document frequency of terms. You can modify it to give collection frequency info. http://search-lucene.com/m/of5Fn1PUOHU/

--- On Wed, 1/12/11, Juan Grande <ju...@gmail.com> wrote:

> From: Juan Grande <ju...@gmail.com>
> Subject: Re: Term frequency across multiple documents
> To: solr-user@lucene.apache.org
> Date: Wednesday, January 12, 2011, 6:56 PM
> Maybe there is a better solution, but
> I think that you can solve this
> problem using facets. You will get the number of documents
> where each term
> appears. Also, you can filter a specific set of terms by
> entering a query
> like "+field:term1 OR +field:term2 OR ...", or using the
> facet.query
> parameter.
> 
> Regards,
> 
> Juan Grande
> 
> On Wed, Jan 12, 2011 at 11:08 AM, Aaron Bycoffe <
> abycoffe@sunlightfoundation.com>
> wrote:
> 
> > I'm attempting to calculate term frequency across
> multiple documents
> > in Solr. I've been able to use TermVectorComponent to
> get this data on
> > a per-document basis but have been unable to find a
> way to do it for
> > multiple documents -- that is, get a list of terms
> appearing in the
> > documents and how many times each one appears. I'd
> also like to be
> > able to filter the list of terms to be able to see how
> many times a
> > specific term appears, though this is less important.
> >
> > Is there a way to do this in Solr?
> >
> >
> > Aaron
> >
> 


      

Re: Term frequency across multiple documents

Posted by Juan Grande <ju...@gmail.com>.
Maybe there is a better solution, but I think that you can solve this
problem using facets. You will get the number of documents where each term
appears. Also, you can filter a specific set of terms by entering a query
like "+field:term1 OR +field:term2 OR ...", or using the facet.query
parameter.

Regards,

Juan Grande

On Wed, Jan 12, 2011 at 11:08 AM, Aaron Bycoffe <
abycoffe@sunlightfoundation.com> wrote:

> I'm attempting to calculate term frequency across multiple documents
> in Solr. I've been able to use TermVectorComponent to get this data on
> a per-document basis but have been unable to find a way to do it for
> multiple documents -- that is, get a list of terms appearing in the
> documents and how many times each one appears. I'd also like to be
> able to filter the list of terms to be able to see how many times a
> specific term appears, though this is less important.
>
> Is there a way to do this in Solr?
>
>
> Aaron
>