You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Dawid Weiss <da...@cs.put.poznan.pl> on 2005/06/01 10:24:07 UTC
Re: Clustering Carrot2 vs TermVector Analysis
Hi Andrew,
Coming up with an answer... sorry for the delay.
> By using the carrot demo:
> http://www.newsarch.com/archive/mailinglist/jakarta/lucene/user/msg03928.html
>
>
> I was able to easliy cluster search results based on the fields used
> by carrot( url, title, and summary). However I was wondering if there
> was a way to do something similar using term vector analysis and the
> built in TermVector / Similarity api.
Yes, most clustering methods are based just on that (term-vector
matrix). Carrot also uses this internally, but builds its own data
structure from the provided data instead of relying on Lucene's. It
shouldn't be a problem to write a clustering plugin to Carrot that
actually uses the term-vector data from Lucene.
> After doing a typical lucene search how can I get the top 5 "key
> terms" for each of the top ten documents. I was thinking that I sum
> these and then have a type of cluster.
The question is ill-defined, I'm afraid. "top 5 key terms" are very
subjecting and depend on the strategy of score calculation, the way
you're pruning stop words, etc.
I also don't get the: "each of the top ten documents". Do you mean: each
of the ten top documents within a cluster?
D.
P.S. Please CC me directly; I read mails to newsgroups in batches every
few days.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org