You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Erin Colvin <es...@yahoo.com.INVALID> on 2014/07/28 02:49:22 UTC
Lucene Help Needed for Doctoral Student
Hi, I am working on my doctoral dissertation in CS and am trying to use Lucene to do custom similarity measures, namely MMM (mixed, min and Max), Paice and p-norm and then compare those results to the traditional Boolean similarity and TF/IDF similarity.
So far I have tried creating custom similarity, and scoring with no luck and right now I am able to pull the IDF value out of a search result but cannot for the life of me get the term frequency.
Is there any way to create a custom similarity measure and run it for results or at least can you help me pull the term frequencies out so I can do the calculations myself?
I am a C++ and VB programmer who hasn't programmed in years so my java is a bit juvenile but I am 6 months away from being down with this degree and really want to graduate with my class so I am in need of help ASAP!!!!
I'm using Eclipse with Lucene 4.0 so I can check my results in Luke.
Please help,
Erin Colvin
Erin Colvin
Computer Science Doctoral Student
Colorado Technical University
BS, MS, M.Ed
Re: Lucene Help Needed for Doctoral Student
Posted by "Yalamarthi, Vineel" <vy...@ebay.com>.
Hey Guys,
Can you tell me how to find TF-IDF using lucene ?
Regards
Vineel
On 7/28/14, 3:11 PM, "Prakash Dubey" <pk...@gmail.com> wrote:
>In Lucene 2.4.0 there is *org.apache.lucene.index.TermFreqVector* method
>*getTermFrequencies()* method. You can use Apache Maths for other
>mathematical operations. Also see following Blog
><http://sujitpal.blogspot.in/2011/10/computing-document-similarity-using.h
>tml>
>for
>more information.
>Hope this helps..
>
>
>On Mon, Jul 28, 2014 at 6:19 AM, Erin Colvin
><es...@yahoo.com.invalid>
>wrote:
>
>> Hi, I am working on my doctoral dissertation in CS and am trying to use
>> Lucene to do custom similarity measures, namely MMM (mixed, min and
>>Max),
>> Paice and p-norm and then compare those results to the traditional
>>Boolean
>> similarity and TF/IDF similarity.
>> So far I have tried creating custom similarity, and scoring with no luck
>> and right now I am able to pull the IDF value out of a search result but
>> cannot for the life of me get the term frequency.
>> Is there any way to create a custom similarity measure and run it for
>> results or at least can you help me pull the term frequencies out so I
>>can
>> do the calculations myself?
>> I am a C++ and VB programmer who hasn't programmed in years so my java
>>is
>> a bit juvenile but I am 6 months away from being down with this degree
>>and
>> really want to graduate with my class so I am in need of help ASAP!!!!
>> I'm using Eclipse with Lucene 4.0 so I can check my results in Luke.
>> Please help,
>> Erin Colvin
>>
>>
>> Erin Colvin
>> Computer Science Doctoral Student
>> Colorado Technical University
>> BS, MS, M.Ed
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Lucene Help Needed for Doctoral Student
Posted by Prakash Dubey <pk...@gmail.com>.
In Lucene 2.4.0 there is *org.apache.lucene.index.TermFreqVector* method
*getTermFrequencies()* method. You can use Apache Maths for other
mathematical operations. Also see following Blog
<http://sujitpal.blogspot.in/2011/10/computing-document-similarity-using.html>
for
more information.
Hope this helps..
On Mon, Jul 28, 2014 at 6:19 AM, Erin Colvin <es...@yahoo.com.invalid>
wrote:
> Hi, I am working on my doctoral dissertation in CS and am trying to use
> Lucene to do custom similarity measures, namely MMM (mixed, min and Max),
> Paice and p-norm and then compare those results to the traditional Boolean
> similarity and TF/IDF similarity.
> So far I have tried creating custom similarity, and scoring with no luck
> and right now I am able to pull the IDF value out of a search result but
> cannot for the life of me get the term frequency.
> Is there any way to create a custom similarity measure and run it for
> results or at least can you help me pull the term frequencies out so I can
> do the calculations myself?
> I am a C++ and VB programmer who hasn't programmed in years so my java is
> a bit juvenile but I am 6 months away from being down with this degree and
> really want to graduate with my class so I am in need of help ASAP!!!!
> I'm using Eclipse with Lucene 4.0 so I can check my results in Luke.
> Please help,
> Erin Colvin
>
>
> Erin Colvin
> Computer Science Doctoral Student
> Colorado Technical University
> BS, MS, M.Ed
>
RE: Lucene Help Needed for Doctoral Student
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,
Which Lucene version are you using? Lucene 4.x allows to do non-TF-IDF scoring; previous versions did not.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Erin Colvin [mailto:eschlapkohl@yahoo.com.INVALID]
> Sent: Monday, July 28, 2014 2:49 AM
> To: java-user@lucene.apache.org
> Subject: Lucene Help Needed for Doctoral Student
>
> Hi, I am working on my doctoral dissertation in CS and am trying to use
> Lucene to do custom similarity measures, namely MMM (mixed, min and
> Max), Paice and p-norm and then compare those results to the traditional
> Boolean similarity and TF/IDF similarity.
> So far I have tried creating custom similarity, and scoring with no luck and
> right now I am able to pull the IDF value out of a search result but cannot for
> the life of me get the term frequency.
> Is there any way to create a custom similarity measure and run it for results
> or at least can you help me pull the term frequencies out so I can do the
> calculations myself?
> I am a C++ and VB programmer who hasn't programmed in years so my java
> is a bit juvenile but I am 6 months away from being down with this degree
> and really want to graduate with my class so I am in need of help ASAP!!!!
> I'm using Eclipse with Lucene 4.0 so I can check my results in Luke.
> Please help,
> Erin Colvin
>
>
> Erin Colvin
> Computer Science Doctoral Student
> Colorado Technical University
> BS, MS, M.Ed
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org