Scoring without normalization!

[Jones G <jo...@rediffmail.com>]

How do I remove document normalization from scoring in Lucene? I just want to stick to TF IDF.

Thanks.

Re: ArrayIndexOutOfBoundsException if stopword on left of bool clause w/ StandardAnalyzer

[Morus Walter <mo...@tanto.de>]

Claude Devarenne writes:
> 
> My question is: should the queryParser catch that there is no term  
> before trying to add a clause when using a StandardAnalyzer?  Is this  
> even possible? Should the burden be on the application to either catch  
> the exception or parse the query before handing it out to the  
> queryParser?
> 
Yes. Yes. No.
There are fixes in bugzilla that would make query parser read that query
as title:bla and simply drop the stop word.

see http://issues.apache.org/bugzilla/show_bug.cgi?id=9110
http://issues.apache.org/bugzilla/show_bug.cgi?id=25820

Morus

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

ArrayIndexOutOfBoundsException if stopword on left of bool clause w/ StandardAnalyzer

[Claude Devarenne <cl...@library.ucsf.edu>]

Hi,

A user mistyped their search terms and entered a query that looked like  
this:

the AND title:bla

I am using lucene 1.4 rc3. My web app,  which is using a  
StandardAnalyzer, got an ArrayIndexOutOfBoundsException (stack trace  
below).  I can reproduce this with the lucene demo (both the jsp and  
the comand line util).

Since I have the queryParser.parse(queryString) call in a try statement  
I am now catching this exception so it fixes the issue.

My question is: should the queryParser catch that there is no term  
before trying to add a clause when using a StandardAnalyzer?  Is this  
even possible? Should the burden be on the application to either catch  
the exception or parse the query before handing it out to the  
queryParser?

Claude


Here is the stack trace:

java.lang.ArrayIndexOutOfBoundsException: -1 < 0
java.util.Vector.elementAt(Vector.java:437) at
org.apache.lucene.queryParser.QueryParser.addClause(QueryParser.java: 
181)
at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:509)
at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:108)
at QueryExec.runQuery(QueryExec.java:245)

RE: Scoring without normalization!

[Anson Lau <al...@fulfil-net.com>]

If you don't mind hacking the source:

In Hits.java

In method "getMoreDocs()"



    // Comment out the following
    //float scoreNorm = 1.0f;
    //if (length > 0 && scoreDocs[0].score > 1.0f) {
    //  scoreNorm = 1.0f / scoreDocs[0].score;
    //}
    
    // And just set scoreNorm to 1.
    int scoreNorm = 1;


I don't know if u can do it without going to the src.

Anson


-----Original Message-----
From: Jones G [mailto:jones.g@rediffmail.com] 
Sent: Thursday, July 15, 2004 6:52 AM
To: lucene-user@jakarta.apache.org
Subject: Scoring without normalization!

How do I remove document normalization from scoring in Lucene? I just want
to stick to TF IDF.

Thanks.


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org