sorting on a field that can have null values

[Praveen Peddi <pp...@contextmedia.com>]

Hi all,
I am getting null pointer exception when I am sorting on a field that has null value for some documents. "Order by" in sql does work on such fields and I think it puts all results with null values at the end of the list. Shouldn't lucene also do the same thing instead of throwing null pointer exception. Is this an expected behaviour? Is lucene always expecting some value on the sortable fields?

I thought of putting empty strings instead of null values but I think empty strings are put first in the list while sorting which is the reverse of what anyone would want. 

Following is the exception I saw in the error log:

java.lang.NullPointerException
 at org.apache.lucene.search.SortComparator$1.compare(Lorg.apache.lucene.search.ScoreDoc;Lorg.apache.lucene.search.ScoreDoc;)I(SortComparator.java:36)
 at org.apache.lucene.search.FieldSortedHitQueue.lessThan(Ljava.lang.Object;Ljava.lang.Object;)Z(FieldSortedHitQueue.java:95)
 at org.apache.lucene.util.PriorityQueue.upHeap()V(PriorityQueue.java:120)
 at org.apache.lucene.util.PriorityQueue.put(Ljava.lang.Object;)V(PriorityQueue.java:47)
 at org.apache.lucene.util.PriorityQueue.insert(Ljava.lang.Object;)Z(PriorityQueue.java:58)
 at org.apache.lucene.search.IndexSearcher$2.collect(IF)V(IndexSearcher.java:130)
 at org.apache.lucene.search.Scorer.score(Lorg.apache.lucene.search.HitCollector;)V(Scorer.java:38)
 at org.apache.lucene.search.IndexSearcher.search(Lorg.apache.lucene.search.Query;Lorg.apache.lucene.search.Filter;ILorg.apache.lucene.search.Sort;)Lorg.apache.lucene.search.TopFieldDocs;(IndexSearcher.java:125)
 at org.apache.lucene.search.Hits.getMoreDocs(I)V(Hits.java:64)
 at org.apache.lucene.search.Hits.<init>(Lorg.apache.lucene.search.Searcher;Lorg.apache.lucene.search.Query;Lorg.apache.lucene.search.Filter;Lorg.apache.lucene.search.Sort;)V(Hits.java:51)
 at org.apache.lucene.search.Searcher.search(Lorg.apache.lucene.search.Query;Lorg.apache.lucene.search.Sort;)Lorg.apache.lucene.search.Hits;(Searcher.java:41)

If its a bug in lucene, Will it be fixed in next release? Any suggestions would be appreciated.

Praveen

************************************************************** 
Praveen Peddi
Sr Software Engg, Context Media, Inc. 
email:ppeddi@contextmedia.com 
Tel:  401.854.3475 
Fax:  401.861.3596 
web: http://www.contextmedia.com 
************************************************************** 
Context Media- "The Leader in Enterprise Content Integration"