You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Reetha Hariharan <re...@gmail.com> on 2008/10/14 10:35:23 UTC

Newbie Question - Lucene Sorting NOT Ignoring NULL values

Hi,

I am a newbie.

I just configured lucene using hibernate search. But I find that the sorting
doesn't ignore null values.

I am searching using one field, say X and want to sort the results using
another, say Y (Which can have null values). But I am expecting Sort to
ignore all the null values and just sort only records that has values in Y
field.

How can I achieve this. Is there anywhere I can set the flag in SortField
that specifies it to ignore all the null values ?

Please help.

Thanks
Reetha.

Re: Newbie Question - Lucene Sorting NOT Ignoring NULL values

Posted by Reetha Hariharan <re...@gmail.com>.
Hi Yonik:

A simple RangeFilter did the trick

> public class NotNullRangeFilter extends org.apache.lucene.search.Filter {
>    private RangeFilter rangeFilter;
>    public BitSet bits(IndexReader reader) throws IOException {
>        rangeFilter = RangeFilter.More("lastUpdatedDate"/*Field to sort*/,
> "" /*atleast an empty string but not null*/);
>        return rangeFilter.bits(reader);
>    }
> }

the above code actually works perfectly.



On 10/14/08, Yonik Seeley <yo...@apache.org> wrote:
>
> You're on the right track I think... perhaps try using RangeFilter
> directly rather than creating your own class.  Something like:
>
> Filter filter = RangeFilter.More("lastUpdatedDate","");
> searcher.search(query, filter)
>
> If that works for you, then the next step would be to look at
> CachingWrapperFilter, which can help prevent calculating the filter
> for every query.
>
> -Yonik
>
> On Tue, Oct 14, 2008 at 12:55 PM, Reetha Hariharan <re...@gmail.com>
> wrote:
> > Hi Yonik,
> >
> > Thanks for your reply.
> >
> > In my case I don't want those document that has Null value for the field
> > that I am willing to sort
> >
> > I tried writing my own filter using RangeFilter, but it doesn't work.
> >
> > I used something like the following in my custom filter.
> >
> > public class NotNullRangeFilter extends org.apache.lucene.search.Filter {
> >    private RangeFilter rangeFilter;
> >    public BitSet bits(IndexReader reader) throws IOException {
> >        rangeFilter = RangeFilter.More("lastUpdatedDate"/*Field to sort*/,
> > "" /*atleast an empty string but not null*/);
> >        return rangeFilter.bits(reader);
> >    }
> > }
> >
> > The above does not work. Let me know if I am doing something wrong.
> >
> > Any inputs on this ?
> >
> > Thanks,
> >
> > On Tue, Oct 14, 2008 at 8:11 AM, Yonik Seeley <yo...@apache.org> wrote:
> >
> >> On Tue, Oct 14, 2008 at 4:35 AM, Reetha Hariharan <reethahari@gmail.com
> >
> >> wrote:
> >> > I am searching using one field, say X and want to sort the results
> using
> >> > another, say Y (Which can have null values). But I am expecting Sort
> to
> >> > ignore all the null values and just sort only records that has values
> in
> >> Y
> >> > field.
> >>
> >> Do you want docs with null (missing) values in the results or not?
> >> If so, they need to go somewhere (before or after the others).
> >> If you don't want those docs at all, exclude them in the query, or
> >> with a filter.
> >>
> >> -Yonik
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Re: Newbie Question - Lucene Sorting NOT Ignoring NULL values

Posted by Yonik Seeley <yo...@apache.org>.
You're on the right track I think... perhaps try using RangeFilter
directly rather than creating your own class.  Something like:

Filter filter = RangeFilter.More("lastUpdatedDate","");
searcher.search(query, filter)

If that works for you, then the next step would be to look at
CachingWrapperFilter, which can help prevent calculating the filter
for every query.

-Yonik

On Tue, Oct 14, 2008 at 12:55 PM, Reetha Hariharan <re...@gmail.com> wrote:
> Hi Yonik,
>
> Thanks for your reply.
>
> In my case I don't want those document that has Null value for the field
> that I am willing to sort
>
> I tried writing my own filter using RangeFilter, but it doesn't work.
>
> I used something like the following in my custom filter.
>
> public class NotNullRangeFilter extends org.apache.lucene.search.Filter {
>    private RangeFilter rangeFilter;
>    public BitSet bits(IndexReader reader) throws IOException {
>        rangeFilter = RangeFilter.More("lastUpdatedDate"/*Field to sort*/,
> "" /*atleast an empty string but not null*/);
>        return rangeFilter.bits(reader);
>    }
> }
>
> The above does not work. Let me know if I am doing something wrong.
>
> Any inputs on this ?
>
> Thanks,
>
> On Tue, Oct 14, 2008 at 8:11 AM, Yonik Seeley <yo...@apache.org> wrote:
>
>> On Tue, Oct 14, 2008 at 4:35 AM, Reetha Hariharan <re...@gmail.com>
>> wrote:
>> > I am searching using one field, say X and want to sort the results using
>> > another, say Y (Which can have null values). But I am expecting Sort to
>> > ignore all the null values and just sort only records that has values in
>> Y
>> > field.
>>
>> Do you want docs with null (missing) values in the results or not?
>> If so, they need to go somewhere (before or after the others).
>> If you don't want those docs at all, exclude them in the query, or
>> with a filter.
>>
>> -Yonik
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Newbie Question - Lucene Sorting NOT Ignoring NULL values

Posted by Reetha Hariharan <re...@gmail.com>.
Hi Yonik,

Thanks for your reply.

In my case I don't want those document that has Null value for the field
that I am willing to sort

I tried writing my own filter using RangeFilter, but it doesn't work.

I used something like the following in my custom filter.

public class NotNullRangeFilter extends org.apache.lucene.search.Filter {
    private RangeFilter rangeFilter;
    public BitSet bits(IndexReader reader) throws IOException {
        rangeFilter = RangeFilter.More("lastUpdatedDate"/*Field to sort*/,
"" /*atleast an empty string but not null*/);
        return rangeFilter.bits(reader);
    }
}

The above does not work. Let me know if I am doing something wrong.

Any inputs on this ?

Thanks,

On Tue, Oct 14, 2008 at 8:11 AM, Yonik Seeley <yo...@apache.org> wrote:

> On Tue, Oct 14, 2008 at 4:35 AM, Reetha Hariharan <re...@gmail.com>
> wrote:
> > I am searching using one field, say X and want to sort the results using
> > another, say Y (Which can have null values). But I am expecting Sort to
> > ignore all the null values and just sort only records that has values in
> Y
> > field.
>
> Do you want docs with null (missing) values in the results or not?
> If so, they need to go somewhere (before or after the others).
> If you don't want those docs at all, exclude them in the query, or
> with a filter.
>
> -Yonik
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Re: Newbie Question - Lucene Sorting NOT Ignoring NULL values

Posted by Yonik Seeley <yo...@apache.org>.
On Tue, Oct 14, 2008 at 4:35 AM, Reetha Hariharan <re...@gmail.com> wrote:
> I am searching using one field, say X and want to sort the results using
> another, say Y (Which can have null values). But I am expecting Sort to
> ignore all the null values and just sort only records that has values in Y
> field.

Do you want docs with null (missing) values in the results or not?
If so, they need to go somewhere (before or after the others).
If you don't want those docs at all, exclude them in the query, or
with a filter.

-Yonik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org