tf and very short text fields

[Walter Underwood <wu...@wunderwood.org>]

And here is another peculiarity of short text fields.

The movie "New York, New York" should not be twice as relevant for the query "new york". Is there a way to use a binary term frequency rather than a count?

wunder
--
Walter Underwood
wunder@wunderwood.org

Re: tf and very short text fields

[Michael Sokolov <ms...@safaribooksonline.com>]

On 4/3/14 7:46 AM, Michael Sokolov wrote:
> On 4/1/14 2:32 PM, Walter Underwood wrote:
>> And here is another peculiarity of short text fields.
>>
>> The movie "New York, New York" should not be twice as relevant for 
>> the query "new york". Is there a way to use a binary term frequency 
>> rather than a count?
>>
>> wunder
>> -- 
>> Walter Underwood
>> wunder@wunderwood.org
>>
>>
>>
>>
> Walter - you can write a custom scoring function in Java, or use 
> function queries to compose one in Solr query language.  I don't see a 
> exists(term) function in the list here 
> https://cwiki.apache.org/confluence/display/solr/Function+Queries that 
> would return 0 or 1, but you could write that?
>
> -Mike
I see I missed Markus' earlier responses - somehow the messages didn't 
get threaded together in my reader.  I may have to try BM25 too!

Re: tf and very short text fields

[Michael Sokolov <ms...@safaribooksonline.com>]

On 4/1/14 2:32 PM, Walter Underwood wrote:
> And here is another peculiarity of short text fields.
>
> The movie "New York, New York" should not be twice as relevant for the query "new york". Is there a way to use a binary term frequency rather than a count?
>
> wunder
> --
> Walter Underwood
> wunder@wunderwood.org
>
>
>
>
Walter - you can write a custom scoring function in Java, or use 
function queries to compose one in Solr query language.  I don't see a 
exists(term) function in the list here 
https://cwiki.apache.org/confluence/display/solr/Function+Queries that 
would return 0 or 1, but you could write that?

-Mike