You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Nagelberg, Kallin" <KN...@globeandmail.com> on 2010/01/06 16:52:14 UTC
ord on TrieDateField always returning max
Hi everyone,
I've been trying to add a date based boost to my queries. I have a field like:
<fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
<field name="datetime" type="tdate" indexed="true" stored="true" required="true" />
When I look at the datetime field in the solr schema browser I can see that there are 9051 distinct dates.
When I try to add the parameter to my query like: bf=ord(datetime) (on a dismax query) I always get 9051 as the result of the function. I see this in the debug data:
1698.6041 = (MATCH) FunctionQuery(top(ord(datetime))), product of:
9051.0 = 9051
1.0 = boost
0.18767032 = queryNorm
It is exactly the same for every result, even though each result has a different value for datetime.
Does anyone have any suggestions as to why this could be happening? I have done extensive googling with no luck.
Thanks,
Kallin Nagelberg.
Re: ord on TrieDateField always returning max
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Wed, Jan 6, 2010 at 11:26 AM, Nagelberg, Kallin
<KN...@globeandmail.com> wrote:
> Thanks Yonik, I was just looking at that actually.
> Trying something like recip(ms(NOW,datetime),3.16e-11,1,1)^10 now.
I'd also recommend looking into a multiplicative boost too - IMO they
normally make more sense.
http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents
-Yonik
http://www.lucidimagination.com
> My 'inspiration' for the ord method was actually the Solr 1.4 Enterprise Search server book. Page 126 has a section 'using reciprocals and rord with dates'. You should let those guys know what's up!
>
> Thanks,
> Kallin.
>
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
> Sent: Wednesday, January 06, 2010 11:23 AM
> To: solr-user@lucene.apache.org
> Subject: Re: ord on TrieDateField always returning max
>
> Besides using up a lot more memory, ord() isn't even going to work for
> a field with multiple tokens indexed per value (like tdate).
> I'd recommend using a function on the date value itself.
> http://wiki.apache.org/solr/FunctionQuery#ms
>
> -Yonik
> http://www.lucidimagination.com
>
>
>
> On Wed, Jan 6, 2010 at 10:52 AM, Nagelberg, Kallin
> <KN...@globeandmail.com> wrote:
>> Hi everyone,
>>
>> I've been trying to add a date based boost to my queries. I have a field like:
>>
>> <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
>> <field name="datetime" type="tdate" indexed="true" stored="true" required="true" />
>>
>> When I look at the datetime field in the solr schema browser I can see that there are 9051 distinct dates.
>>
>> When I try to add the parameter to my query like: bf=ord(datetime) (on a dismax query) I always get 9051 as the result of the function. I see this in the debug data:
>>
>>
>> 1698.6041 = (MATCH) FunctionQuery(top(ord(datetime))), product of:
>>
>> 9051.0 = 9051
>>
>> 1.0 = boost
>>
>> 0.18767032 = queryNorm
>>
>>
>>
>> It is exactly the same for every result, even though each result has a different value for datetime.
>>
>>
>>
>> Does anyone have any suggestions as to why this could be happening? I have done extensive googling with no luck.
>>
>>
>>
>> Thanks,
>>
>> Kallin Nagelberg.
>>
>>
>
RE: ord on TrieDateField always returning max
Posted by "David Smiley @MITRE.org" <DS...@mitre.org>.
Thanks for helping out this user Hoss. I needed to highlight the
ramifications of both the trie types and the impact of the ms() function on
chapter 5 so I enhanced this addendum a bit more. I feel a little bad with
"1.4" in the book title given the book came out before its release, but
Packt basically required a version # and 1.4 made much more sense than 1.3.
The behavior the user sees shouldn't happen, IMO. When you're doing
something that requires a field to be indexed with a single term (e.g.
ord,rord, and others), instead of getting an error you get odd behavior.
It's unfortunate. Perhaps I'll file a bug... a field would have to be
declared to have no more than 1 token and features that require it would
have to check.
~ David Smiley
hossman wrote:
>
>
> : My 'inspiration' for the ord method was actually the Solr 1.4 Enterprise
> : Search server book. Page 126 has a section 'using reciprocals and rord
> : with dates'. You should let those guys know what's up!
>
> 1) Although it's not 100% obvious, those examples in the book use the
> "DateField" type -- and ord() should still work for those. If you are
> using TrieDateField then things work differently.
>
> 2) David and EricP are regulars on the solr-user list, so rest assured
> they already know.
>
> 3) There is a wiki page with an Addendum section they wrote to cover
> things that changed in Solr prior to the 1.4 release but after they went
> to press. Trie fields are mentioned, as is the new ms() function, but i
> don't see anything explicitly addressing the snag you hit ... yet. It is
> a wiki afterall, so you can feel free to add things as you find them.
>
>
>
> -Hoss
>
>
>
--
View this message in context: http://old.nabble.com/ord-on-TrieDateField-always-returning-max-tp27045931p27095782.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: ord on TrieDateField always returning max
Posted by Chris Hostetter <ho...@fucit.org>.
: My 'inspiration' for the ord method was actually the Solr 1.4 Enterprise
: Search server book. Page 126 has a section 'using reciprocals and rord
: with dates'. You should let those guys know what's up!
1) Although it's not 100% obvious, those examples in the book use the
"DateField" type -- and ord() should still work for those. If you are
using TrieDateField then things work differently.
2) David and EricP are regulars on the solr-user list, so rest assured
they already know.
3) There is a wiki page with an Addendum section they wrote to cover
things that changed in Solr prior to the 1.4 release but after they went
to press. Trie fields are mentioned, as is the new ms() function, but i
don't see anything explicitly addressing the snag you hit ... yet. It is
a wiki afterall, so you can feel free to add things as you find them.
-Hoss
RE: ord on TrieDateField always returning max
Posted by "Nagelberg, Kallin" <KN...@globeandmail.com>.
Thanks Yonik, I was just looking at that actually.
Trying something like recip(ms(NOW,datetime),3.16e-11,1,1)^10 now.
My 'inspiration' for the ord method was actually the Solr 1.4 Enterprise Search server book. Page 126 has a section 'using reciprocals and rord with dates'. You should let those guys know what's up!
Thanks,
Kallin.
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Wednesday, January 06, 2010 11:23 AM
To: solr-user@lucene.apache.org
Subject: Re: ord on TrieDateField always returning max
Besides using up a lot more memory, ord() isn't even going to work for
a field with multiple tokens indexed per value (like tdate).
I'd recommend using a function on the date value itself.
http://wiki.apache.org/solr/FunctionQuery#ms
-Yonik
http://www.lucidimagination.com
On Wed, Jan 6, 2010 at 10:52 AM, Nagelberg, Kallin
<KN...@globeandmail.com> wrote:
> Hi everyone,
>
> I've been trying to add a date based boost to my queries. I have a field like:
>
> <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
> <field name="datetime" type="tdate" indexed="true" stored="true" required="true" />
>
> When I look at the datetime field in the solr schema browser I can see that there are 9051 distinct dates.
>
> When I try to add the parameter to my query like: bf=ord(datetime) (on a dismax query) I always get 9051 as the result of the function. I see this in the debug data:
>
>
> 1698.6041 = (MATCH) FunctionQuery(top(ord(datetime))), product of:
>
> 9051.0 = 9051
>
> 1.0 = boost
>
> 0.18767032 = queryNorm
>
>
>
> It is exactly the same for every result, even though each result has a different value for datetime.
>
>
>
> Does anyone have any suggestions as to why this could be happening? I have done extensive googling with no luck.
>
>
>
> Thanks,
>
> Kallin Nagelberg.
>
>
Re: ord on TrieDateField always returning max
Posted by Yonik Seeley <yo...@lucidimagination.com>.
Besides using up a lot more memory, ord() isn't even going to work for
a field with multiple tokens indexed per value (like tdate).
I'd recommend using a function on the date value itself.
http://wiki.apache.org/solr/FunctionQuery#ms
-Yonik
http://www.lucidimagination.com
On Wed, Jan 6, 2010 at 10:52 AM, Nagelberg, Kallin
<KN...@globeandmail.com> wrote:
> Hi everyone,
>
> I've been trying to add a date based boost to my queries. I have a field like:
>
> <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
> <field name="datetime" type="tdate" indexed="true" stored="true" required="true" />
>
> When I look at the datetime field in the solr schema browser I can see that there are 9051 distinct dates.
>
> When I try to add the parameter to my query like: bf=ord(datetime) (on a dismax query) I always get 9051 as the result of the function. I see this in the debug data:
>
>
> 1698.6041 = (MATCH) FunctionQuery(top(ord(datetime))), product of:
>
> 9051.0 = 9051
>
> 1.0 = boost
>
> 0.18767032 = queryNorm
>
>
>
> It is exactly the same for every result, even though each result has a different value for datetime.
>
>
>
> Does anyone have any suggestions as to why this could be happening? I have done extensive googling with no luck.
>
>
>
> Thanks,
>
> Kallin Nagelberg.
>
>