You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Jon Stewart <jo...@lightboxtechnologies.com> on 2015/02/11 05:21:45 UTC
StandardQueryParser with date/time fields stored as longs
Hello,
I've done a lot of googling, but haven't stumbled upon the magic
answer: how does one use StandardQueryParser with numeric fields
representing timestamps, to allow for range queries?
When indexing, my timestamp fields are ISO 8601 strings. I'm parsing
them and then storing the milliseconds epoch time as a long, i.e.:
doc.add(new LongField("created", ts.getMillis(), Field.Store.NO));
>From reading around, this seems to be the preferred method to index a
timestamp (makes sense). However, how can you get StandardQueryParser
to handle a query like "created:[2010-01-01 TO 2014-12-31]"?
For other numeric fields, StandardQueryParser.setNumericConfigMap() is
working just fine for me. It would seem that the "created" field would
have to be part of this map in order to execute the range query
properly, but that there must also be a component to parse the
date/time strings in the query and convert them to long values, right?
Thanks in advance,
Jon
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: StandardQueryParser with date/time fields stored as longs
Posted by Ian Lea <ia...@gmail.com>.
In my case, for any non-trivial search, I always build a boolean query
with relevant parsing for each field, where applicable using something
like oal.queryparser.classic.QueryParser.
So if I had some free text docs with a date field, the latter stored
as you suggest, and a query along the lines of datefrom: 2010-01-01
dateto: 2014-12-31 words: whatever, passed to my app by whatever
means, I'd parse the date fields into a NumericRangeQuery, pass the
words through a lucene supplied parser, getting back some Query
object, then glue them all in to a BooleanQuery with the relevant
MUST/SHOULD logic and boosts and whatever else I wanted.
--
Ian.
On Wed, Feb 11, 2015 at 2:37 PM, Jon Stewart
<jo...@lightboxtechnologies.com> wrote:
> Ok... so how does anyone ever use date-time queries in lucene with the
> new recommended way of using longs?
>
>
> Jon
>
>
> On Wed, Feb 11, 2015 at 9:26 AM, Ian Lea <ia...@gmail.com> wrote:
>> Ah well, you've got me there. I'm not a lucene developer and rather
>> thought that I'd leave the implementation as an exercise for the
>> reader. Good luck!
>>
>>
>> --
>> Ian.
>>
>>
>> On Wed, Feb 11, 2015 at 2:20 PM, Jon Stewart
>> <jo...@lightboxtechnologies.com> wrote:
>>> Eek. So is there a parsing component somewhere that gets handed a
>>> field name and query components (e.g., "created", "2010-01-01",
>>> "2014-12-31"), which I can derive from, parse the timestamp strings,
>>> and then turn the whole thing into a numeric range query component?
>>>
>>>
>>> Jon
>>>
>>>
>>> On Wed, Feb 11, 2015 at 9:10 AM, Ian Lea <ia...@gmail.com> wrote:
>>>> To the best of my knowledge you are spot on with everything you say,
>>>> except that the component to parse the strings doesn't exist. I
>>>> suspect that a contribution to add that to StandardQueryParser might
>>>> well be accepted.
>>>>
>>>>
>>>> --
>>>> Ian.
>>>>
>>>>
>>>> On Wed, Feb 11, 2015 at 4:21 AM, Jon Stewart
>>>> <jo...@lightboxtechnologies.com> wrote:
>>>>> Hello,
>>>>>
>>>>> I've done a lot of googling, but haven't stumbled upon the magic
>>>>> answer: how does one use StandardQueryParser with numeric fields
>>>>> representing timestamps, to allow for range queries?
>>>>>
>>>>> When indexing, my timestamp fields are ISO 8601 strings. I'm parsing
>>>>> them and then storing the milliseconds epoch time as a long, i.e.:
>>>>>
>>>>> doc.add(new LongField("created", ts.getMillis(), Field.Store.NO));
>>>>>
>>>>> From reading around, this seems to be the preferred method to index a
>>>>> timestamp (makes sense). However, how can you get StandardQueryParser
>>>>> to handle a query like "created:[2010-01-01 TO 2014-12-31]"?
>>>>>
>>>>> For other numeric fields, StandardQueryParser.setNumericConfigMap() is
>>>>> working just fine for me. It would seem that the "created" field would
>>>>> have to be part of this map in order to execute the range query
>>>>> properly, but that there must also be a component to parse the
>>>>> date/time strings in the query and convert them to long values, right?
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> Jon
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>
>>>
>>>
>>> --
>>> Jon Stewart, Principal
>>> (646) 719-0317 | jon@lightboxtechnologies.com | Arlington, VA
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
>
> --
> Jon Stewart, Principal
> (646) 719-0317 | jon@lightboxtechnologies.com | Arlington, VA
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: StandardQueryParser with date/time fields stored as longs
Posted by Jon Stewart <jo...@lightboxtechnologies.com>.
Ok... so how does anyone ever use date-time queries in lucene with the
new recommended way of using longs?
Jon
On Wed, Feb 11, 2015 at 9:26 AM, Ian Lea <ia...@gmail.com> wrote:
> Ah well, you've got me there. I'm not a lucene developer and rather
> thought that I'd leave the implementation as an exercise for the
> reader. Good luck!
>
>
> --
> Ian.
>
>
> On Wed, Feb 11, 2015 at 2:20 PM, Jon Stewart
> <jo...@lightboxtechnologies.com> wrote:
>> Eek. So is there a parsing component somewhere that gets handed a
>> field name and query components (e.g., "created", "2010-01-01",
>> "2014-12-31"), which I can derive from, parse the timestamp strings,
>> and then turn the whole thing into a numeric range query component?
>>
>>
>> Jon
>>
>>
>> On Wed, Feb 11, 2015 at 9:10 AM, Ian Lea <ia...@gmail.com> wrote:
>>> To the best of my knowledge you are spot on with everything you say,
>>> except that the component to parse the strings doesn't exist. I
>>> suspect that a contribution to add that to StandardQueryParser might
>>> well be accepted.
>>>
>>>
>>> --
>>> Ian.
>>>
>>>
>>> On Wed, Feb 11, 2015 at 4:21 AM, Jon Stewart
>>> <jo...@lightboxtechnologies.com> wrote:
>>>> Hello,
>>>>
>>>> I've done a lot of googling, but haven't stumbled upon the magic
>>>> answer: how does one use StandardQueryParser with numeric fields
>>>> representing timestamps, to allow for range queries?
>>>>
>>>> When indexing, my timestamp fields are ISO 8601 strings. I'm parsing
>>>> them and then storing the milliseconds epoch time as a long, i.e.:
>>>>
>>>> doc.add(new LongField("created", ts.getMillis(), Field.Store.NO));
>>>>
>>>> From reading around, this seems to be the preferred method to index a
>>>> timestamp (makes sense). However, how can you get StandardQueryParser
>>>> to handle a query like "created:[2010-01-01 TO 2014-12-31]"?
>>>>
>>>> For other numeric fields, StandardQueryParser.setNumericConfigMap() is
>>>> working just fine for me. It would seem that the "created" field would
>>>> have to be part of this map in order to execute the range query
>>>> properly, but that there must also be a component to parse the
>>>> date/time strings in the query and convert them to long values, right?
>>>>
>>>> Thanks in advance,
>>>>
>>>> Jon
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Jon Stewart, Principal
>> (646) 719-0317 | jon@lightboxtechnologies.com | Arlington, VA
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
--
Jon Stewart, Principal
(646) 719-0317 | jon@lightboxtechnologies.com | Arlington, VA
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: StandardQueryParser with date/time fields stored as longs
Posted by Ian Lea <ia...@gmail.com>.
Ah well, you've got me there. I'm not a lucene developer and rather
thought that I'd leave the implementation as an exercise for the
reader. Good luck!
--
Ian.
On Wed, Feb 11, 2015 at 2:20 PM, Jon Stewart
<jo...@lightboxtechnologies.com> wrote:
> Eek. So is there a parsing component somewhere that gets handed a
> field name and query components (e.g., "created", "2010-01-01",
> "2014-12-31"), which I can derive from, parse the timestamp strings,
> and then turn the whole thing into a numeric range query component?
>
>
> Jon
>
>
> On Wed, Feb 11, 2015 at 9:10 AM, Ian Lea <ia...@gmail.com> wrote:
>> To the best of my knowledge you are spot on with everything you say,
>> except that the component to parse the strings doesn't exist. I
>> suspect that a contribution to add that to StandardQueryParser might
>> well be accepted.
>>
>>
>> --
>> Ian.
>>
>>
>> On Wed, Feb 11, 2015 at 4:21 AM, Jon Stewart
>> <jo...@lightboxtechnologies.com> wrote:
>>> Hello,
>>>
>>> I've done a lot of googling, but haven't stumbled upon the magic
>>> answer: how does one use StandardQueryParser with numeric fields
>>> representing timestamps, to allow for range queries?
>>>
>>> When indexing, my timestamp fields are ISO 8601 strings. I'm parsing
>>> them and then storing the milliseconds epoch time as a long, i.e.:
>>>
>>> doc.add(new LongField("created", ts.getMillis(), Field.Store.NO));
>>>
>>> From reading around, this seems to be the preferred method to index a
>>> timestamp (makes sense). However, how can you get StandardQueryParser
>>> to handle a query like "created:[2010-01-01 TO 2014-12-31]"?
>>>
>>> For other numeric fields, StandardQueryParser.setNumericConfigMap() is
>>> working just fine for me. It would seem that the "created" field would
>>> have to be part of this map in order to execute the range query
>>> properly, but that there must also be a component to parse the
>>> date/time strings in the query and convert them to long values, right?
>>>
>>> Thanks in advance,
>>>
>>> Jon
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
>
> --
> Jon Stewart, Principal
> (646) 719-0317 | jon@lightboxtechnologies.com | Arlington, VA
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: StandardQueryParser with date/time fields stored as longs
Posted by Jon Stewart <jo...@lightboxtechnologies.com>.
Eek. So is there a parsing component somewhere that gets handed a
field name and query components (e.g., "created", "2010-01-01",
"2014-12-31"), which I can derive from, parse the timestamp strings,
and then turn the whole thing into a numeric range query component?
Jon
On Wed, Feb 11, 2015 at 9:10 AM, Ian Lea <ia...@gmail.com> wrote:
> To the best of my knowledge you are spot on with everything you say,
> except that the component to parse the strings doesn't exist. I
> suspect that a contribution to add that to StandardQueryParser might
> well be accepted.
>
>
> --
> Ian.
>
>
> On Wed, Feb 11, 2015 at 4:21 AM, Jon Stewart
> <jo...@lightboxtechnologies.com> wrote:
>> Hello,
>>
>> I've done a lot of googling, but haven't stumbled upon the magic
>> answer: how does one use StandardQueryParser with numeric fields
>> representing timestamps, to allow for range queries?
>>
>> When indexing, my timestamp fields are ISO 8601 strings. I'm parsing
>> them and then storing the milliseconds epoch time as a long, i.e.:
>>
>> doc.add(new LongField("created", ts.getMillis(), Field.Store.NO));
>>
>> From reading around, this seems to be the preferred method to index a
>> timestamp (makes sense). However, how can you get StandardQueryParser
>> to handle a query like "created:[2010-01-01 TO 2014-12-31]"?
>>
>> For other numeric fields, StandardQueryParser.setNumericConfigMap() is
>> working just fine for me. It would seem that the "created" field would
>> have to be part of this map in order to execute the range query
>> properly, but that there must also be a component to parse the
>> date/time strings in the query and convert them to long values, right?
>>
>> Thanks in advance,
>>
>> Jon
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
--
Jon Stewart, Principal
(646) 719-0317 | jon@lightboxtechnologies.com | Arlington, VA
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: StandardQueryParser with date/time fields stored as longs
Posted by Ian Lea <ia...@gmail.com>.
To the best of my knowledge you are spot on with everything you say,
except that the component to parse the strings doesn't exist. I
suspect that a contribution to add that to StandardQueryParser might
well be accepted.
--
Ian.
On Wed, Feb 11, 2015 at 4:21 AM, Jon Stewart
<jo...@lightboxtechnologies.com> wrote:
> Hello,
>
> I've done a lot of googling, but haven't stumbled upon the magic
> answer: how does one use StandardQueryParser with numeric fields
> representing timestamps, to allow for range queries?
>
> When indexing, my timestamp fields are ISO 8601 strings. I'm parsing
> them and then storing the milliseconds epoch time as a long, i.e.:
>
> doc.add(new LongField("created", ts.getMillis(), Field.Store.NO));
>
> From reading around, this seems to be the preferred method to index a
> timestamp (makes sense). However, how can you get StandardQueryParser
> to handle a query like "created:[2010-01-01 TO 2014-12-31]"?
>
> For other numeric fields, StandardQueryParser.setNumericConfigMap() is
> working just fine for me. It would seem that the "created" field would
> have to be part of this map in order to execute the range query
> properly, but that there must also be a component to parse the
> date/time strings in the query and convert them to long values, right?
>
> Thanks in advance,
>
> Jon
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org