You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by jon kerling <jo...@yahoo.com.INVALID> on 2015/06/09 16:06:42 UTC
TZ & rounding
Hi,
I'm using Solr 4.10.0.I'm trying to figure out how to use the TZ param.I've noticed that i have to use date math in order for this to work,also I've got to use rounding when I query Solr in order to use the TZ param.
The issue is that when i query I'd like to get recent hours for my search, something like the query for last 1 hour. The Date Math supports it but the TZ works only if my rounding is DAY or more, and i need MINUTE rounding for my query.
I wanted to know is it possible to use TZ feature when I'm using MINUTE rounding?
My query looks like this:
q=*:*fq=realDate:[NOW/MINUTE-3HOURS TO NOW/MINUTE-1MINUTES]&rows=20&NOW=1433777301000&facet.range=realDate&facet=true&facet.range.start=NOW/MINUTE-3HOURS&facet.range.end=NOW/MINUTE-1MINUTES&facet.range.gap=%2B10MINUTE&TZ=GMT
<field name="realDate" type="tdate" indexed="true" stored="true" required="false" multiValued="false" />
Thank you,Jon
Re: TZ & rounding
Posted by Chris Hostetter <ho...@fucit.org>.
: So my question is: can I get offset of time if I use NOW/MINUTE and not NOW/DAY rounding?
i'm sorry, but your question is still too terse, vague, and ambiguious for
me to really make much sense of it; and the example queries you provided
really don't have enough context for me to understand what it is about
these queries is like or dislike what you ultimately want.
please re-read my previous request for clarification of your question --
in particular, stop assuming anything about Solr, and just tell us about
your data, and the type of problem you are trying to solve in normal
sentences with some real concrete examples...
> You've given an example of a query you are currently using -- but you
> haven't given us any of hte other info we really need ot try and
> understand your question (ie: what do some of the documents you've
> indexed look like? what results do you get from your query? what do you
> see in those results that isn't matching your goal? what documents are
> matched by your query (or by a facet) that you don't want to be matched?
> what documents aren't matched that you want to be matched?
>
> The best thing to do would be if you could just describe for us in words
> what you want, and give a specific example -- such as "If it's 4:37PM in
> my local timezone Foo/Bar, i want to send a query to Solr and have it
> return results based on teh time range X to Y with facets like ...."
-Hoss
http://www.lucidworks.com/
Re: TZ & rounding
Posted by jon kerling <jo...@yahoo.com.INVALID>.
Thank you for your reply.
So my question is: can I get offset of time if I use NOW/MINUTE and not NOW/DAY rounding?
You said " TZ affects what timezone is used when defining the concept of a "day" for
the purposes of rounding by day. " I understand from this answer that query like I mentioned could not be treated as I want: different TZ give me different results.
Here you see different time zones but the results is always the same, even though it is not accurate to get the same response for last 5 hours in GMT time zone and in GMT+07 time zone.
http://localhost:8983/solr/CORE_1/select?q=*:*&fq=realDate:[NOW/MINUTE-5HOURS%20TO%20NOW/MINUTE-1MINUTES]&rows=20&NOW=1433934000000&facet.range=realDate&facet=true&facet.range.start=NOW/MINUTE-5HOURS&facet.range.end=NOW/MINUTE-1MINUTES&facet.range.gap=%2B1HOUR&fl=realDate&fl=f1&TZ=GMT
http://localhost:8983/solr/CORE_1/select?q=*:*&fq=realDate:[NOW/MINUTE-5HOURS%20TO%20NOW/MINUTE-1MINUTES]&rows=20&NOW=1433934000000&facet.range=realDate&facet=true&facet.range.start=NOW/MINUTE-5HOURS&facet.range.end=NOW/MINUTE-1MINUTES&facet.range.gap=%2B1HOUR&fl=realDate&fl=f1&TZ=America/Los_Angeles
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">8</int>
<lst name="params">
<str name="facet">true</str>
<arr name="fl">
<str>realDate</str>
<str>f1</str>
</arr>
<str name="NOW">1433934000000</str>
<str name="q">*:*</str>
<str name="facet.range.start">NOW/MINUTE-5HOURS</str>
<str name="facet.range">realDate</str>
<str name="TZ">GMT</str>
<str name="facet.range.gap">+1HOUR</str>
<str name="facet.range.end">NOW/MINUTE-1MINUTES</str>
<str name="fq">realDate:[NOW/MINUTE-5HOURS TO NOW/MINUTE-1MINUTES]</str>
<str name="rows">20</str>
</lst>
</lst>
<result name="response" numFound="1" start="0">
<doc>
<str name="f1">BOOOOO1</str>
<date name="realDate">2015-06-10T07:00:00Z</date>
</doc>
</result>
<lst name="facet_counts">
<lst name="facet_queries" />
<lst name="facet_fields" />
<lst name="facet_dates" />
<lst name="facet_ranges">
<lst name="realDate">
<lst name="counts">
<int name="2015-06-10T06:00:00Z">0</int>
<int name="2015-06-10T07:00:00Z">1</int>
<int name="2015-06-10T08:00:00Z">0</int>
<int name="2015-06-10T09:00:00Z">0</int>
<int name="2015-06-10T10:00:00Z">0</int>
</lst>
<str name="gap">+1HOUR</str>
<date name="start">2015-06-10T06:00:00Z</date>
<date name="end">2015-06-10T11:00:00Z</date>
</lst>
</lst>
<lst name="facet_intervals" />
</lst>
</response>
if I would use the same Query with DAY rounding I'll get different results since the TimeZone feature will work:
http://localhost:8983/solr/CORE_1/select?q=*:*&fq=realDate:[NOW/DAY-5HOURS%20TO%20NOW/DAY-1MINUTES]&rows=20&NOW=1433934000000&facet.range=realDate&facet=true&facet.range.start=NOW/DAY-5HOURS&facet.range.end=NOW/DAY-1MINUTES&facet.range.gap=%2B1HOUR&fl=realDate&fl=f1&TZ=GMT
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">7</int>
<lst name="params">
<str name="facet">true</str>
<arr name="fl">
<str>realDate</str>
<str>f1</str>
</arr>
<str name="NOW">1433934000000</str>
<str name="q">*:*</str>
<str name="facet.range.start">NOW/DAY-5HOURS</str>
<str name="facet.range">realDate</str>
<str name="TZ">GMT</str>
<str name="facet.range.gap">+1HOUR</str>
<str name="facet.range.end">NOW/DAY-1MINUTES</str>
<str name="fq">realDate:[NOW/DAY-5HOURS TO NOW/DAY-1MINUTES]</str>
<str name="rows">20</str>
</lst>
</lst>
<result name="response" numFound="0" start="0" />
<lst name="facet_counts">
<lst name="facet_queries" />
<lst name="facet_fields" />
<lst name="facet_dates" />
<lst name="facet_ranges">
<lst name="realDate">
<lst name="counts">
<int name="2015-06-09T19:00:00Z">0</int>
<int name="2015-06-09T20:00:00Z">0</int>
<int name="2015-06-09T21:00:00Z">0</int>
<int name="2015-06-09T22:00:00Z">0</int>
<int name="2015-06-09T23:00:00Z">0</int>
</lst>
<str name="gap">+1HOUR</str>
<date name="start">2015-06-09T19:00:00Z</date>
<date name="end">2015-06-10T00:00:00Z</date>
</lst>
</lst>
<lst name="facet_intervals" />
</lst>
</response>
here you can see the offset:
http://localhost:8983/solr/CORE_1/select?q=*:*&fq=realDate:[NOW/DAY-5HOURS%20TO%20NOW/DAY-1MINUTES]&rows=20&NOW=1433934000000&facet.range=realDate&facet=true&facet.range.start=NOW/DAY-5HOURS&facet.range.end=NOW/DAY-1MINUTES&facet.range.gap=%2B1HOUR&fl=realDate&fl=f1&TZ=America/Los_Angeles
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">8</int>
<lst name="params">
<str name="facet">true</str>
<arr name="fl">
<str>realDate</str>
<str>f1</str>
</arr>
<str name="NOW">1433934000000</str>
<str name="q">*:*</str>
<str name="facet.range.start">NOW/DAY-5HOURS</str>
<str name="facet.range">realDate</str>
<str name="TZ">America/Los_Angeles</str>
<str name="facet.range.gap">+1HOUR</str>
<str name="facet.range.end">NOW/DAY-1MINUTES</str>
<str name="fq">realDate:[NOW/DAY-5HOURS TO NOW/DAY-1MINUTES]</str>
<str name="rows">20</str>
</lst>
</lst>
<result name="response" numFound="1" start="0">
<doc>
<str name="f1">BOOOOO4</str>
<date name="realDate">2015-06-10T05:00:00Z</date>
</doc>
</result>
<lst name="facet_counts">
<lst name="facet_queries" />
<lst name="facet_fields" />
<lst name="facet_dates" />
<lst name="facet_ranges">
<lst name="realDate">
<lst name="counts">
<int name="2015-06-10T02:00:00Z">0</int>
<int name="2015-06-10T03:00:00Z">0</int>
<int name="2015-06-10T04:00:00Z">0</int>
<int name="2015-06-10T05:00:00Z">1</int>
<int name="2015-06-10T06:00:00Z">0</int>
</lst>
<str name="gap">+1HOUR</str>
<date name="start">2015-06-10T02:00:00Z</date>
<date name="end">2015-06-10T07:00:00Z</date>
</lst>
</lst>
<lst name="facet_intervals" />
</lst>
</response>
Thank you,
Jon.
On Tuesday, June 9, 2015 7:42 PM, Chris Hostetter <ho...@fucit.org> wrote:
: So, are you saying that you are expected to store UTC dates in your
: index, but if you happen to know that a user is in a different timezone,
: you can round those dates for them according to their timezone instead
: of UTC?
:
: That's how I'd interpret it, but useful to confirm.
Date formatting and Date Math are two completley different things.
0) All dates, in the index, are stored in UTC.
1) All dates, if expressed as Strings, must be formatted in UTC when
provided to TrieDateField either to index or for the purposes of query
parsing. (if you use SolrJ to send a Date object there is
no String representation and UTC is irelevant, likewise things like
ParseDateFieldUpdateProcessorFactory can parse other formats and be
configured with other timezones)
2) By default, all date math expressions are evaluated relative to the UTC
TimeZone, but the TZ parameter can be specified to override this
behaviour, by forcing all date based addition and rounding to be relative
to the specified time zone.
https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
-Hoss
http://www.lucidworks.com/
Re: TZ & rounding
Posted by Chris Hostetter <ho...@fucit.org>.
: So, are you saying that you are expected to store UTC dates in your
: index, but if you happen to know that a user is in a different timezone,
: you can round those dates for them according to their timezone instead
: of UTC?
:
: That's how I'd interpret it, but useful to confirm.
Date formatting and Date Math are two completley different things.
0) All dates, in the index, are stored in UTC.
1) All dates, if expressed as Strings, must be formatted in UTC when
provided to TrieDateField either to index or for the purposes of query
parsing. (if you use SolrJ to send a Date object there is
no String representation and UTC is irelevant, likewise things like
ParseDateFieldUpdateProcessorFactory can parse other formats and be
configured with other timezones)
2) By default, all date math expressions are evaluated relative to the UTC
TimeZone, but the TZ parameter can be specified to override this
behaviour, by forcing all date based addition and rounding to be relative
to the specified time zone.
https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
-Hoss
http://www.lucidworks.com/
Re: TZ & rounding
Posted by Upayavira <uv...@odoko.co.uk>.
On Tue, Jun 9, 2015, at 05:27 PM, Chris Hostetter wrote:
>
> : I'm using Solr 4.10.0.I'm trying to figure out how to use the TZ
> : param.I've noticed that i have to use date math in order for this to
> : work,also I've got to use rounding when I query Solr in order to use
> the
> : TZ param.
>
> I'm having trouble understanding your question. The TZ param, as
> documented, only affects what TimeZone is used for the purpose of
> rounding
> dates -- so of course, if you aren't using rounding then the TZ param
> does
> nothing. And since rounding is a form of date math, then yes it's also
> true you must use date math for TZ to have any meaning.
>
> https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
>
> : The issue is that when i query I'd like to get recent hours for my
> : search, something like the query for last 1 hour. The Date Math
> : supports it but the TZ works only if my rounding is DAY or more, and i
> : need MINUTE rounding for my query.
>
> TZ affects what timezone is used when defining the concept of a "day" for
> the purposes of rounding by day -- if you used a TimeZone like
> TZ=Asia/Tehran you'll see the correct rounding even though that timezone
> is +04:30 relative to UTC.
>
> : I wanted to know is it possible to use TZ feature when I'm using MINUTE
> rounding?
> :
> : My query looks like this:
> : q=*:*fq=realDate:[NOW/MINUTE-3HOURS TO
> NOW/MINUTE-1MINUTES]&rows=20&NOW=1433777301000&facet.range=realDate&facet=true&facet.range.start=NOW/MINUTE-3HOURS&facet.range.end=NOW/MINUTE-1MINUTES&facet.range.gap=%2B10MINUTE&TZ=GMT
> : <field name="realDate" type="tdate" indexed="true" stored="true"
> required="false" multiValued="false" />
>
> You've given an example of a query you are currently using -- but you
> haven't given us any of hte other info we really need ot try and
> understand your question (ie: what do some of the documents you've
> indexed
> look like? what results do you get from your query? what do you see in
> those results that isn't matching your goal? what documents are matched
> by
> your query (or by a facet) that you don't want to be matched? what
> documents aren't matched that you want to be matched?
>
> The best thing to do would be if you could just describe for us in words
> what you want, and give a specific example -- such as "If it's 4:37PM in
> my local timezone Foo/Bar, i want to send a query to Solr and have it
> return results based on teh time range X to Y with facets like ...."
Hoss,
So, are you saying that you are expected to store UTC dates in your
index, but if you happen to know that a user is in a different timezone,
you can round those dates for them according to their timezone instead
of UTC?
That's how I'd interpret it, but useful to confirm.
Thx!
Upayavira
Re: TZ & rounding
Posted by Chris Hostetter <ho...@fucit.org>.
: I'm using Solr 4.10.0.I'm trying to figure out how to use the TZ
: param.I've noticed that i have to use date math in order for this to
: work,also I've got to use rounding when I query Solr in order to use the
: TZ param.
I'm having trouble understanding your question. The TZ param, as
documented, only affects what TimeZone is used for the purpose of rounding
dates -- so of course, if you aren't using rounding then the TZ param does
nothing. And since rounding is a form of date math, then yes it's also
true you must use date math for TZ to have any meaning.
https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
: The issue is that when i query I'd like to get recent hours for my
: search, something like the query for last 1 hour. The Date Math
: supports it but the TZ works only if my rounding is DAY or more, and i
: need MINUTE rounding for my query.
TZ affects what timezone is used when defining the concept of a "day" for
the purposes of rounding by day -- if you used a TimeZone like
TZ=Asia/Tehran you'll see the correct rounding even though that timezone
is +04:30 relative to UTC.
: I wanted to know is it possible to use TZ feature when I'm using MINUTE rounding?
:
: My query looks like this:
: q=*:*fq=realDate:[NOW/MINUTE-3HOURS TO NOW/MINUTE-1MINUTES]&rows=20&NOW=1433777301000&facet.range=realDate&facet=true&facet.range.start=NOW/MINUTE-3HOURS&facet.range.end=NOW/MINUTE-1MINUTES&facet.range.gap=%2B10MINUTE&TZ=GMT
: <field name="realDate" type="tdate" indexed="true" stored="true" required="false" multiValued="false" />
You've given an example of a query you are currently using -- but you
haven't given us any of hte other info we really need ot try and
understand your question (ie: what do some of the documents you've indexed
look like? what results do you get from your query? what do you see in
those results that isn't matching your goal? what documents are matched by
your query (or by a facet) that you don't want to be matched? what
documents aren't matched that you want to be matched?
The best thing to do would be if you could just describe for us in words
what you want, and give a specific example -- such as "If it's 4:37PM in
my local timezone Foo/Bar, i want to send a query to Solr and have it
return results based on teh time range X to Y with facets like ...."
-Hoss
http://www.lucidworks.com/