Indexing date types

[Steven White <sw...@gmail.com>]

Hi everyone,

This is two part question about date in Solr.

Question #1:

My understanding is, in order for me to index date types, the date data
must be formatted and indexed as such:

    YYYY-MM-DDThh:mm:ssZ

What if I do not have the time part, should I be indexing it as such and
still get all the features of facet search on date (obviously, excluding
time):

    YYYY-MM-DD

I have setup my Solr schema as such to index dates:

    <fieldType name="dateRange" class="solr.DateRangeField"/>
    <field name="other_dates" docValues="true" type="dateRange"
multiValued="true" indexed="true" required="false" stored="false"/>

Question #2:

Per the above schema design, I will be indexing my date type as
"multiValued" which, as you know, more than 1 date data will be indexed
into the field "other_dates".  Will this be a problem when I facet search
on this field?  That is, will all the date facet capability still work,
such as range and math per
https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
(obviously, excluding time)?

Thanks in advance.

Steve

Re: Indexing date types

[Emir Arnautovic <em...@sematext.com>]

Hi Steve,
The best way to make sure everything work is to test, but without 
testing on target version, my answers would be:
1. if Solr accepts date without time it'll be the same as time 00:00:00 
so if it does not accept, you can always append.
2. it'll work just expect that sum of facet count can be larger than 
total doc since same doc will count in more than one bucket.
3. doc values work only on Str and Trie fields and question is why you 
need DateRangeField - are you indexing ranges or points in time? If it 
is just multiple points, you can use TrieDateField with default 
precision to enable fast range queries.

HTH,
Emir

On 02.06.2016 18:10, Steven White wrote:
> I forgot to mention another issue I run into.  Looks like "docValues" is
> not supported with DateRangeField, is this true?
>
> If I have:
>
>      <fieldType name="dateRange" class="solr.DateRangeField"/>
>      <field name="other_dates" docValues="true" type="dateRange"
> multiValued="true" indexed="true" required="false" stored="false"/>
>
> Solr will fail to start, reporting the following error:
>
>      org.apache.solr.core.CoreContainer; Error creating core [openpages]:
> Could not load conf for core openpages: Field type
> dateRange{class=org.apache.solr.schema.DateRangeField,analyzer=org.apache.solr.schema.FieldType$DefaultAnalyzer,args={class=solr.DateRangeField}}
> does not support doc values.
>
> I have to remove "docValues" to fix this.  Is this the case or have I
> missed something?
>
> Thanks.
>
> Steve
>
> On Thu, Jun 2, 2016 at 11:46 AM, Steven White <sw...@gmail.com> wrote:
>
>> Hi everyone,
>>
>> This is two part question about date in Solr.
>>
>> Question #1:
>>
>> My understanding is, in order for me to index date types, the date data
>> must be formatted and indexed as such:
>>
>>      YYYY-MM-DDThh:mm:ssZ
>>
>> What if I do not have the time part, should I be indexing it as such and
>> still get all the features of facet search on date (obviously, excluding
>> time):
>>
>>      YYYY-MM-DD
>>
>> I have setup my Solr schema as such to index dates:
>>
>>      <fieldType name="dateRange" class="solr.DateRangeField"/>
>>      <field name="other_dates" docValues="true" type="dateRange"
>> multiValued="true" indexed="true" required="false" stored="false"/>
>>
>> Question #2:
>>
>> Per the above schema design, I will be indexing my date type as
>> "multiValued" which, as you know, more than 1 date data will be indexed
>> into the field "other_dates".  Will this be a problem when I facet search
>> on this field?  That is, will all the date facet capability still work,
>> such as range and math per
>> https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
>> (obviously, excluding time)?
>>
>> Thanks in advance.
>>
>> Steve
>>

-- 
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

Re: Indexing date types

[Steven White <sw...@gmail.com>]

I forgot to mention another issue I run into.  Looks like "docValues" is
not supported with DateRangeField, is this true?

If I have:

    <fieldType name="dateRange" class="solr.DateRangeField"/>
    <field name="other_dates" docValues="true" type="dateRange"
multiValued="true" indexed="true" required="false" stored="false"/>

Solr will fail to start, reporting the following error:

    org.apache.solr.core.CoreContainer; Error creating core [openpages]:
Could not load conf for core openpages: Field type
dateRange{class=org.apache.solr.schema.DateRangeField,analyzer=org.apache.solr.schema.FieldType$DefaultAnalyzer,args={class=solr.DateRangeField}}
does not support doc values.

I have to remove "docValues" to fix this.  Is this the case or have I
missed something?

Thanks.

Steve

On Thu, Jun 2, 2016 at 11:46 AM, Steven White <sw...@gmail.com> wrote:

> Hi everyone,
>
> This is two part question about date in Solr.
>
> Question #1:
>
> My understanding is, in order for me to index date types, the date data
> must be formatted and indexed as such:
>
>     YYYY-MM-DDThh:mm:ssZ
>
> What if I do not have the time part, should I be indexing it as such and
> still get all the features of facet search on date (obviously, excluding
> time):
>
>     YYYY-MM-DD
>
> I have setup my Solr schema as such to index dates:
>
>     <fieldType name="dateRange" class="solr.DateRangeField"/>
>     <field name="other_dates" docValues="true" type="dateRange"
> multiValued="true" indexed="true" required="false" stored="false"/>
>
> Question #2:
>
> Per the above schema design, I will be indexing my date type as
> "multiValued" which, as you know, more than 1 date data will be indexed
> into the field "other_dates".  Will this be a problem when I facet search
> on this field?  That is, will all the date facet capability still work,
> such as range and math per
> https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
> (obviously, excluding time)?
>
> Thanks in advance.
>
> Steve
>