SOLR 8.1.1 index on pdate field included in search results

["Hodder, Rick" <RH...@navg.com>]

I am migrating from SOLR 4.10.2 to 8.1.1. For some reason, in the 8.1.1 core, a pdate index named IDX_ExpirationDate is appearing as a field in the search results documents.
I have several other indexes that are defined and (correctly) do not appear in the results. But the index I am having trouble with is the only one based on a pdate.
Here is a sample 8.1.1 response that demonstrates the issue:
"response":{"numFound":58871,"start":0,"docs":[
      {
        "id":"11111",
        "ExpirationDate":"2018-01-26T00:00:00Z",
        "_version_":1641033044033798170,
        "IDX_ExpirationDate":["2018-01-26T00:00:00Z"]},
      {
        "id":"22222",
        "ExpirationDate":"2018-02-20T00:00:00Z",
        "_version_":1641032965380112384,
        "IDX_ExpirationDate":["2018-02-20T00:00:00Z"]},

ExpirationDate is supposed to be there, but IDX_ExpirationDate should not. I know that I can probably keep using date, but it is deprecated, and part of the reason for upgrading to 8.1.1 is to use the latest non-deprecated stuff ;-)
I have an index named IDX_ExpirationDate based on a field called ExpirationDate that was a date field in 4.10.2:
<fieldType name="date" class="solr.TrieDateField" precisionStep="0" positionIncrementGap="0"/>
<field name="IDX_ExpirationDate" type="date" indexed="true" stored="false" multiValued="true" />
<field name="ExpirationDate" type = "date" indexed = "true" stored = "true" />
<copyField source="ExpirationDate" dest="IDX_ExpirationDate"/>

In the 8.1.1 core, I have this configured as a pdate:
<fieldType name="pdate" class="solr.DatePointField" docValues="true"/>
<field name="IDX_ExpirationDate" type="pdate" indexed="true" stored="false" multiValued="true" />
<field name="ExpirationDate" type = "pdate" indexed = "true" stored = "true" />
<copyField source="ExpirationDate" dest="IDX_ExpirationDate"/>

RE: SOLR 8.1.1 index on pdate field included in search results

["Hodder, Rick" <RH...@navg.com>]

Hi Shawn,

>The DatePointField class defaults to docValues="true" and useDocValuesAsStored="true".  Unless those parameters are changed, 
>if the field is defined for a document, it will typically be in search results.

Just checking, I'm fine with ExpirationDate appearing in the results, it's the index IDX_ExpirationDate that I don't want in the results. 

So you are saying that I should add  docValues="false" or docValuesAsStored="false" to the indexed but not stored field?:

<field name="IDX_ExpirationDate" type="pdate" indexed="true" stored="false" multiValued="true" />    

I have other IDX_ fields defined that are not pdate and they don't appear in results, that's what's confusing me, for example:

<field name="IDX_Country" type="text_general" indexed="true" stored="false" multiValued="true" />		
	
Thanks,
Rick

RE: SOLR 8.1.1 index on pdate field included in search results

["Hodder, Rick" <RH...@navg.com>]

You are right of course, Shawn.

I added useDocValuesAsStored="false" to the IDX_ExpirationDate field definition, and it no longer shows up

Thanks,
Rick

-----Original Message-----
From: Hodder, Rick 
Sent: Monday, August 05, 2019 2:02 PM
To: solr-user@lucene.apache.org
Subject: RE: SOLR 8.1.1 index on pdate field included in search results

Hi Shawn,

>The DatePointField class defaults to docValues="true" and 
>useDocValuesAsStored="true".  Unless those parameters are changed, if the field is defined for a document, it will typically be in search results.

Just checking, I'm fine with ExpirationDate appearing in the results, it's the index IDX_ExpirationDate that I don't want in the results. 

So you are saying that I should add  docValues="false" or docValuesAsStored="false" to the indexed but not stored field?:

<field name="IDX_ExpirationDate" type="pdate" indexed="true" stored="false" multiValued="true" />    

I have other IDX_ fields defined that are not pdate and they don't appear in results, that's what's confusing me, for example:

<field name="IDX_Country" type="text_general" indexed="true" stored="false" multiValued="true" />		
	
Thanks,
Rick

Re: SOLR 8.1.1 index on pdate field included in search results

[Shawn Heisey <ap...@elyograg.org>]

On 8/5/2019 10:37 AM, Hodder, Rick wrote:
> ExpirationDate is supposed to be there, but IDX_ExpirationDate should not. I know that I can probably keep using date, but it is deprecated, and part of the reason for upgrading to 8.1.1 is to use the latest non-deprecated stuff ;-)

The DatePointField class defaults to docValues="true" and 
useDocValuesAsStored="true".  Unless those parameters are changed, if 
the field is defined for a document, it will typically be in search results.

https://lucene.apache.org/solr/guide/6_6/docvalues.html#DocValues-RetrievingDocValuesDuringSearch

Thanks,
Shawn