You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Karthikeyan Shanmugasundaram <ks...@gmail.com> on 2018/07/06 17:06:52 UTC
CSVResponseWriter doesnt return non-stored field even when docValues
is enabled
Hi,
As part of this Jira,
https://issues.apache.org/jira/browse/SOLR-2970?attachmentSortBy=fileName
CSVResponseWriter doesnt return fields whose stored attribute set to false,
but looks docvalues is not considered. Whereas it is not same for response
writer like json/xml ie., fields with only docvalued are retrievable.
//Code snippet from CSVResponseWriter.java
// Return only stored fields, unless an explicit field list is
specified
if (returnOnlyStored && sf != null && !sf.stored()) {
continue;
}
SolrVersion: 7.2.1
Please let me know, if any.
Thanks,
Karthik
Re: CSVResponseWriter doesnt return non-stored field even when
docValues is enabled
Posted by Erick Erickson <er...@gmail.com>.
OK, I missed the bit about
"...no explicit fl was specified..."
Please raise a JIRA. Also, make sure you highlight the bit about no fl
list specified.
Patches welcome of course!
Best,
Erick
On Sat, Jul 7, 2018 at 1:29 PM, Ganesh Sethuraman
<ga...@gmail.com> wrote:
> Yes, i have the same problem too. DocValues=True and Stored as false does
> not come out in CSV. It works fine in JSON format.
>
> On Fri, Jul 6, 2018 at 6:37 PM, Karthikeyan Shanmugasundaram <
> ksundaram.cv@gmail.com> wrote:
>
>> Yea i just tried again and below is the info,
>>
>> - Solr/Lucene Version : 7.2.1
>>
>>
>>
>> - Created collection with below fields
>>
>> <field docValues="true" indexed="true" name="contentid" required="true"
>> type="string"/>
>>
>> <field docValues="true" indexed="false" multiValued="false" name="testint"
>> type="int" stored="false"/>
>>
>> <field docValues="true" indexed="false" multiValued="false" name="testlong"
>> type="plong" stored="false"/>
>>
>>
>>
>>
>>
>> <fieldType class="solr.TrieIntField" name="int" positionIncrementGap="0"
>> precisionStep="0"/>
>>
>> <fieldType name="plong" class="solr.LongPointField" docValues="true"/>
>>
>>
>>
>>
>>
>> - Added few documents
>>
>> contentid,testint,testlong
>>
>> id,1,56
>>
>> id2,2,66
>>
>>
>>
>> - http://machine:port/solr/testdocvalue/select?q=*:*&wt=json
>>
>> [{"contentid":"id",
>>
>> "_version_":1605281886069850112,
>>
>> "timestamp":"2018-07-06T22:28:25.335Z",
>>
>> "testint":1,
>>
>> "testlong":56},
>>
>> {
>>
>> "contentid":"id2",
>>
>> "_version_":1605281886075092992,
>>
>> "timestamp":"2018-07-06T22:28:25.335Z",
>>
>> "testint":2,
>>
>> "testlong":66}]
>>
>>
>>
>> - http://machine:port/solr/testdocvalue/select?q=*:*&wt=csv
>>
>> "_version_",contentid,timestamp
>>
>> 1605281886069850112,id,2018-07-06T22:28:25.335Z
>>
>> 1605281886075092992,id2,2018-07-06T22:28:25.335Z
>>
>>
>>
>> - On attaching debugger, fields testint and testlong skipped adding to
>> csvFields list at below point since sf.stored = false and no explicit fl
>> was specified
>>
>> // Return only stored fields, unless an explicit field list is
>> specified
>>
>> *if* (returnOnlyStored && sf != *null* && !sf.stored()) {
>>
>> *continue*;
>>
>> }
>>
>>
>>
>> Ps: initially while writing down the steps, when stored=false is not
>> explicitly set in the field, it took the defaultvalue true and returned the
>> fields.
>>
>>
>>
>> Karthik
>>
>>
>>
>> On Fri, Jul 6, 2018 at 4:05 PM, Erick Erickson <er...@gmail.com>
>> wrote:
>>
>> > Do you have any evidence that this doesn't work? Because when I tried
>> > it just now it works fine. This was on master.
>> >
>> > Best,
>> > Erick
>> >
>> > On Fri, Jul 6, 2018 at 10:06 AM, Karthikeyan Shanmugasundaram
>> > <ks...@gmail.com> wrote:
>> > > Hi,
>> > >
>> > > As part of this Jira,
>> > >
>> > > https://issues.apache.org/jira/browse/SOLR-2970?
>> > attachmentSortBy=fileName
>> > >
>> > > CSVResponseWriter doesnt return fields whose stored attribute set to
>> > false,
>> > > but looks docvalues is not considered. Whereas it is not same for
>> > response
>> > > writer like json/xml ie., fields with only docvalued are retrievable.
>> > >
>> > >
>> > > //Code snippet from CSVResponseWriter.java
>> > >
>> > > // Return only stored fields, unless an explicit field list is
>> > > specified
>> > > if (returnOnlyStored && sf != null && !sf.stored()) {
>> > > continue;
>> > > }
>> > >
>> > >
>> > > SolrVersion: 7.2.1
>> > >
>> > >
>> > > Please let me know, if any.
>> > >
>> > > Thanks,
>> > > Karthik
>> >
>>
Re: CSVResponseWriter doesnt return non-stored field even when
docValues is enabled
Posted by Ganesh Sethuraman <ga...@gmail.com>.
Yes, i have the same problem too. DocValues=True and Stored as false does
not come out in CSV. It works fine in JSON format.
On Fri, Jul 6, 2018 at 6:37 PM, Karthikeyan Shanmugasundaram <
ksundaram.cv@gmail.com> wrote:
> Yea i just tried again and below is the info,
>
> - Solr/Lucene Version : 7.2.1
>
>
>
> - Created collection with below fields
>
> <field docValues="true" indexed="true" name="contentid" required="true"
> type="string"/>
>
> <field docValues="true" indexed="false" multiValued="false" name="testint"
> type="int" stored="false"/>
>
> <field docValues="true" indexed="false" multiValued="false" name="testlong"
> type="plong" stored="false"/>
>
>
>
>
>
> <fieldType class="solr.TrieIntField" name="int" positionIncrementGap="0"
> precisionStep="0"/>
>
> <fieldType name="plong" class="solr.LongPointField" docValues="true"/>
>
>
>
>
>
> - Added few documents
>
> contentid,testint,testlong
>
> id,1,56
>
> id2,2,66
>
>
>
> - http://machine:port/solr/testdocvalue/select?q=*:*&wt=json
>
> [{"contentid":"id",
>
> "_version_":1605281886069850112,
>
> "timestamp":"2018-07-06T22:28:25.335Z",
>
> "testint":1,
>
> "testlong":56},
>
> {
>
> "contentid":"id2",
>
> "_version_":1605281886075092992,
>
> "timestamp":"2018-07-06T22:28:25.335Z",
>
> "testint":2,
>
> "testlong":66}]
>
>
>
> - http://machine:port/solr/testdocvalue/select?q=*:*&wt=csv
>
> "_version_",contentid,timestamp
>
> 1605281886069850112,id,2018-07-06T22:28:25.335Z
>
> 1605281886075092992,id2,2018-07-06T22:28:25.335Z
>
>
>
> - On attaching debugger, fields testint and testlong skipped adding to
> csvFields list at below point since sf.stored = false and no explicit fl
> was specified
>
> // Return only stored fields, unless an explicit field list is
> specified
>
> *if* (returnOnlyStored && sf != *null* && !sf.stored()) {
>
> *continue*;
>
> }
>
>
>
> Ps: initially while writing down the steps, when stored=false is not
> explicitly set in the field, it took the defaultvalue true and returned the
> fields.
>
>
>
> Karthik
>
>
>
> On Fri, Jul 6, 2018 at 4:05 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
> > Do you have any evidence that this doesn't work? Because when I tried
> > it just now it works fine. This was on master.
> >
> > Best,
> > Erick
> >
> > On Fri, Jul 6, 2018 at 10:06 AM, Karthikeyan Shanmugasundaram
> > <ks...@gmail.com> wrote:
> > > Hi,
> > >
> > > As part of this Jira,
> > >
> > > https://issues.apache.org/jira/browse/SOLR-2970?
> > attachmentSortBy=fileName
> > >
> > > CSVResponseWriter doesnt return fields whose stored attribute set to
> > false,
> > > but looks docvalues is not considered. Whereas it is not same for
> > response
> > > writer like json/xml ie., fields with only docvalued are retrievable.
> > >
> > >
> > > //Code snippet from CSVResponseWriter.java
> > >
> > > // Return only stored fields, unless an explicit field list is
> > > specified
> > > if (returnOnlyStored && sf != null && !sf.stored()) {
> > > continue;
> > > }
> > >
> > >
> > > SolrVersion: 7.2.1
> > >
> > >
> > > Please let me know, if any.
> > >
> > > Thanks,
> > > Karthik
> >
>
Re: CSVResponseWriter doesnt return non-stored field even when
docValues is enabled
Posted by Karthikeyan Shanmugasundaram <ks...@gmail.com>.
Yea i just tried again and below is the info,
- Solr/Lucene Version : 7.2.1
- Created collection with below fields
<field docValues="true" indexed="true" name="contentid" required="true"
type="string"/>
<field docValues="true" indexed="false" multiValued="false" name="testint"
type="int" stored="false"/>
<field docValues="true" indexed="false" multiValued="false" name="testlong"
type="plong" stored="false"/>
<fieldType class="solr.TrieIntField" name="int" positionIncrementGap="0"
precisionStep="0"/>
<fieldType name="plong" class="solr.LongPointField" docValues="true"/>
- Added few documents
contentid,testint,testlong
id,1,56
id2,2,66
- http://machine:port/solr/testdocvalue/select?q=*:*&wt=json
[{"contentid":"id",
"_version_":1605281886069850112,
"timestamp":"2018-07-06T22:28:25.335Z",
"testint":1,
"testlong":56},
{
"contentid":"id2",
"_version_":1605281886075092992,
"timestamp":"2018-07-06T22:28:25.335Z",
"testint":2,
"testlong":66}]
- http://machine:port/solr/testdocvalue/select?q=*:*&wt=csv
"_version_",contentid,timestamp
1605281886069850112,id,2018-07-06T22:28:25.335Z
1605281886075092992,id2,2018-07-06T22:28:25.335Z
- On attaching debugger, fields testint and testlong skipped adding to
csvFields list at below point since sf.stored = false and no explicit fl
was specified
// Return only stored fields, unless an explicit field list is
specified
*if* (returnOnlyStored && sf != *null* && !sf.stored()) {
*continue*;
}
Ps: initially while writing down the steps, when stored=false is not
explicitly set in the field, it took the defaultvalue true and returned the
fields.
Karthik
On Fri, Jul 6, 2018 at 4:05 PM, Erick Erickson <er...@gmail.com>
wrote:
> Do you have any evidence that this doesn't work? Because when I tried
> it just now it works fine. This was on master.
>
> Best,
> Erick
>
> On Fri, Jul 6, 2018 at 10:06 AM, Karthikeyan Shanmugasundaram
> <ks...@gmail.com> wrote:
> > Hi,
> >
> > As part of this Jira,
> >
> > https://issues.apache.org/jira/browse/SOLR-2970?
> attachmentSortBy=fileName
> >
> > CSVResponseWriter doesnt return fields whose stored attribute set to
> false,
> > but looks docvalues is not considered. Whereas it is not same for
> response
> > writer like json/xml ie., fields with only docvalued are retrievable.
> >
> >
> > //Code snippet from CSVResponseWriter.java
> >
> > // Return only stored fields, unless an explicit field list is
> > specified
> > if (returnOnlyStored && sf != null && !sf.stored()) {
> > continue;
> > }
> >
> >
> > SolrVersion: 7.2.1
> >
> >
> > Please let me know, if any.
> >
> > Thanks,
> > Karthik
>
Re: CSVResponseWriter doesnt return non-stored field even when
docValues is enabled
Posted by Erick Erickson <er...@gmail.com>.
Do you have any evidence that this doesn't work? Because when I tried
it just now it works fine. This was on master.
Best,
Erick
On Fri, Jul 6, 2018 at 10:06 AM, Karthikeyan Shanmugasundaram
<ks...@gmail.com> wrote:
> Hi,
>
> As part of this Jira,
>
> https://issues.apache.org/jira/browse/SOLR-2970?attachmentSortBy=fileName
>
> CSVResponseWriter doesnt return fields whose stored attribute set to false,
> but looks docvalues is not considered. Whereas it is not same for response
> writer like json/xml ie., fields with only docvalued are retrievable.
>
>
> //Code snippet from CSVResponseWriter.java
>
> // Return only stored fields, unless an explicit field list is
> specified
> if (returnOnlyStored && sf != null && !sf.stored()) {
> continue;
> }
>
>
> SolrVersion: 7.2.1
>
>
> Please let me know, if any.
>
> Thanks,
> Karthik