You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Hung Huynh <hh...@vinaconsulting.com> on 2008/04/04 15:25:57 UTC
why don't all stored fields show up?
I have about 20 stored fields in string, text, and int, but only about 10
fields show up when I query for them, whether I do fl=*,score or list them
out. What's my problem? How do I retrieve all of fields? Thanks.
Re: why don't all stored fields show up?
Posted by Yonik Seeley <yo...@apache.org>.
Could you try with a recent solr nightly build?
I just tried this with the example schema (I just changed guid to id)
and it worked fine.
http://localhost:8983/solr/select/?q=id%3A1&version=2.2&start=0&rows=10&indent=on&fl=*,score
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
<lst name="params">
<str name="rows">10</str>
<str name="start">0</str>
<str name="indent">on</str>
<str name="q">id:1</str>
<str name="fl">*,score</str>
<str name="version">2.2</str>
</lst>
</lst>
<result name="response" numFound="1" start="0" maxScore="3.0476928">
<doc>
<float name="score">3.0476928</float>
<str name="id">1</str>
<int name="popularity">0</int>
<arr name="sku"><str>1</str><str>ABC001</str></arr>
<date name="timestamp">2008-04-06T16:14:17.406Z</date>
</doc>
</result>
</response>
Some other things to check:
- make sure that sku is not defined more than once in your schema
- check your line endings on the text file... I'm not sure if CSV can
handle mac-style endings... only unix or windows.
- you can check the actual line endings by looking at the binary: od
-tx1 test1.txt
-Yonik
On Fri, Apr 4, 2008 at 3:40 PM, Hung Huynh <hh...@vinaconsulting.com> wrote:
> Thanks for spending time on this issue.
>
> I removed most the fields, and it's still not working:
>
> http://localhost:8983/solr/update/csv?commit=true&separator=|&escape=\&strea
> m.file=exampledocs/test1.txt
>
> test1.txt content
> guid|sku
> 1|ABC001
>
> Query:
> http://localhost:8983/solr/select/?q=guid%3A1&version=2.2&start=0&rows=10&in
> dent=on&fl=*,score
>
> output:
> <?xml version="1.0" encoding="UTF-8"?>
> <response>
>
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">0</int>
> <lst name="params">
> <str name="fl">*,score</str>
> <str name="indent">on</str>
> <str name="start">0</str>
> <str name="q">guid:1</str>
> <str name="version">2.2</str>
> <str name="rows">10</str>
> </lst>
> </lst>
> <result name="response" numFound="1" start="0" maxScore="0.71231794">
> <doc>
> <float name="score">0.71231794</float>
> <long name="guid">1</long>
> <date name="timestamp">2008-04-04T19:35:44.427Z</date>
> </doc>
> </result>
> </response>
>
> Schema:
>
> <field name="guid" type="slong" indexed="true" stored="true"
> required="true" />
> <field name="sku" type="string" indexed="true" stored="true"/>
>
> Guid is the unique numeric field.
>
> Thanks,
>
> Hung
>
>
> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
>
> Sent: Friday, April 04, 2008 12:02 PM
> To: solr-user@lucene.apache.org
> Subject: Re: why don't all stored fields show up?
>
>
>
> On Fri, Apr 4, 2008 at 11:57 AM, Hung Huynh <hh...@vinaconsulting.com> wrote:
> > Do you think it might be a problem with my schema and data loading?
>
> Maybe.
>
> > I loaded
> > CSV with 39 fields and didn't get any error message. I have a total of 39
> > stored fields, but not all of them are reported back when I query for
> them.
>
> Try to tackle it by getting more specific.
> Look at a single row in the CSV, and query for the id of that document
> in the index and see what's missing. Check the schema for those
> missing fields. Try to replicate the problem with another CSV file
> with just that single record.
>
> If you still can't figure it out, give us the following info:
> - the URL used to load the CSV data
> - the single record CSV file
> - the result of querying for that single record
> - your schema
>
> -Yonik
>
>
RE: why don't all stored fields show up?
Posted by Hung Huynh <hh...@vinaconsulting.com>.
Thanks for spending time on this issue.
I removed most the fields, and it's still not working:
http://localhost:8983/solr/update/csv?commit=true&separator=|&escape=\&strea
m.file=exampledocs/test1.txt
test1.txt content
guid|sku
1|ABC001
Query:
http://localhost:8983/solr/select/?q=guid%3A1&version=2.2&start=0&rows=10&in
dent=on&fl=*,score
output:
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
<lst name="params">
<str name="fl">*,score</str>
<str name="indent">on</str>
<str name="start">0</str>
<str name="q">guid:1</str>
<str name="version">2.2</str>
<str name="rows">10</str>
</lst>
</lst>
<result name="response" numFound="1" start="0" maxScore="0.71231794">
<doc>
<float name="score">0.71231794</float>
<long name="guid">1</long>
<date name="timestamp">2008-04-04T19:35:44.427Z</date>
</doc>
</result>
</response>
Schema:
<field name="guid" type="slong" indexed="true" stored="true"
required="true" />
<field name="sku" type="string" indexed="true" stored="true"/>
Guid is the unique numeric field.
Thanks,
Hung
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Friday, April 04, 2008 12:02 PM
To: solr-user@lucene.apache.org
Subject: Re: why don't all stored fields show up?
On Fri, Apr 4, 2008 at 11:57 AM, Hung Huynh <hh...@vinaconsulting.com> wrote:
> Do you think it might be a problem with my schema and data loading?
Maybe.
> I loaded
> CSV with 39 fields and didn't get any error message. I have a total of 39
> stored fields, but not all of them are reported back when I query for
them.
Try to tackle it by getting more specific.
Look at a single row in the CSV, and query for the id of that document
in the index and see what's missing. Check the schema for those
missing fields. Try to replicate the problem with another CSV file
with just that single record.
If you still can't figure it out, give us the following info:
- the URL used to load the CSV data
- the single record CSV file
- the result of querying for that single record
- your schema
-Yonik
Re: why don't all stored fields show up?
Posted by Yonik Seeley <yo...@apache.org>.
On Fri, Apr 4, 2008 at 11:57 AM, Hung Huynh <hh...@vinaconsulting.com> wrote:
> Do you think it might be a problem with my schema and data loading?
Maybe.
> I loaded
> CSV with 39 fields and didn't get any error message. I have a total of 39
> stored fields, but not all of them are reported back when I query for them.
Try to tackle it by getting more specific.
Look at a single row in the CSV, and query for the id of that document
in the index and see what's missing. Check the schema for those
missing fields. Try to replicate the problem with another CSV file
with just that single record.
If you still can't figure it out, give us the following info:
- the URL used to load the CSV data
- the single record CSV file
- the result of querying for that single record
- your schema
-Yonik
RE: why don't all stored fields show up?
Posted by Hung Huynh <hh...@vinaconsulting.com>.
Do you think it might be a problem with my schema and data loading? I loaded
CSV with 39 fields and didn't get any error message. I have a total of 39
stored fields, but not all of them are reported back when I query for them.
Should I reload the Index? Is there a way for me to check if the Index has
all 39 fields?
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Friday, April 04, 2008 10:48 AM
To: solr-user@lucene.apache.org
Subject: Re: why don't all stored fields show up?
On Fri, Apr 4, 2008 at 9:25 AM, Hung Huynh <hh...@vinaconsulting.com> wrote:
> I have about 20 stored fields in string, text, and int, but only about 10
> fields show up when I query for them, whether I do fl=*,score or list
them
> out. What's my problem? How do I retrieve all of fields? Thanks.
You should be getting back all stored fields for every document.
Documents will only show fields they have (fields are sparse, it's not
like a DB table).
-Yonik
Re: why don't all stored fields show up?
Posted by Yonik Seeley <yo...@apache.org>.
On Fri, Apr 4, 2008 at 9:25 AM, Hung Huynh <hh...@vinaconsulting.com> wrote:
> I have about 20 stored fields in string, text, and int, but only about 10
> fields show up when I query for them, whether I do fl=*,score or list them
> out. What's my problem? How do I retrieve all of fields? Thanks.
You should be getting back all stored fields for every document.
Documents will only show fields they have (fields are sparse, it's not
like a DB table).
-Yonik