You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Reik Schatz <re...@gmail.com> on 2012/11/16 11:13:46 UTC
admin query showing unstored fields
This might be a silly question but if I search *.* in the admin tool, how
can it show me the full document including all the fields that are set to
stored="false" or that don't have stored="true" at least. Where does Solr
get the information about the original content from? For Text fields
shouldn't it only keep the indexed terms.
Re: admin query showing unstored fields
Posted by Jack Krupansky <ja...@basetechnology.com>.
Also, go to the Schema Browser in Solr Admin and confirm that the "players"
field is listed as "Index: (unstored field)". Select a stored field just to
see how they should differ.
-- Jack Krupansky
-----Original Message-----
From: Jack Krupansky
Sent: Friday, November 16, 2012 8:09 AM
To: solr-user@lucene.apache.org
Subject: Re: admin query showing unstored fields
I just noticed that you are using an old schema version, 1.1. Any reason for
that? This suggests that you had an existing, old, Solr that you migrated to
4.0. What Solr release was it? Was the stored attribute working as you
expected before you upgraded to Solr 4.0? I don't know the specific
semantics of stored fields with such an old schema version, but I don't see
any doc that suggests that the stored attribute would behave differently.
I did just try your scenario in a fresh copy of the Solr 4.0 example and it
works as expected - "players" CANNOT be returned on a query. So, please try
your scenario on such a fresh copy of Solr 4.0 and see if you can reproduce
your symptom. Are there any other steps you took that led up to the
unexpected behavior?
The bottom line is that unless you did the two scenarios that Erik and I
suggested, or maybe Schema version 1.1 has some odd, undocumented quirks,
there must be some other steps you are performing that cause the unexpected
behavior.
-- Jack Krupansky
-----Original Message-----
From: Reik Schatz
Sent: Friday, November 16, 2012 3:02 AM
To: solr-user@lucene.apache.org
Subject: Re: admin query showing unstored fields
I did this test. Here is my schema.xml (setting stored="false" explicitly
though it should be default):
<schema name="minimal" version="1.1">
<types>
<fieldType name="string" class="solr.StrField" />
<fieldType name="score" class="solr.TrieFloatField"
precisionStep="32" omitNorms="true" omitTermFreqAndPositions="true" /> <!--
not optimized for RangeQueries -->
<fieldType name="text" class="solr.TextField">
<analyzer
class="org.apache.lucene.analysis.standard.StandardAnalyzer" />
</fieldType>
</types>
<fields>
<field name="id" type="string" indexed="true" required="true"
stored="false" />
<field name="players" type="text" indexed="true" stored="false" />
<dynamicField name="score_*" type="score" indexed="true"
stored="false" />
</fields>
<uniqueKey>id</uniqueKey>
<defaultSearchField>players</defaultSearchField>
<solrQueryParser defaultOperator="OR"/>
</schema>
I indexed a single document via the sol4j api:
// todo: make addings Fields extensible, i.e. by creating a
SolrInputDocumentBuilder class using multiple ReportToField subclasses
final SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", String.valueOf(UUID.randomUUID().toString()));
doc.addField("players", "One morning, when Gregor Samsa woke from
troubled dreams, he found himself transformed in his bed into a horrible
vermin. He lay on his armour-like back, and if he lifted his head a little
he could see his brown belly, slightly domed and divided by arches into
stiff sections. The bedding was hardly able to cover it and seemed ready to
slide off any moment. His many legs, pitifully thin compared with the size
of the rest of him, waved about helplessly as he looked. \"What's happened
to me? \" he thought. It wasn't a dream. His room, a proper human room
although a little too small, lay peacefully between its four familiar
walls. A collection of textile samples lay spread out on the table - Samsa
was a travelling salesman - and above it there hung a picture that he had
recently cut out of an illustrated magazine and housed in a nice, gilded
frame. It showed a lady fitted out with a fur hat and fur boa who sat
upright, raising a heavy fur muff that covered the whole of her lower arm
towards the viewer. Gregor then turned to look out the window at the dull
weather.");
// dynamic score field no shown here
return doc;
Then I went to http://localhost:8080/solr/#/history/query (my core being
called history) and ran a search for *:* which gives me:
{
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"indent":"true",
"q":"*:*",
"wt":"json"}},
"response":{"numFound":1,"start":0,"docs":[
{
"id":"670e976d-6b36-45fe-9f1a-38499cbd36f2",
"players":"One morning, when Gregor Samsa woke from troubled
dreams, he found himself transformed in his bed into a horrible
vermin. He lay on his armour-like back, and if he lifted his head a
little he could see his brown belly, slightly domed and divided by
arches into stiff sections. The bedding was hardly able to cover it
and seemed ready to slide off any moment. His many legs, pitifully
thin compared with the size of the rest of him, waved about helplessly
as he looked. \"What's happened to me? \" he thought. It wasn't a
dream. His room, a proper human room although a little too small, lay
peacefully between its four familiar walls. A collection of textile
samples lay spread out on the table - Samsa was a travelling salesman
- and above it there hung a picture that he had recently cut out of an
illustrated magazine and housed in a nice, gilded frame. It showed a
lady fitted out with a fur hat and fur boa who sat upright, raising a
heavy fur muff that covered the whole of her lower arm towards the
viewer. Gregor then turned to look out the window at the dull
weather.",
"score_344835693":1269.0,
"score_383098488":1423.0}]
}}
So how can it rebuild the contents of the "players" field if it is
stored="false"?
On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
> Er, it can't. What are you seeing that seems wrong?
>
> Upayavira
>
> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
> > This might be a silly question but if I search *.* in the admin tool,
> > how
> > can it show me the full document including all the fields that are set
> > to
> > stored="false" or that don't have stored="true" at least. Where does
> > Solr
> > get the information about the original content from? For Text fields
> > shouldn't it only keep the indexed terms.
>
Re: admin query showing unstored fields
Posted by Reik Schatz <re...@gmail.com>.
Hi,
yes all good thanks. It would be nice on this page:
http://wiki.apache.org/solr/SchemaXml#Fields under "Common field options"
and also "Data Types - common options" to specify what Solr will use as
default is if an attribute is not specified (i.e. field will be stored if
the stored attribute is not specified for a Field)
Reik
On Fri, Nov 16, 2012 at 8:18 PM, Jack Krupansky <ja...@basetechnology.com>wrote:
> So, you're all set?
>
> Maybe you could suggest where in the wiki the text could be clarified to
> help others avoid the same confusion.
>
>
> -- Jack Krupansky
>
> -----Original Message----- From: Reik Schatz
> Sent: Friday, November 16, 2012 8:35 AM
>
> To: solr-user@lucene.apache.org
> Subject: Re: admin query showing unstored fields
>
> Hi Jack,
>
> I just did some testing again and can confirm it works! I am new to Solr so
> by reading through http://wiki.apache.org/solr/**SchemaXml#Fields<http://wiki.apache.org/solr/SchemaXml#Fields>I was under
> the impression that not specifying the stored attribute at all, would
> default to stored="false" - which apparently is not the case. So running
> without the stored attribute in both the fieldType and the field element
> will store the contents. When I added stored="false" to either fieldType or
> the field element it worked. As for the 1.1 schema, I'am not upgrading from
> a previous version. I was looking for the most minimal configuration to get
> started with Solr, so I just copied some snippets from the web, which
> probably were older :)
>
>
> On Fri, Nov 16, 2012 at 5:09 PM, Jack Krupansky <ja...@basetechnology.com>*
> *wrote:
>
> I just noticed that you are using an old schema version, 1.1. Any reason
>> for that? This suggests that you had an existing, old, Solr that you
>> migrated to 4.0. What Solr release was it? Was the stored attribute
>> working
>> as you expected before you upgraded to Solr 4.0? I don't know the specific
>> semantics of stored fields with such an old schema version, but I don't
>> see
>> any doc that suggests that the stored attribute would behave differently.
>>
>> I did just try your scenario in a fresh copy of the Solr 4.0 example and
>> it works as expected - "players" CANNOT be returned on a query. So, please
>> try your scenario on such a fresh copy of Solr 4.0 and see if you can
>> reproduce your symptom. Are there any other steps you took that led up to
>> the unexpected behavior?
>>
>> The bottom line is that unless you did the two scenarios that Erik and I
>> suggested, or maybe Schema version 1.1 has some odd, undocumented quirks,
>> there must be some other steps you are performing that cause the
>> unexpected
>> behavior.
>>
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Reik Schatz
>> Sent: Friday, November 16, 2012 3:02 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: admin query showing unstored fields
>>
>> I did this test. Here is my schema.xml (setting stored="false" explicitly
>> though it should be default):
>>
>> <schema name="minimal" version="1.1">
>> <types>
>> <fieldType name="string" class="solr.StrField" />
>> <fieldType name="score" class="solr.TrieFloatField"
>> precisionStep="32" omitNorms="true" omitTermFreqAndPositions="****true"
>> />
>>
>> <!--
>>
>> not optimized for RangeQueries -->
>> <fieldType name="text" class="solr.TextField">
>> <analyzer
>> class="org.apache.lucene.****analysis.standard.****StandardAnalyzer" />
>>
>>
>> </fieldType>
>> </types>
>> <fields>
>> <field name="id" type="string" indexed="true" required="true"
>> stored="false" />
>> <field name="players" type="text" indexed="true" stored="false" />
>> <dynamicField name="score_*" type="score" indexed="true"
>> stored="false" />
>> </fields>
>> <uniqueKey>id</uniqueKey>
>> <defaultSearchField>players</****defaultSearchField>
>>
>>
>> <solrQueryParser defaultOperator="OR"/>
>> </schema>
>>
>> I indexed a single document via the sol4j api:
>>
>> // todo: make addings Fields extensible, i.e. by creating a
>> SolrInputDocumentBuilder class using multiple ReportToField subclasses
>> final SolrInputDocument doc = new SolrInputDocument();
>> doc.addField("id", String.valueOf(UUID.****
>> randomUUID().toString()));
>>
>>
>>
>> doc.addField("players", "One morning, when Gregor Samsa woke from
>> troubled dreams, he found himself transformed in his bed into a horrible
>> vermin. He lay on his armour-like back, and if he lifted his head a little
>> he could see his brown belly, slightly domed and divided by arches into
>> stiff sections. The bedding was hardly able to cover it and seemed ready
>> to
>> slide off any moment. His many legs, pitifully thin compared with the size
>> of the rest of him, waved about helplessly as he looked. \"What's happened
>> to me? \" he thought. It wasn't a dream. His room, a proper human room
>> although a little too small, lay peacefully between its four familiar
>> walls. A collection of textile samples lay spread out on the table - Samsa
>> was a travelling salesman - and above it there hung a picture that he had
>> recently cut out of an illustrated magazine and housed in a nice, gilded
>> frame. It showed a lady fitted out with a fur hat and fur boa who sat
>> upright, raising a heavy fur muff that covered the whole of her lower arm
>> towards the viewer. Gregor then turned to look out the window at the dull
>> weather.");
>>
>> // dynamic score field no shown here
>>
>> return doc;
>>
>>
>> Then I went to http://localhost:8080/solr/#/****history/query<http://localhost:8080/solr/#/**history/query>
>> <http://**localhost:8080/solr/#/history/**query<http://localhost:8080/solr/#/history/query>>(my
>> core being
>>
>>
>> called history) and ran a search for *:* which gives me:
>>
>> {
>> "responseHeader":{
>> "status":0,
>> "QTime":1,
>> "params":{
>> "indent":"true",
>> "q":"*:*",
>> "wt":"json"}},
>> "response":{"numFound":1,"****start":0,"docs":[
>> {
>> "id":"670e976d-6b36-45fe-9f1a-****38499cbd36f2",
>>
>>
>> "players":"One morning, when Gregor Samsa woke from troubled
>> dreams, he found himself transformed in his bed into a horrible
>> vermin. He lay on his armour-like back, and if he lifted his head a
>> little he could see his brown belly, slightly domed and divided by
>> arches into stiff sections. The bedding was hardly able to cover it
>> and seemed ready to slide off any moment. His many legs, pitifully
>> thin compared with the size of the rest of him, waved about helplessly
>> as he looked. \"What's happened to me? \" he thought. It wasn't a
>> dream. His room, a proper human room although a little too small, lay
>> peacefully between its four familiar walls. A collection of textile
>> samples lay spread out on the table - Samsa was a travelling salesman
>> - and above it there hung a picture that he had recently cut out of an
>> illustrated magazine and housed in a nice, gilded frame. It showed a
>> lady fitted out with a fur hat and fur boa who sat upright, raising a
>> heavy fur muff that covered the whole of her lower arm towards the
>> viewer. Gregor then turned to look out the window at the dull
>> weather.",
>> "score_344835693":1269.0,
>> "score_383098488":1423.0}]
>> }}
>>
>>
>> So how can it rebuild the contents of the "players" field if it is
>> stored="false"?
>>
>>
>>
>> On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
>>
>> Er, it can't. What are you seeing that seems wrong?
>>
>>>
>>> Upayavira
>>>
>>> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
>>> > This might be a silly question but if I search *.* in the admin tool, >
>>> how
>>> > can it show me the full document including all the fields that are set
>>> > to
>>> > stored="false" or that don't have stored="true" at least. Where does >
>>> Solr
>>> > get the information about the original content from? For Text fields
>>> > shouldn't it only keep the indexed terms.
>>>
>>>
>>>
>>
>
Re: admin query showing unstored fields
Posted by Jack Krupansky <ja...@basetechnology.com>.
So, you're all set?
Maybe you could suggest where in the wiki the text could be clarified to
help others avoid the same confusion.
-- Jack Krupansky
-----Original Message-----
From: Reik Schatz
Sent: Friday, November 16, 2012 8:35 AM
To: solr-user@lucene.apache.org
Subject: Re: admin query showing unstored fields
Hi Jack,
I just did some testing again and can confirm it works! I am new to Solr so
by reading through http://wiki.apache.org/solr/SchemaXml#Fields I was under
the impression that not specifying the stored attribute at all, would
default to stored="false" - which apparently is not the case. So running
without the stored attribute in both the fieldType and the field element
will store the contents. When I added stored="false" to either fieldType or
the field element it worked. As for the 1.1 schema, I'am not upgrading from
a previous version. I was looking for the most minimal configuration to get
started with Solr, so I just copied some snippets from the web, which
probably were older :)
On Fri, Nov 16, 2012 at 5:09 PM, Jack Krupansky
<ja...@basetechnology.com>wrote:
> I just noticed that you are using an old schema version, 1.1. Any reason
> for that? This suggests that you had an existing, old, Solr that you
> migrated to 4.0. What Solr release was it? Was the stored attribute
> working
> as you expected before you upgraded to Solr 4.0? I don't know the specific
> semantics of stored fields with such an old schema version, but I don't
> see
> any doc that suggests that the stored attribute would behave differently.
>
> I did just try your scenario in a fresh copy of the Solr 4.0 example and
> it works as expected - "players" CANNOT be returned on a query. So, please
> try your scenario on such a fresh copy of Solr 4.0 and see if you can
> reproduce your symptom. Are there any other steps you took that led up to
> the unexpected behavior?
>
> The bottom line is that unless you did the two scenarios that Erik and I
> suggested, or maybe Schema version 1.1 has some odd, undocumented quirks,
> there must be some other steps you are performing that cause the
> unexpected
> behavior.
>
>
> -- Jack Krupansky
>
> -----Original Message----- From: Reik Schatz
> Sent: Friday, November 16, 2012 3:02 AM
> To: solr-user@lucene.apache.org
> Subject: Re: admin query showing unstored fields
>
> I did this test. Here is my schema.xml (setting stored="false" explicitly
> though it should be default):
>
> <schema name="minimal" version="1.1">
> <types>
> <fieldType name="string" class="solr.StrField" />
> <fieldType name="score" class="solr.TrieFloatField"
> precisionStep="32" omitNorms="true" omitTermFreqAndPositions="**true" />
> <!--
>
> not optimized for RangeQueries -->
> <fieldType name="text" class="solr.TextField">
> <analyzer
> class="org.apache.lucene.**analysis.standard.**StandardAnalyzer" />
>
> </fieldType>
> </types>
> <fields>
> <field name="id" type="string" indexed="true" required="true"
> stored="false" />
> <field name="players" type="text" indexed="true" stored="false" />
> <dynamicField name="score_*" type="score" indexed="true"
> stored="false" />
> </fields>
> <uniqueKey>id</uniqueKey>
> <defaultSearchField>players</**defaultSearchField>
>
> <solrQueryParser defaultOperator="OR"/>
> </schema>
>
> I indexed a single document via the sol4j api:
>
> // todo: make addings Fields extensible, i.e. by creating a
> SolrInputDocumentBuilder class using multiple ReportToField subclasses
> final SolrInputDocument doc = new SolrInputDocument();
> doc.addField("id", String.valueOf(UUID.**randomUUID().toString()));
>
>
> doc.addField("players", "One morning, when Gregor Samsa woke from
> troubled dreams, he found himself transformed in his bed into a horrible
> vermin. He lay on his armour-like back, and if he lifted his head a little
> he could see his brown belly, slightly domed and divided by arches into
> stiff sections. The bedding was hardly able to cover it and seemed ready
> to
> slide off any moment. His many legs, pitifully thin compared with the size
> of the rest of him, waved about helplessly as he looked. \"What's happened
> to me? \" he thought. It wasn't a dream. His room, a proper human room
> although a little too small, lay peacefully between its four familiar
> walls. A collection of textile samples lay spread out on the table - Samsa
> was a travelling salesman - and above it there hung a picture that he had
> recently cut out of an illustrated magazine and housed in a nice, gilded
> frame. It showed a lady fitted out with a fur hat and fur boa who sat
> upright, raising a heavy fur muff that covered the whole of her lower arm
> towards the viewer. Gregor then turned to look out the window at the dull
> weather.");
>
> // dynamic score field no shown here
>
> return doc;
>
>
> Then I went to
> http://localhost:8080/solr/#/**history/query<http://localhost:8080/solr/#/history/query>(my
> core being
>
> called history) and ran a search for *:* which gives me:
>
> {
> "responseHeader":{
> "status":0,
> "QTime":1,
> "params":{
> "indent":"true",
> "q":"*:*",
> "wt":"json"}},
> "response":{"numFound":1,"**start":0,"docs":[
> {
> "id":"670e976d-6b36-45fe-9f1a-**38499cbd36f2",
>
> "players":"One morning, when Gregor Samsa woke from troubled
> dreams, he found himself transformed in his bed into a horrible
> vermin. He lay on his armour-like back, and if he lifted his head a
> little he could see his brown belly, slightly domed and divided by
> arches into stiff sections. The bedding was hardly able to cover it
> and seemed ready to slide off any moment. His many legs, pitifully
> thin compared with the size of the rest of him, waved about helplessly
> as he looked. \"What's happened to me? \" he thought. It wasn't a
> dream. His room, a proper human room although a little too small, lay
> peacefully between its four familiar walls. A collection of textile
> samples lay spread out on the table - Samsa was a travelling salesman
> - and above it there hung a picture that he had recently cut out of an
> illustrated magazine and housed in a nice, gilded frame. It showed a
> lady fitted out with a fur hat and fur boa who sat upright, raising a
> heavy fur muff that covered the whole of her lower arm towards the
> viewer. Gregor then turned to look out the window at the dull
> weather.",
> "score_344835693":1269.0,
> "score_383098488":1423.0}]
> }}
>
>
> So how can it rebuild the contents of the "players" field if it is
> stored="false"?
>
>
>
> On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
>
> Er, it can't. What are you seeing that seems wrong?
>>
>> Upayavira
>>
>> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
>> > This might be a silly question but if I search *.* in the admin tool, >
>> how
>> > can it show me the full document including all the fields that are set
>> > to
>> > stored="false" or that don't have stored="true" at least. Where does >
>> Solr
>> > get the information about the original content from? For Text fields
>> > shouldn't it only keep the indexed terms.
>>
>>
>
Re: admin query showing unstored fields
Posted by Reik Schatz <re...@gmail.com>.
Hi Jack,
I just did some testing again and can confirm it works! I am new to Solr so
by reading through http://wiki.apache.org/solr/SchemaXml#Fields I was under
the impression that not specifying the stored attribute at all, would
default to stored="false" - which apparently is not the case. So running
without the stored attribute in both the fieldType and the field element
will store the contents. When I added stored="false" to either fieldType or
the field element it worked. As for the 1.1 schema, I'am not upgrading from
a previous version. I was looking for the most minimal configuration to get
started with Solr, so I just copied some snippets from the web, which
probably were older :)
On Fri, Nov 16, 2012 at 5:09 PM, Jack Krupansky <ja...@basetechnology.com>wrote:
> I just noticed that you are using an old schema version, 1.1. Any reason
> for that? This suggests that you had an existing, old, Solr that you
> migrated to 4.0. What Solr release was it? Was the stored attribute working
> as you expected before you upgraded to Solr 4.0? I don't know the specific
> semantics of stored fields with such an old schema version, but I don't see
> any doc that suggests that the stored attribute would behave differently.
>
> I did just try your scenario in a fresh copy of the Solr 4.0 example and
> it works as expected - "players" CANNOT be returned on a query. So, please
> try your scenario on such a fresh copy of Solr 4.0 and see if you can
> reproduce your symptom. Are there any other steps you took that led up to
> the unexpected behavior?
>
> The bottom line is that unless you did the two scenarios that Erik and I
> suggested, or maybe Schema version 1.1 has some odd, undocumented quirks,
> there must be some other steps you are performing that cause the unexpected
> behavior.
>
>
> -- Jack Krupansky
>
> -----Original Message----- From: Reik Schatz
> Sent: Friday, November 16, 2012 3:02 AM
> To: solr-user@lucene.apache.org
> Subject: Re: admin query showing unstored fields
>
> I did this test. Here is my schema.xml (setting stored="false" explicitly
> though it should be default):
>
> <schema name="minimal" version="1.1">
> <types>
> <fieldType name="string" class="solr.StrField" />
> <fieldType name="score" class="solr.TrieFloatField"
> precisionStep="32" omitNorms="true" omitTermFreqAndPositions="**true" />
> <!--
>
> not optimized for RangeQueries -->
> <fieldType name="text" class="solr.TextField">
> <analyzer
> class="org.apache.lucene.**analysis.standard.**StandardAnalyzer" />
>
> </fieldType>
> </types>
> <fields>
> <field name="id" type="string" indexed="true" required="true"
> stored="false" />
> <field name="players" type="text" indexed="true" stored="false" />
> <dynamicField name="score_*" type="score" indexed="true"
> stored="false" />
> </fields>
> <uniqueKey>id</uniqueKey>
> <defaultSearchField>players</**defaultSearchField>
>
> <solrQueryParser defaultOperator="OR"/>
> </schema>
>
> I indexed a single document via the sol4j api:
>
> // todo: make addings Fields extensible, i.e. by creating a
> SolrInputDocumentBuilder class using multiple ReportToField subclasses
> final SolrInputDocument doc = new SolrInputDocument();
> doc.addField("id", String.valueOf(UUID.**randomUUID().toString()));
>
>
> doc.addField("players", "One morning, when Gregor Samsa woke from
> troubled dreams, he found himself transformed in his bed into a horrible
> vermin. He lay on his armour-like back, and if he lifted his head a little
> he could see his brown belly, slightly domed and divided by arches into
> stiff sections. The bedding was hardly able to cover it and seemed ready to
> slide off any moment. His many legs, pitifully thin compared with the size
> of the rest of him, waved about helplessly as he looked. \"What's happened
> to me? \" he thought. It wasn't a dream. His room, a proper human room
> although a little too small, lay peacefully between its four familiar
> walls. A collection of textile samples lay spread out on the table - Samsa
> was a travelling salesman - and above it there hung a picture that he had
> recently cut out of an illustrated magazine and housed in a nice, gilded
> frame. It showed a lady fitted out with a fur hat and fur boa who sat
> upright, raising a heavy fur muff that covered the whole of her lower arm
> towards the viewer. Gregor then turned to look out the window at the dull
> weather.");
>
> // dynamic score field no shown here
>
> return doc;
>
>
> Then I went to http://localhost:8080/solr/#/**history/query<http://localhost:8080/solr/#/history/query>(my core being
>
> called history) and ran a search for *:* which gives me:
>
> {
> "responseHeader":{
> "status":0,
> "QTime":1,
> "params":{
> "indent":"true",
> "q":"*:*",
> "wt":"json"}},
> "response":{"numFound":1,"**start":0,"docs":[
> {
> "id":"670e976d-6b36-45fe-9f1a-**38499cbd36f2",
>
> "players":"One morning, when Gregor Samsa woke from troubled
> dreams, he found himself transformed in his bed into a horrible
> vermin. He lay on his armour-like back, and if he lifted his head a
> little he could see his brown belly, slightly domed and divided by
> arches into stiff sections. The bedding was hardly able to cover it
> and seemed ready to slide off any moment. His many legs, pitifully
> thin compared with the size of the rest of him, waved about helplessly
> as he looked. \"What's happened to me? \" he thought. It wasn't a
> dream. His room, a proper human room although a little too small, lay
> peacefully between its four familiar walls. A collection of textile
> samples lay spread out on the table - Samsa was a travelling salesman
> - and above it there hung a picture that he had recently cut out of an
> illustrated magazine and housed in a nice, gilded frame. It showed a
> lady fitted out with a fur hat and fur boa who sat upright, raising a
> heavy fur muff that covered the whole of her lower arm towards the
> viewer. Gregor then turned to look out the window at the dull
> weather.",
> "score_344835693":1269.0,
> "score_383098488":1423.0}]
> }}
>
>
> So how can it rebuild the contents of the "players" field if it is
> stored="false"?
>
>
>
> On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
>
> Er, it can't. What are you seeing that seems wrong?
>>
>> Upayavira
>>
>> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
>> > This might be a silly question but if I search *.* in the admin tool, >
>> how
>> > can it show me the full document including all the fields that are set
>> > to
>> > stored="false" or that don't have stored="true" at least. Where does >
>> Solr
>> > get the information about the original content from? For Text fields
>> > shouldn't it only keep the indexed terms.
>>
>>
>
Re: admin query showing unstored fields
Posted by Jack Krupansky <ja...@basetechnology.com>.
I just noticed that you are using an old schema version, 1.1. Any reason for
that? This suggests that you had an existing, old, Solr that you migrated to
4.0. What Solr release was it? Was the stored attribute working as you
expected before you upgraded to Solr 4.0? I don't know the specific
semantics of stored fields with such an old schema version, but I don't see
any doc that suggests that the stored attribute would behave differently.
I did just try your scenario in a fresh copy of the Solr 4.0 example and it
works as expected - "players" CANNOT be returned on a query. So, please try
your scenario on such a fresh copy of Solr 4.0 and see if you can reproduce
your symptom. Are there any other steps you took that led up to the
unexpected behavior?
The bottom line is that unless you did the two scenarios that Erik and I
suggested, or maybe Schema version 1.1 has some odd, undocumented quirks,
there must be some other steps you are performing that cause the unexpected
behavior.
-- Jack Krupansky
-----Original Message-----
From: Reik Schatz
Sent: Friday, November 16, 2012 3:02 AM
To: solr-user@lucene.apache.org
Subject: Re: admin query showing unstored fields
I did this test. Here is my schema.xml (setting stored="false" explicitly
though it should be default):
<schema name="minimal" version="1.1">
<types>
<fieldType name="string" class="solr.StrField" />
<fieldType name="score" class="solr.TrieFloatField"
precisionStep="32" omitNorms="true" omitTermFreqAndPositions="true" /> <!--
not optimized for RangeQueries -->
<fieldType name="text" class="solr.TextField">
<analyzer
class="org.apache.lucene.analysis.standard.StandardAnalyzer" />
</fieldType>
</types>
<fields>
<field name="id" type="string" indexed="true" required="true"
stored="false" />
<field name="players" type="text" indexed="true" stored="false" />
<dynamicField name="score_*" type="score" indexed="true"
stored="false" />
</fields>
<uniqueKey>id</uniqueKey>
<defaultSearchField>players</defaultSearchField>
<solrQueryParser defaultOperator="OR"/>
</schema>
I indexed a single document via the sol4j api:
// todo: make addings Fields extensible, i.e. by creating a
SolrInputDocumentBuilder class using multiple ReportToField subclasses
final SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", String.valueOf(UUID.randomUUID().toString()));
doc.addField("players", "One morning, when Gregor Samsa woke from
troubled dreams, he found himself transformed in his bed into a horrible
vermin. He lay on his armour-like back, and if he lifted his head a little
he could see his brown belly, slightly domed and divided by arches into
stiff sections. The bedding was hardly able to cover it and seemed ready to
slide off any moment. His many legs, pitifully thin compared with the size
of the rest of him, waved about helplessly as he looked. \"What's happened
to me? \" he thought. It wasn't a dream. His room, a proper human room
although a little too small, lay peacefully between its four familiar
walls. A collection of textile samples lay spread out on the table - Samsa
was a travelling salesman - and above it there hung a picture that he had
recently cut out of an illustrated magazine and housed in a nice, gilded
frame. It showed a lady fitted out with a fur hat and fur boa who sat
upright, raising a heavy fur muff that covered the whole of her lower arm
towards the viewer. Gregor then turned to look out the window at the dull
weather.");
// dynamic score field no shown here
return doc;
Then I went to http://localhost:8080/solr/#/history/query (my core being
called history) and ran a search for *:* which gives me:
{
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"indent":"true",
"q":"*:*",
"wt":"json"}},
"response":{"numFound":1,"start":0,"docs":[
{
"id":"670e976d-6b36-45fe-9f1a-38499cbd36f2",
"players":"One morning, when Gregor Samsa woke from troubled
dreams, he found himself transformed in his bed into a horrible
vermin. He lay on his armour-like back, and if he lifted his head a
little he could see his brown belly, slightly domed and divided by
arches into stiff sections. The bedding was hardly able to cover it
and seemed ready to slide off any moment. His many legs, pitifully
thin compared with the size of the rest of him, waved about helplessly
as he looked. \"What's happened to me? \" he thought. It wasn't a
dream. His room, a proper human room although a little too small, lay
peacefully between its four familiar walls. A collection of textile
samples lay spread out on the table - Samsa was a travelling salesman
- and above it there hung a picture that he had recently cut out of an
illustrated magazine and housed in a nice, gilded frame. It showed a
lady fitted out with a fur hat and fur boa who sat upright, raising a
heavy fur muff that covered the whole of her lower arm towards the
viewer. Gregor then turned to look out the window at the dull
weather.",
"score_344835693":1269.0,
"score_383098488":1423.0}]
}}
So how can it rebuild the contents of the "players" field if it is
stored="false"?
On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
> Er, it can't. What are you seeing that seems wrong?
>
> Upayavira
>
> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
> > This might be a silly question but if I search *.* in the admin tool,
> > how
> > can it show me the full document including all the fields that are set
> > to
> > stored="false" or that don't have stored="true" at least. Where does
> > Solr
> > get the information about the original content from? For Text fields
> > shouldn't it only keep the indexed terms.
>
Re: admin query showing unstored fields
Posted by Reik Schatz <re...@gmail.com>.
Erik, yes I did. I am stopping Tomcat, rm the index dir inside the data
folder. Change the schema.xml inside the conf dir. Restart Tomcat and run
my little class which adds a single SolrInputDocument to my core. Then I
check via admin. So I take it that it shouldn't be displaying the contents
of a field set to stored="false"?
On Fri, Nov 16, 2012 at 4:08 PM, Erik Hatcher <er...@gmail.com>wrote:
> Did you also restart Solr after changing things?
>
> On Nov 16, 2012, at 10:04, Reik Schatz <re...@gmail.com> wrote:
>
> > I am using Solr 4.0 (new admin interface) and I am sure I don't have
> > anything left in my index because I empty the data directory every time
> > before testing.
> >
> >
> > On Fri, Nov 16, 2012 at 3:39 PM, Jack Krupansky <jack@basetechnology.com
> >wrote:
> >
> >> Is there any chance that you had added the document and then changed the
> >> schema to have stored="false"? Changing the schema doesn't affect the
> >> existing index/stored values.
> >>
> >> Also, what release are you using?
> >>
> >> -- Jack Krupansky
> >>
> >> -----Original Message----- From: Reik Schatz
> >> Sent: Friday, November 16, 2012 3:02 AM
> >> To: solr-user@lucene.apache.org
> >> Subject: Re: admin query showing unstored fields
> >>
> >> I did this test. Here is my schema.xml (setting stored="false"
> explicitly
> >> though it should be default):
> >>
> >> <schema name="minimal" version="1.1">
> >> <types>
> >> <fieldType name="string" class="solr.StrField" />
> >> <fieldType name="score" class="solr.TrieFloatField"
> >> precisionStep="32" omitNorms="true" omitTermFreqAndPositions="**true" />
> >> <!--
> >> not optimized for RangeQueries -->
> >> <fieldType name="text" class="solr.TextField">
> >> <analyzer
> >> class="org.apache.lucene.**analysis.standard.**StandardAnalyzer" />
> >> </fieldType>
> >> </types>
> >> <fields>
> >> <field name="id" type="string" indexed="true" required="true"
> >> stored="false" />
> >> <field name="players" type="text" indexed="true" stored="false" />
> >> <dynamicField name="score_*" type="score" indexed="true"
> >> stored="false" />
> >> </fields>
> >> <uniqueKey>id</uniqueKey>
> >> <defaultSearchField>players</**defaultSearchField>
> >> <solrQueryParser defaultOperator="OR"/>
> >> </schema>
> >>
> >> I indexed a single document via the sol4j api:
> >>
> >> // todo: make addings Fields extensible, i.e. by creating a
> >> SolrInputDocumentBuilder class using multiple ReportToField subclasses
> >> final SolrInputDocument doc = new SolrInputDocument();
> >> doc.addField("id",
> String.valueOf(UUID.**randomUUID().toString()));
> >>
> >> doc.addField("players", "One morning, when Gregor Samsa woke from
> >> troubled dreams, he found himself transformed in his bed into a horrible
> >> vermin. He lay on his armour-like back, and if he lifted his head a
> little
> >> he could see his brown belly, slightly domed and divided by arches into
> >> stiff sections. The bedding was hardly able to cover it and seemed
> ready to
> >> slide off any moment. His many legs, pitifully thin compared with the
> size
> >> of the rest of him, waved about helplessly as he looked. \"What's
> happened
> >> to me? \" he thought. It wasn't a dream. His room, a proper human room
> >> although a little too small, lay peacefully between its four familiar
> >> walls. A collection of textile samples lay spread out on the table -
> Samsa
> >> was a travelling salesman - and above it there hung a picture that he
> had
> >> recently cut out of an illustrated magazine and housed in a nice, gilded
> >> frame. It showed a lady fitted out with a fur hat and fur boa who sat
> >> upright, raising a heavy fur muff that covered the whole of her lower
> arm
> >> towards the viewer. Gregor then turned to look out the window at the
> dull
> >> weather.");
> >>
> >> // dynamic score field no shown here
> >>
> >> return doc;
> >>
> >>
> >> Then I went to http://localhost:8080/solr/#/**history/query<
> http://localhost:8080/solr/#/history/query>(my core being
> >> called history) and ran a search for *:* which gives me:
> >>
> >> {
> >> "responseHeader":{
> >> "status":0,
> >> "QTime":1,
> >> "params":{
> >> "indent":"true",
> >> "q":"*:*",
> >> "wt":"json"}},
> >> "response":{"numFound":1,"**start":0,"docs":[
> >> {
> >> "id":"670e976d-6b36-45fe-9f1a-**38499cbd36f2",
> >> "players":"One morning, when Gregor Samsa woke from troubled
> >> dreams, he found himself transformed in his bed into a horrible
> >> vermin. He lay on his armour-like back, and if he lifted his head a
> >> little he could see his brown belly, slightly domed and divided by
> >> arches into stiff sections. The bedding was hardly able to cover it
> >> and seemed ready to slide off any moment. His many legs, pitifully
> >> thin compared with the size of the rest of him, waved about helplessly
> >> as he looked. \"What's happened to me? \" he thought. It wasn't a
> >> dream. His room, a proper human room although a little too small, lay
> >> peacefully between its four familiar walls. A collection of textile
> >> samples lay spread out on the table - Samsa was a travelling salesman
> >> - and above it there hung a picture that he had recently cut out of an
> >> illustrated magazine and housed in a nice, gilded frame. It showed a
> >> lady fitted out with a fur hat and fur boa who sat upright, raising a
> >> heavy fur muff that covered the whole of her lower arm towards the
> >> viewer. Gregor then turned to look out the window at the dull
> >> weather.",
> >> "score_344835693":1269.0,
> >> "score_383098488":1423.0}]
> >> }}
> >>
> >>
> >> So how can it rebuild the contents of the "players" field if it is
> >> stored="false"?
> >>
> >>
> >>
> >> On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
> >>
> >> Er, it can't. What are you seeing that seems wrong?
> >>>
> >>> Upayavira
> >>>
> >>> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
> >>>> This might be a silly question but if I search *.* in the admin tool,
> >
> >>> how
> >>>> can it show me the full document including all the fields that are set
> >>>> to
> >>>> stored="false" or that don't have stored="true" at least. Where does >
> >>> Solr
> >>>> get the information about the original content from? For Text fields
> >>>> shouldn't it only keep the indexed terms.
> >>
>
Re: admin query showing unstored fields
Posted by Erik Hatcher <er...@gmail.com>.
Did you also restart Solr after changing things?
On Nov 16, 2012, at 10:04, Reik Schatz <re...@gmail.com> wrote:
> I am using Solr 4.0 (new admin interface) and I am sure I don't have
> anything left in my index because I empty the data directory every time
> before testing.
>
>
> On Fri, Nov 16, 2012 at 3:39 PM, Jack Krupansky <ja...@basetechnology.com>wrote:
>
>> Is there any chance that you had added the document and then changed the
>> schema to have stored="false"? Changing the schema doesn't affect the
>> existing index/stored values.
>>
>> Also, what release are you using?
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Reik Schatz
>> Sent: Friday, November 16, 2012 3:02 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: admin query showing unstored fields
>>
>> I did this test. Here is my schema.xml (setting stored="false" explicitly
>> though it should be default):
>>
>> <schema name="minimal" version="1.1">
>> <types>
>> <fieldType name="string" class="solr.StrField" />
>> <fieldType name="score" class="solr.TrieFloatField"
>> precisionStep="32" omitNorms="true" omitTermFreqAndPositions="**true" />
>> <!--
>> not optimized for RangeQueries -->
>> <fieldType name="text" class="solr.TextField">
>> <analyzer
>> class="org.apache.lucene.**analysis.standard.**StandardAnalyzer" />
>> </fieldType>
>> </types>
>> <fields>
>> <field name="id" type="string" indexed="true" required="true"
>> stored="false" />
>> <field name="players" type="text" indexed="true" stored="false" />
>> <dynamicField name="score_*" type="score" indexed="true"
>> stored="false" />
>> </fields>
>> <uniqueKey>id</uniqueKey>
>> <defaultSearchField>players</**defaultSearchField>
>> <solrQueryParser defaultOperator="OR"/>
>> </schema>
>>
>> I indexed a single document via the sol4j api:
>>
>> // todo: make addings Fields extensible, i.e. by creating a
>> SolrInputDocumentBuilder class using multiple ReportToField subclasses
>> final SolrInputDocument doc = new SolrInputDocument();
>> doc.addField("id", String.valueOf(UUID.**randomUUID().toString()));
>>
>> doc.addField("players", "One morning, when Gregor Samsa woke from
>> troubled dreams, he found himself transformed in his bed into a horrible
>> vermin. He lay on his armour-like back, and if he lifted his head a little
>> he could see his brown belly, slightly domed and divided by arches into
>> stiff sections. The bedding was hardly able to cover it and seemed ready to
>> slide off any moment. His many legs, pitifully thin compared with the size
>> of the rest of him, waved about helplessly as he looked. \"What's happened
>> to me? \" he thought. It wasn't a dream. His room, a proper human room
>> although a little too small, lay peacefully between its four familiar
>> walls. A collection of textile samples lay spread out on the table - Samsa
>> was a travelling salesman - and above it there hung a picture that he had
>> recently cut out of an illustrated magazine and housed in a nice, gilded
>> frame. It showed a lady fitted out with a fur hat and fur boa who sat
>> upright, raising a heavy fur muff that covered the whole of her lower arm
>> towards the viewer. Gregor then turned to look out the window at the dull
>> weather.");
>>
>> // dynamic score field no shown here
>>
>> return doc;
>>
>>
>> Then I went to http://localhost:8080/solr/#/**history/query<http://localhost:8080/solr/#/history/query>(my core being
>> called history) and ran a search for *:* which gives me:
>>
>> {
>> "responseHeader":{
>> "status":0,
>> "QTime":1,
>> "params":{
>> "indent":"true",
>> "q":"*:*",
>> "wt":"json"}},
>> "response":{"numFound":1,"**start":0,"docs":[
>> {
>> "id":"670e976d-6b36-45fe-9f1a-**38499cbd36f2",
>> "players":"One morning, when Gregor Samsa woke from troubled
>> dreams, he found himself transformed in his bed into a horrible
>> vermin. He lay on his armour-like back, and if he lifted his head a
>> little he could see his brown belly, slightly domed and divided by
>> arches into stiff sections. The bedding was hardly able to cover it
>> and seemed ready to slide off any moment. His many legs, pitifully
>> thin compared with the size of the rest of him, waved about helplessly
>> as he looked. \"What's happened to me? \" he thought. It wasn't a
>> dream. His room, a proper human room although a little too small, lay
>> peacefully between its four familiar walls. A collection of textile
>> samples lay spread out on the table - Samsa was a travelling salesman
>> - and above it there hung a picture that he had recently cut out of an
>> illustrated magazine and housed in a nice, gilded frame. It showed a
>> lady fitted out with a fur hat and fur boa who sat upright, raising a
>> heavy fur muff that covered the whole of her lower arm towards the
>> viewer. Gregor then turned to look out the window at the dull
>> weather.",
>> "score_344835693":1269.0,
>> "score_383098488":1423.0}]
>> }}
>>
>>
>> So how can it rebuild the contents of the "players" field if it is
>> stored="false"?
>>
>>
>>
>> On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
>>
>> Er, it can't. What are you seeing that seems wrong?
>>>
>>> Upayavira
>>>
>>> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
>>>> This might be a silly question but if I search *.* in the admin tool, >
>>> how
>>>> can it show me the full document including all the fields that are set
>>>> to
>>>> stored="false" or that don't have stored="true" at least. Where does >
>>> Solr
>>>> get the information about the original content from? For Text fields
>>>> shouldn't it only keep the indexed terms.
>>
Re: admin query showing unstored fields
Posted by Reik Schatz <re...@gmail.com>.
I am using Solr 4.0 (new admin interface) and I am sure I don't have
anything left in my index because I empty the data directory every time
before testing.
On Fri, Nov 16, 2012 at 3:39 PM, Jack Krupansky <ja...@basetechnology.com>wrote:
> Is there any chance that you had added the document and then changed the
> schema to have stored="false"? Changing the schema doesn't affect the
> existing index/stored values.
>
> Also, what release are you using?
>
> -- Jack Krupansky
>
> -----Original Message----- From: Reik Schatz
> Sent: Friday, November 16, 2012 3:02 AM
> To: solr-user@lucene.apache.org
> Subject: Re: admin query showing unstored fields
>
> I did this test. Here is my schema.xml (setting stored="false" explicitly
> though it should be default):
>
> <schema name="minimal" version="1.1">
> <types>
> <fieldType name="string" class="solr.StrField" />
> <fieldType name="score" class="solr.TrieFloatField"
> precisionStep="32" omitNorms="true" omitTermFreqAndPositions="**true" />
> <!--
> not optimized for RangeQueries -->
> <fieldType name="text" class="solr.TextField">
> <analyzer
> class="org.apache.lucene.**analysis.standard.**StandardAnalyzer" />
> </fieldType>
> </types>
> <fields>
> <field name="id" type="string" indexed="true" required="true"
> stored="false" />
> <field name="players" type="text" indexed="true" stored="false" />
> <dynamicField name="score_*" type="score" indexed="true"
> stored="false" />
> </fields>
> <uniqueKey>id</uniqueKey>
> <defaultSearchField>players</**defaultSearchField>
> <solrQueryParser defaultOperator="OR"/>
> </schema>
>
> I indexed a single document via the sol4j api:
>
> // todo: make addings Fields extensible, i.e. by creating a
> SolrInputDocumentBuilder class using multiple ReportToField subclasses
> final SolrInputDocument doc = new SolrInputDocument();
> doc.addField("id", String.valueOf(UUID.**randomUUID().toString()));
>
> doc.addField("players", "One morning, when Gregor Samsa woke from
> troubled dreams, he found himself transformed in his bed into a horrible
> vermin. He lay on his armour-like back, and if he lifted his head a little
> he could see his brown belly, slightly domed and divided by arches into
> stiff sections. The bedding was hardly able to cover it and seemed ready to
> slide off any moment. His many legs, pitifully thin compared with the size
> of the rest of him, waved about helplessly as he looked. \"What's happened
> to me? \" he thought. It wasn't a dream. His room, a proper human room
> although a little too small, lay peacefully between its four familiar
> walls. A collection of textile samples lay spread out on the table - Samsa
> was a travelling salesman - and above it there hung a picture that he had
> recently cut out of an illustrated magazine and housed in a nice, gilded
> frame. It showed a lady fitted out with a fur hat and fur boa who sat
> upright, raising a heavy fur muff that covered the whole of her lower arm
> towards the viewer. Gregor then turned to look out the window at the dull
> weather.");
>
> // dynamic score field no shown here
>
> return doc;
>
>
> Then I went to http://localhost:8080/solr/#/**history/query<http://localhost:8080/solr/#/history/query>(my core being
> called history) and ran a search for *:* which gives me:
>
> {
> "responseHeader":{
> "status":0,
> "QTime":1,
> "params":{
> "indent":"true",
> "q":"*:*",
> "wt":"json"}},
> "response":{"numFound":1,"**start":0,"docs":[
> {
> "id":"670e976d-6b36-45fe-9f1a-**38499cbd36f2",
> "players":"One morning, when Gregor Samsa woke from troubled
> dreams, he found himself transformed in his bed into a horrible
> vermin. He lay on his armour-like back, and if he lifted his head a
> little he could see his brown belly, slightly domed and divided by
> arches into stiff sections. The bedding was hardly able to cover it
> and seemed ready to slide off any moment. His many legs, pitifully
> thin compared with the size of the rest of him, waved about helplessly
> as he looked. \"What's happened to me? \" he thought. It wasn't a
> dream. His room, a proper human room although a little too small, lay
> peacefully between its four familiar walls. A collection of textile
> samples lay spread out on the table - Samsa was a travelling salesman
> - and above it there hung a picture that he had recently cut out of an
> illustrated magazine and housed in a nice, gilded frame. It showed a
> lady fitted out with a fur hat and fur boa who sat upright, raising a
> heavy fur muff that covered the whole of her lower arm towards the
> viewer. Gregor then turned to look out the window at the dull
> weather.",
> "score_344835693":1269.0,
> "score_383098488":1423.0}]
> }}
>
>
> So how can it rebuild the contents of the "players" field if it is
> stored="false"?
>
>
>
> On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
>
> Er, it can't. What are you seeing that seems wrong?
>>
>> Upayavira
>>
>> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
>> > This might be a silly question but if I search *.* in the admin tool, >
>> how
>> > can it show me the full document including all the fields that are set
>> > to
>> > stored="false" or that don't have stored="true" at least. Where does >
>> Solr
>> > get the information about the original content from? For Text fields
>> > shouldn't it only keep the indexed terms.
>>
>>
>
Re: admin query showing unstored fields
Posted by Jack Krupansky <ja...@basetechnology.com>.
Is there any chance that you had added the document and then changed the
schema to have stored="false"? Changing the schema doesn't affect the
existing index/stored values.
Also, what release are you using?
-- Jack Krupansky
-----Original Message-----
From: Reik Schatz
Sent: Friday, November 16, 2012 3:02 AM
To: solr-user@lucene.apache.org
Subject: Re: admin query showing unstored fields
I did this test. Here is my schema.xml (setting stored="false" explicitly
though it should be default):
<schema name="minimal" version="1.1">
<types>
<fieldType name="string" class="solr.StrField" />
<fieldType name="score" class="solr.TrieFloatField"
precisionStep="32" omitNorms="true" omitTermFreqAndPositions="true" /> <!--
not optimized for RangeQueries -->
<fieldType name="text" class="solr.TextField">
<analyzer
class="org.apache.lucene.analysis.standard.StandardAnalyzer" />
</fieldType>
</types>
<fields>
<field name="id" type="string" indexed="true" required="true"
stored="false" />
<field name="players" type="text" indexed="true" stored="false" />
<dynamicField name="score_*" type="score" indexed="true"
stored="false" />
</fields>
<uniqueKey>id</uniqueKey>
<defaultSearchField>players</defaultSearchField>
<solrQueryParser defaultOperator="OR"/>
</schema>
I indexed a single document via the sol4j api:
// todo: make addings Fields extensible, i.e. by creating a
SolrInputDocumentBuilder class using multiple ReportToField subclasses
final SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", String.valueOf(UUID.randomUUID().toString()));
doc.addField("players", "One morning, when Gregor Samsa woke from
troubled dreams, he found himself transformed in his bed into a horrible
vermin. He lay on his armour-like back, and if he lifted his head a little
he could see his brown belly, slightly domed and divided by arches into
stiff sections. The bedding was hardly able to cover it and seemed ready to
slide off any moment. His many legs, pitifully thin compared with the size
of the rest of him, waved about helplessly as he looked. \"What's happened
to me? \" he thought. It wasn't a dream. His room, a proper human room
although a little too small, lay peacefully between its four familiar
walls. A collection of textile samples lay spread out on the table - Samsa
was a travelling salesman - and above it there hung a picture that he had
recently cut out of an illustrated magazine and housed in a nice, gilded
frame. It showed a lady fitted out with a fur hat and fur boa who sat
upright, raising a heavy fur muff that covered the whole of her lower arm
towards the viewer. Gregor then turned to look out the window at the dull
weather.");
// dynamic score field no shown here
return doc;
Then I went to http://localhost:8080/solr/#/history/query (my core being
called history) and ran a search for *:* which gives me:
{
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"indent":"true",
"q":"*:*",
"wt":"json"}},
"response":{"numFound":1,"start":0,"docs":[
{
"id":"670e976d-6b36-45fe-9f1a-38499cbd36f2",
"players":"One morning, when Gregor Samsa woke from troubled
dreams, he found himself transformed in his bed into a horrible
vermin. He lay on his armour-like back, and if he lifted his head a
little he could see his brown belly, slightly domed and divided by
arches into stiff sections. The bedding was hardly able to cover it
and seemed ready to slide off any moment. His many legs, pitifully
thin compared with the size of the rest of him, waved about helplessly
as he looked. \"What's happened to me? \" he thought. It wasn't a
dream. His room, a proper human room although a little too small, lay
peacefully between its four familiar walls. A collection of textile
samples lay spread out on the table - Samsa was a travelling salesman
- and above it there hung a picture that he had recently cut out of an
illustrated magazine and housed in a nice, gilded frame. It showed a
lady fitted out with a fur hat and fur boa who sat upright, raising a
heavy fur muff that covered the whole of her lower arm towards the
viewer. Gregor then turned to look out the window at the dull
weather.",
"score_344835693":1269.0,
"score_383098488":1423.0}]
}}
So how can it rebuild the contents of the "players" field if it is
stored="false"?
On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
> Er, it can't. What are you seeing that seems wrong?
>
> Upayavira
>
> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
> > This might be a silly question but if I search *.* in the admin tool,
> > how
> > can it show me the full document including all the fields that are set
> > to
> > stored="false" or that don't have stored="true" at least. Where does
> > Solr
> > get the information about the original content from? For Text fields
> > shouldn't it only keep the indexed terms.
>
Re: admin query showing unstored fields
Posted by Reik Schatz <re...@gmail.com>.
I did this test. Here is my schema.xml (setting stored="false" explicitly
though it should be default):
<schema name="minimal" version="1.1">
<types>
<fieldType name="string" class="solr.StrField" />
<fieldType name="score" class="solr.TrieFloatField"
precisionStep="32" omitNorms="true" omitTermFreqAndPositions="true" /> <!--
not optimized for RangeQueries -->
<fieldType name="text" class="solr.TextField">
<analyzer
class="org.apache.lucene.analysis.standard.StandardAnalyzer" />
</fieldType>
</types>
<fields>
<field name="id" type="string" indexed="true" required="true"
stored="false" />
<field name="players" type="text" indexed="true" stored="false" />
<dynamicField name="score_*" type="score" indexed="true"
stored="false" />
</fields>
<uniqueKey>id</uniqueKey>
<defaultSearchField>players</defaultSearchField>
<solrQueryParser defaultOperator="OR"/>
</schema>
I indexed a single document via the sol4j api:
// todo: make addings Fields extensible, i.e. by creating a
SolrInputDocumentBuilder class using multiple ReportToField subclasses
final SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", String.valueOf(UUID.randomUUID().toString()));
doc.addField("players", "One morning, when Gregor Samsa woke from
troubled dreams, he found himself transformed in his bed into a horrible
vermin. He lay on his armour-like back, and if he lifted his head a little
he could see his brown belly, slightly domed and divided by arches into
stiff sections. The bedding was hardly able to cover it and seemed ready to
slide off any moment. His many legs, pitifully thin compared with the size
of the rest of him, waved about helplessly as he looked. \"What's happened
to me? \" he thought. It wasn't a dream. His room, a proper human room
although a little too small, lay peacefully between its four familiar
walls. A collection of textile samples lay spread out on the table - Samsa
was a travelling salesman - and above it there hung a picture that he had
recently cut out of an illustrated magazine and housed in a nice, gilded
frame. It showed a lady fitted out with a fur hat and fur boa who sat
upright, raising a heavy fur muff that covered the whole of her lower arm
towards the viewer. Gregor then turned to look out the window at the dull
weather.");
// dynamic score field no shown here
return doc;
Then I went to http://localhost:8080/solr/#/history/query (my core being
called history) and ran a search for *:* which gives me:
{
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"indent":"true",
"q":"*:*",
"wt":"json"}},
"response":{"numFound":1,"start":0,"docs":[
{
"id":"670e976d-6b36-45fe-9f1a-38499cbd36f2",
"players":"One morning, when Gregor Samsa woke from troubled
dreams, he found himself transformed in his bed into a horrible
vermin. He lay on his armour-like back, and if he lifted his head a
little he could see his brown belly, slightly domed and divided by
arches into stiff sections. The bedding was hardly able to cover it
and seemed ready to slide off any moment. His many legs, pitifully
thin compared with the size of the rest of him, waved about helplessly
as he looked. \"What's happened to me? \" he thought. It wasn't a
dream. His room, a proper human room although a little too small, lay
peacefully between its four familiar walls. A collection of textile
samples lay spread out on the table - Samsa was a travelling salesman
- and above it there hung a picture that he had recently cut out of an
illustrated magazine and housed in a nice, gilded frame. It showed a
lady fitted out with a fur hat and fur boa who sat upright, raising a
heavy fur muff that covered the whole of her lower arm towards the
viewer. Gregor then turned to look out the window at the dull
weather.",
"score_344835693":1269.0,
"score_383098488":1423.0}]
}}
So how can it rebuild the contents of the "players" field if it is
stored="false"?
On Fri, Nov 16, 2012 at 11:52 AM, Upayavira <uv...@odoko.co.uk> wrote:
> Er, it can't. What are you seeing that seems wrong?
>
> Upayavira
>
> On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
> > This might be a silly question but if I search *.* in the admin tool, how
> > can it show me the full document including all the fields that are set to
> > stored="false" or that don't have stored="true" at least. Where does Solr
> > get the information about the original content from? For Text fields
> > shouldn't it only keep the indexed terms.
>
Re: admin query showing unstored fields
Posted by Upayavira <uv...@odoko.co.uk>.
Er, it can't. What are you seeing that seems wrong?
Upayavira
On Fri, Nov 16, 2012, at 10:13 AM, Reik Schatz wrote:
> This might be a silly question but if I search *.* in the admin tool, how
> can it show me the full document including all the fields that are set to
> stored="false" or that don't have stored="true" at least. Where does Solr
> get the information about the original content from? For Text fields
> shouldn't it only keep the indexed terms.