RE: Stored or indexed?

["Olson, Ron" <RO...@lbpc.com>]

Thanks for the great info! I appreciate everybody's help in getting started with Solr, hopefully I'll be able to get my stuff working and move on to more difficult questions. :)

-----Original Message-----
From: Elizabeth L. Murnane [mailto:emurnane@architexa.com]
Sent: Friday, October 29, 2010 12:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Stored or indexed?

Hi Ron,

In a nutshell - an indexed field is searchable, and a stored field has its content stored in the index so it is retrievable. Here are some examples that will hopefully give you a feel for how to set the indexed and stored options:

indexed="true" stored="true"
Use this for information you want to search on and also display in search results - for example, book title or author.

indexed="false" stored="true"
Use this for fields that you want displayed with search results but that don't need to be searchable - for example, destination URL, file system path, time stamp, or icon image.

indexed="true" stored="false"
Use this for fields you want to search on but don't need to get their values in search results. Here are some of the common reasons you would want this:

Large fields and a database: Storing a field makes your index larger, so set stored to false when possible, especially for big fields. For this case a database is often used, as the previous responder said. Use a separate identifier field to get the field's content from the database.

Ordering results: Say you define field name="bookName" type="text" indexed="true" stored="true" that is tokenized and used for searching. If you want to sort results based on book name, you could copy the field into a separate nonretrievable, nontokenized field that can be used just for sorting -
field name="bookSort" type="string" indexed="true" stored="false"
copyField source="bookName" dest="bookSort"

Easier searching: If you define the field <field name="text" type="text" indexed="true" stored="false" multiValued="true"/> you can use it as a catch-all field that contains all of the other text fields. Since solr looks in a default field when given a text query without field names, you can support this type of general phrase query by making the catch-all the default field.

indexed="false" stored="false"
Use this when you want to ignore fields. For example, the following will ignore unknown fields that don't match a defined field rather than throwing an error by default.
fieldtype name="ignored" stored="false" indexed="false"
dynamicField name="*" type="ignored"


Elizabeth Murnane
emurnane@architexa.com
Architexa Lead Developer - www.architexa.com
Understand & Document Code In Seconds


--- On Thu, 10/28/10, Savvas-Andreas Moysidis <sa...@googlemail.com> wrote:

From: Savvas-Andreas Moysidis <sa...@googlemail.com>
Subject: Re: Stored or indexed?
To: solr-user@lucene.apache.org
Date: Thursday, October 28, 2010, 4:25 AM

In our case, we just store a database id and do a secondary db query when
displaying the results.
This is handy and leads to a more centralised architecture when you need to
display properties of a domain object which you don't index/search.

On 28 October 2010 05:02, kenf_nc <ke...@realestate.com> wrote:

>
> Interesting wiki link, I hadn't seen that table before.
>
> And to answer your specific question about indexed=true, stored=false, this
> is most often done when you are using analyzers/tokenizers on your field.
> This field is for search only, you would never retrieve it's contents for
> display. It may in fact be an amalgam of several fields into one 'content'
> field. You have your display copy stored in another field marked
> indexed=false, stored=true and optionally compressed. I also have simple
> string fields set to lowercase so searching is case-insensitive, and have a
> duplicate field where the string is normal case. the first one is
> indexed/not stored, the second is stored/not indexed.
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Stored-or-indexed-tp1782805p1784315.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

DISCLAIMER: This electronic message, including any attachments, files or documents, is intended only for the addressee and may contain CONFIDENTIAL, PROPRIETARY or LEGALLY PRIVILEGED information.  If you are not the intended recipient, you are hereby notified that any use, disclosure, copying or distribution of this message or any of the information included in or with it is  unauthorized and strictly prohibited.  If you have received this message in error, please notify the sender immediately by reply e-mail and permanently delete and destroy this message and its attachments, along with any copies thereof. This message does not create any contractual obligation on behalf of the sender or Law Bulletin Publishing Company.
Thank you.

Re: Stored or indexed?

[Erick Erickson <er...@gmail.com>]

IMO, the very, very best way to increase your grasp of all things Solr is
to
try to answer questions on this list. Folks are pretty gentle about
correcting
mistaken posts. And I certainly remember any advice I've given that's been
corrected <G>.....

Besides, if you try to answer the things you *do* understand, it leave more
time for the committers to answer *your* questions <G>...

Best
Erick

On Tue, Nov 2, 2010 at 4:39 PM, Olson, Ron <RO...@lbpc.com> wrote:

> Thanks for the great info! I appreciate everybody's help in getting started
> with Solr, hopefully I'll be able to get my stuff working and move on to
> more difficult questions. :)
>
> -----Original Message-----
> From: Elizabeth L. Murnane [mailto:emurnane@architexa.com]
> Sent: Friday, October 29, 2010 12:42 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Stored or indexed?
>
> Hi Ron,
>
> In a nutshell - an indexed field is searchable, and a stored field has its
> content stored in the index so it is retrievable. Here are some examples
> that will hopefully give you a feel for how to set the indexed and stored
> options:
>
> indexed="true" stored="true"
> Use this for information you want to search on and also display in search
> results - for example, book title or author.
>
> indexed="false" stored="true"
> Use this for fields that you want displayed with search results but that
> don't need to be searchable - for example, destination URL, file system
> path, time stamp, or icon image.
>
> indexed="true" stored="false"
> Use this for fields you want to search on but don't need to get their
> values in search results. Here are some of the common reasons you would want
> this:
>
> Large fields and a database: Storing a field makes your index larger, so
> set stored to false when possible, especially for big fields. For this case
> a database is often used, as the previous responder said. Use a separate
> identifier field to get the field's content from the database.
>
> Ordering results: Say you define field name="bookName" type="text"
> indexed="true" stored="true" that is tokenized and used for searching. If
> you want to sort results based on book name, you could copy the field into a
> separate nonretrievable, nontokenized field that can be used just for
> sorting -
> field name="bookSort" type="string" indexed="true" stored="false"
> copyField source="bookName" dest="bookSort"
>
> Easier searching: If you define the field <field name="text" type="text"
> indexed="true" stored="false" multiValued="true"/> you can use it as a
> catch-all field that contains all of the other text fields. Since solr looks
> in a default field when given a text query without field names, you can
> support this type of general phrase query by making the catch-all the
> default field.
>
> indexed="false" stored="false"
> Use this when you want to ignore fields. For example, the following will
> ignore unknown fields that don't match a defined field rather than throwing
> an error by default.
> fieldtype name="ignored" stored="false" indexed="false"
> dynamicField name="*" type="ignored"
>
>
> Elizabeth Murnane
> emurnane@architexa.com
> Architexa Lead Developer - www.architexa.com
> Understand & Document Code In Seconds
>
>
> --- On Thu, 10/28/10, Savvas-Andreas Moysidis <
> savvas.andreas.moysidis@googlemail.com> wrote:
>
> From: Savvas-Andreas Moysidis <sa...@googlemail.com>
> Subject: Re: Stored or indexed?
> To: solr-user@lucene.apache.org
> Date: Thursday, October 28, 2010, 4:25 AM
>
> In our case, we just store a database id and do a secondary db query when
> displaying the results.
> This is handy and leads to a more centralised architecture when you need to
> display properties of a domain object which you don't index/search.
>
> On 28 October 2010 05:02, kenf_nc <ke...@realestate.com> wrote:
>
> >
> > Interesting wiki link, I hadn't seen that table before.
> >
> > And to answer your specific question about indexed=true, stored=false,
> this
> > is most often done when you are using analyzers/tokenizers on your field.
> > This field is for search only, you would never retrieve it's contents for
> > display. It may in fact be an amalgam of several fields into one
> 'content'
> > field. You have your display copy stored in another field marked
> > indexed=false, stored=true and optionally compressed. I also have simple
> > string fields set to lowercase so searching is case-insensitive, and have
> a
> > duplicate field where the string is normal case. the first one is
> > indexed/not stored, the second is stored/not indexed.
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/Stored-or-indexed-tp1782805p1784315.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>
> DISCLAIMER: This electronic message, including any attachments, files or
> documents, is intended only for the addressee and may contain CONFIDENTIAL,
> PROPRIETARY or LEGALLY PRIVILEGED information.  If you are not the intended
> recipient, you are hereby notified that any use, disclosure, copying or
> distribution of this message or any of the information included in or with
> it is  unauthorized and strictly prohibited.  If you have received this
> message in error, please notify the sender immediately by reply e-mail and
> permanently delete and destroy this message and its attachments, along with
> any copies thereof. This message does not create any contractual obligation
> on behalf of the sender or Law Bulletin Publishing Company.
> Thank you.
>