You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Charles Wardell <ch...@bcsolution.com> on 2011/03/29 03:32:31 UTC
Fields not being indexed?
Can someone take a look at this and let me know what I am doing wrong. According to luke, only guid, tags, and aquiDate are available.
Schema is below as well.
<add>
<doc>
<field name="guid">http://twitter.com/AshleyxArsenic/statuses/52164920388763648</field>
<![CDATA[<field name="title">@Richard_Colo I realy don't think its for awhile. Ill study when it comes up</field>]]>
<![CDATA[<field name="authorName">AshleyxArsenic (Ashley Hoffman)</field>]]>
<![CDATA[<field name="authorEmail"></field>]]>
<![CDATA[<field name="authorLinkMimeType"></field>]]>
<![CDATA[<field name="authorLink">http://twitter.com/AshleyxArsenic</field>]]>
<![CDATA[<field name="lang">en</field>]]>
<![CDATA[<field name="publishDate">2011-03-28T00:27:40Z</field>]]>
<![CDATA[<field name="aquiDate">2011-03-28T00:27:52Z</field>]]>
<![CDATA[<field name="source"></field>]]>
<![CDATA[<field name="feedURL">http://twitter.com/AshleyxArsenic/statuses/52164920388763648</field>]]>
<![CDATA[<field name="feedContentMimeType">text/html</field>]]>
<![CDATA[<field name="feedContentEncoding"></field>]]>
<![CDATA[<field name="feedContent">null</field>]]>
<![CDATA[<field name="inboundLinks">0</field>]]>
<![CDATA[<field name="publisherType">MICROBLOG</field>]]>
<![CDATA[<field name="postTitle">@Richard_Colo I realy don't think its for awhile. Ill study when it comes up</field>]]>
<![CDATA[<field name="postBodyMimeType">text/html</field>]]>
<![CDATA[<field name="postBodyEncoding">zlib</field>]]>
<![CDATA[<field name="postBody">mime_type: "text/html"
data: ""
</field>]]>
<field name="tags">[]</field>
</doc>
</add>
<schema name="myschema" version="1.1">
<types>
<fieldtype name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
<fieldtype name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
<fieldtype name="integer" class="solr.IntField" omitNorms="true"/>
<fieldtype name="long" class="solr.LongField" omitNorms="true"/>
<fieldtype name="float" class="solr.FloatField" omitNorms="true"/>
<fieldtype name="double" class="solr.DoubleField" omitNorms="true"/>
<fieldtype name="sint" class="solr.SortableIntField" sortMissingLast="true" omitNorms="true"/>
<fieldtype name="slong" class="solr.SortableLongField" sortMissingLast="true" omitNorms="true"/>
<fieldtype name="sfloat" class="solr.SortableFloatField" sortMissingLast="true" omitNorms="true"/>
<fieldtype name="sdouble" class="solr.SortableDoubleField" sortMissingLast="true" omitNorms="true"/>
<fieldtype name="date" class="solr.DateField" sortMissingLast="true" omitNorms="true"/>
<fieldtype name="text_ws" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
</analyzer>
</fieldtype>
<fieldtype name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<charfilter class="solr.HTMLStripCharFilterFactory"/>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldtype>
</types>
<!-- Valid attributes for fields:
name: mandatory - the name for the field
type: mandatory - the name of a previously defined type from the <types>
section indexed: true if this field should be indexed (searchable or sortable)
stored: true if this field should be retrievable
compressed: [false] if this field should be stored using gzip compression (this will only apply if the
field type is compressable; among the standard field types, only TextField
and StrField are)
multiValued: true if this field may contain multiple values
per document
omitNorms: (expert) set to true to omit the norms associated
with this field (this disables length normalization and index-time boosting
for the field, and saves some memory). Only full-text fields or fields that
need an index-time boost need norms. -->
<fields>
<field name="guid" type="string" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="title" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="authorName" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="authorEmail" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="authorLinkMimeType" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="authorLink" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="lang" type="text" indexed="true" stored="true" compressed="false" multiValued="false" />
<field name="publishDate" type="date" indexed="true" stored="true" compressed="false" multiValued="false" />
<field name="aquiDate" type="date" indexed="true" stored="true" compressed="false" multiValued="false" default="NOW" />
<field name="source" type="string" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="feedURL" type="string" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="feedContentMimeType" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="feedContentEncoding" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="feedContent" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="inboundLinks" type="integer" indexed="true" stored="true" compressed="false" multiValued="false" />
<field name="publisherType" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="postTitle" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="postBodyMimeType" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="postBodyEncoding" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="postBody" type="text" indexed="true" stored="true" compressed="true" multiValued="false" />
<field name="tags" type="text_ws" indexed="true" stored="true" compressed="true" multiValued="true" omitNorms="true"/>
</fields>
<uniqueKey>guid</uniqueKey>
<solrQueryParser defaultOperator="OR"/>
</schema>
Re: Fields not being indexed?
Posted by Stefan Matheis <ma...@googlemail.com>.
Charles,
On Tue, Mar 29, 2011 at 3:32 AM, Charles Wardell
<ch...@bcsolution.com> wrote:
> <add>
> <doc>
> <field name="guid">http://twitter.com/AshleyxArsenic/statuses/52164920388763648</field>
> <![CDATA[<field name="title">@Richard_Colo I realy don't think its for awhile. Ill study when it comes up</field>]]>
> <![CDATA[<field name="authorName">AshleyxArsenic (Ashley Hoffman)</field>]]>
did you see the difference between the first and the following two
definitions? actually it's really well-formed (they are treated as
textnode, as a colleague told me actually) - but they are no used from
the solr-perspective.
do it like this, should work:
<field name="title"><![CDATA[@Richard_Colo I realy don't think its for
awhile. Ill study when it comes up]]></field>
and have a look on http://en.wikipedia.org/wiki/CDATA
Regards
Stefan
Re: Fields not being indexed?
Posted by Chris Hostetter <ho...@fucit.org>.
: Subject: Fields not being indexed?
: In-Reply-To: <AA...@mail.gmail.com>
: References: <AA...@mail.gmail.com>
: <al...@bester>
: <AA...@mail.gmail.com>
http://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which thread
you replied to and your question is "hidden" in that thread and gets less
attention. It makes following discussions in the mailing list archives
particularly difficult.
-Hoss