You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Zheng Lin Edwin Yeo <ed...@gmail.com> on 2015/03/03 07:00:13 UTC
Unable to show the indexed content in Solr 5.0
Hi,
The content field is unable to be shown during searching, even though the
following line has been added to the schema using curl from the resource
named in 'managedSchemaResourceName'.
<field name="content" stored="true" type="text_general" indexed="true"/>
I'm using the schema from ManagedIndexSchemaFactory.
As the ExtractRequestHandler has already been defined in solrconfig.xml by
default, and I'm using the ManagedIndexSchemaFactory. I have add the
content field line to allow the indexed content to be shown when user does
a query, as the default setting is not for the content to be shown. I added
in using curl as follows:
$ curl -X POST -H 'Content-type:application/json' --data-binary '{
"update-field" :
{ "name":"text", "type":"text_general", "stored":true, "indexed":true,
"storeOffsetsWithPositions":true}
}' http://localhost:8983/solr/collection1/schema
I have indexed the document using the following command:
java -Dc=collection1 -Dauto=true -jar example\exampledocs\post.jar
example\exampledcos\solr-word.pdf.
The document is successfully indexed, and when I does a search of any words
from the content, the search is able to return document ID and other
informations like subject, author, date, etc. However, the content of the
document is not shown.
This is what I got from the result.
If I didn't request the content field in the fl parameters, this is what I
got.
{
"responseHeader": {
"status": 0,
"QTime": 0,
"params": {
"indent": "true",
"q": "*:*",
"_": "1425362114731",
"wt": "json"
}
},
"response": {
"numFound": 2,
"start": 0,
"docs": [
{
"id": "C:\\Users\\GHI\\solr-5.0.0\\example\\exampledocs\\solr-word.pdf",
"meta_save_date": [
"2008-11-13T00:00:00Z"
],
"dc_subject": [
"solr, word, pdf"
],
"subject": [
"solr word"
],
"author": [
"Grant Ingersoll"
],
"dcterms_created": [
"2008-11-13T00:00:00Z"
],
"date": [
"2008-11-13T00:00:00Z"
],
"creator": [
"Grant Ingersoll"
],
"creation_date": [
"2008-11-13T00:00:00Z"
],
"title": [
"solr-word"
],
"meta_author": [
"Grant Ingersoll"
],
"stream_content_type": [
"application/pdf"
],
"created": [
"Thu Nov 13 13:35:51 UTC 2008"
],
"stream_size": [
21052
],
"meta_keyword": [
"solr, word, pdf"
],
"cp_subject": [
"solr word"
],
"dc_format": [
"application/pdf; version=1.3"
],
"xmp_creatortool": [
"Microsoft Word"
],
"resourcename": [
"C:\\Users\\GHI\\solr-5.0.0\\example\\exampledocs\\solr-word.pdf"
],
"keywords": [
"solr, word, pdf"
],
"last_save_date": [
"2008-11-13T00:00:00Z"
],
"dc_title": [
"solr-word"
],
"dcterms_modified": [
"2008-11-13T00:00:00Z"
],
"meta_creation_date": [
"2008-11-13T00:00:00Z"
],
"dc_creator": [
"Grant Ingersoll"
],
"pdf_pdfversion": [
1.3
],
"last_modified": [
"2008-11-13T00:00:00Z"
],
"aapl_keywords": [
"solr, word, pdf"
],
"x_parsed_by": [
"org.apache.tika.parser.DefaultParser",
"org.apache.tika.parser.pdf.PDFParser"
],
"modified": [
"2008-11-13T00:00:00Z"
],
"xmptpg_npages": [
1
],
"pdf_encrypted": [
false
],
"producer": [
"Mac OS X 10.5.5 Quartz PDFContext"
],
"content_type": [
"application/pdf"
],
"_version_": 1494155334466404300
},
{
"id": "C:\\Users\\GHI\\solr-5.0.0\\example\\exampledocs\\solr-word2.pdf",
"meta_save_date": [
"2015-02-25T00:00:00Z"
],
"author": [
"GHI"
],
"dcterms_created": [
"2015-02-25T00:00:00Z"
],
"date": [
"2015-02-25T00:00:00Z"
],
"creator": [
"GHI"
],
"creation_date": [
"2015-02-25T00:00:00Z"
],
"title": [
"This is another test of PDF extraction in Solr"
],
"meta_author": [
"GHI"
],
"stream_content_type": [
"application/pdf"
],
"created": [
"Wed Feb 25 08:32:19 UTC 2015"
],
"stream_size": [
10345
],
"dc_format": [
"application/pdf; version=1.4"
],
"xmp_creatortool": [
"PDFCreator Version 1.3.2"
],
"resourcename": [
"C:\\Users\\GHI\\solr-5.0.0\\example\\exampledocs\\solr-word2.pdf"
],
"last_save_date": [
"2015-02-25T00:00:00Z"
],
"dc_title": [
"This is another test of PDF extraction in Solr"
],
"dcterms_modified": [
"2015-02-25T00:00:00Z"
],
"meta_creation_date": [
"2015-02-25T00:00:00Z"
],
"dc_creator": [
"GHI"
],
"pdf_pdfversion": [
1.4
],
"last_modified": [
"2015-02-25T00:00:00Z"
],
"x_parsed_by": [
"org.apache.tika.parser.DefaultParser",
"org.apache.tika.parser.pdf.PDFParser"
],
"modified": [
"2015-02-25T00:00:00Z"
],
"xmptpg_npages": [
1
],
"pdf_encrypted": [
false
],
"producer": [
"GPL Ghostscript 9.05"
],
"content_type": [
"application/pdf"
],
"_version_": 1494155342991327200
}
]
}
}
If I request for the content field in the fl parameters, this is what I got.
{
"responseHeader": {
"status": 0,
"QTime": 1,
"params": {
"fl": "content",
"indent": "true",
"q": "*:*",
"_": "1425362147661",
"wt": "json"
}
},
"response": {
"numFound": 2,
"start": 0,
"docs": [
{},
{}
]
}
}
If I do a query like q=content:[* TO *]&fl=id,content
{
"responseHeader":{
"status":0,
"QTime":5,
"params":{
"fl":"id,content",
"q":"content:[* TO *]"}},
"response":{"numFound":0,"start":0,"docs":[]
}}
I'm able to get it to work in Solr 4.10.1, but it's not working in
Solr 5.0. Is there anything that I need to take note for Solr 5.0
which is different from the previous versions of Solr?
Regards,
Edwin
Re: Fwd: Unable to show the indexed content in Solr 5.0
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
Yes, I have reindex all my documents. In fact I have delete the entire data
directory to start fresh.
There's nothing interesting shown in the logs when I sent the doc to Solr,
it just says that the doc is successfully indexed as per normal.
Here's the schema which I'm using. The content field is already added and
indexed=true. Can help to see if anything is wrong?
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
<?xml version="1.0" encoding="UTF-8"?>
-<schema version="1.5" name="example-data-driven-schema">
<uniqueKey>id</uniqueKey>
-<fieldType name="ancestor_path" class="solr.TextField">
-<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory"/>
</analyzer>
-<analyzer type="query">
<tokenizer class="solr.PathHierarchyTokenizerFactory" delimiter="/"/>
</analyzer>
</fieldType>
<fieldType name="binary" class="solr.BinaryField"/>
<fieldType name="boolean" class="solr.BoolField" sortMissingLast="true"/>
<fieldType name="booleans" class="solr.BoolField" sortMissingLast="true"
multiValued="true"/>
<fieldType name="currency" class="solr.CurrencyField" defaultCurrency="USD"
currencyConfig="currency.xml" precisionStep="8"/>
<fieldType name="date" class="solr.TrieDateField" precisionStep="0"
positionIncrementGap="0"/>
<fieldType name="dates" class="solr.TrieDateField" multiValued="true"
precisionStep="0" positionIncrementGap="0"/>
-<fieldType name="descendent_path" class="solr.TextField">
-<analyzer type="index">
<tokenizer class="solr.PathHierarchyTokenizerFactory" delimiter="/"/>
</analyzer>
-<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory"/>
</analyzer>
</fieldType>
<fieldType name="double" class="solr.TrieDoubleField" precisionStep="0"
positionIncrementGap="0"/>
<fieldType name="doubles" class="solr.TrieDoubleField" multiValued="true"
precisionStep="0" positionIncrementGap="0"/>
<fieldType name="float" class="solr.TrieFloatField" precisionStep="0"
positionIncrementGap="0"/>
<fieldType name="floats" class="solr.TrieFloatField" multiValued="true"
precisionStep="0" positionIncrementGap="0"/>
<fieldType name="ignored" class="solr.StrField" multiValued="true"
stored="false" indexed="false"/>
<fieldType name="int" class="solr.TrieIntField" precisionStep="0"
positionIncrementGap="0"/>
<fieldType name="ints" class="solr.TrieIntField" multiValued="true"
precisionStep="0" positionIncrementGap="0"/>
<fieldType name="location" class="solr.LatLonType"
subFieldSuffix="_coordinate"/>
<fieldType name="location_rpt"
class="solr.SpatialRecursivePrefixTreeFieldType" maxDistErr="0.001"
geo="true" distErrPct="0.025" distanceUnits="kilometers"/>
<fieldType name="long" class="solr.TrieLongField" precisionStep="0"
positionIncrementGap="0"/>
<fieldType name="longs" class="solr.TrieLongField" multiValued="true"
precisionStep="0" positionIncrementGap="0"/>
-<fieldType name="lowercase" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="phonetic_en" class="solr.TextField" stored="false"
indexed="true">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.DoubleMetaphoneFilterFactory" inject="false"/>
</analyzer>
</fieldType>
<fieldType name="point" class="solr.PointType" subFieldSuffix="_d"
dimension="2"/>
<fieldType name="random" class="solr.RandomSortField" indexed="true"/>
<fieldType name="string" class="solr.StrField" sortMissingLast="true"/>
<fieldType name="strings" class="solr.StrField" sortMissingLast="true"
multiValued="true"/>
<fieldType name="tdate" class="solr.TrieDateField" precisionStep="6"
positionIncrementGap="0"/>
<fieldType name="tdates" class="solr.TrieDateField" multiValued="true"
precisionStep="6" positionIncrementGap="0"/>
<fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8"
positionIncrementGap="0"/>
<fieldType name="tdoubles" class="solr.TrieDoubleField" multiValued="true"
precisionStep="8" positionIncrementGap="0"/>
-<fieldType name="text_ar" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_ar.txt"/>
<filter class="solr.ArabicNormalizationFilterFactory"/>
<filter class="solr.ArabicStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_bg" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_bg.txt"/>
<filter class="solr.BulgarianStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_ca" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.ElisionFilterFactory" ignoreCase="true"
articles="lang/contractions_ca.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_ca.txt"/>
<filter language="Catalan" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_cjk" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.CJKWidthFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.CJKBigramFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_cz" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_cz.txt"/>
<filter class="solr.CzechStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_da" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_da.txt" format="snowball"/>
<filter language="Danish" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_de" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_de.txt" format="snowball"/>
<filter class="solr.GermanNormalizationFilterFactory"/>
<filter class="solr.GermanLightStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_el" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.GreekLowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="false"
words="lang/stopwords_el.txt"/>
<filter class="solr.GreekStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_en" class="solr.TextField"
positionIncrementGap="100">
-<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_en.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPossessiveFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
-<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" ignoreCase="true"
synonyms="synonyms.txt" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_en.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPossessiveFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_en_splitting" class="solr.TextField"
positionIncrementGap="100" autoGeneratePhraseQueries="true">
-<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_en.txt"/>
<filter class="solr.WordDelimiterFilterFactory" catenateNumbers="1"
catenateAll="0" generateWordParts="1" catenateWords="1"
splitOnCaseChange="1" generateNumberParts="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
-<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" ignoreCase="true"
synonyms="synonyms.txt" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_en.txt"/>
<filter class="solr.WordDelimiterFilterFactory" catenateNumbers="0"
catenateAll="0" generateWordParts="1" catenateWords="0"
splitOnCaseChange="1" generateNumberParts="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_en_splitting_tight" class="solr.TextField"
positionIncrementGap="100" autoGeneratePhraseQueries="true">
-<analyzer>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" ignoreCase="true"
synonyms="synonyms.txt" expand="false"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_en.txt"/>
<filter class="solr.WordDelimiterFilterFactory" catenateNumbers="1"
catenateAll="0" generateWordParts="0" catenateWords="1"
generateNumberParts="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<filter class="solr.EnglishMinimalStemFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_es" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_es.txt" format="snowball"/>
<filter class="solr.SpanishLightStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_eu" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_eu.txt"/>
<filter language="Basque" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_fa" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<charFilter class="solr.PersianCharFilterFactory"/>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ArabicNormalizationFilterFactory"/>
<filter class="solr.PersianNormalizationFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_fa.txt"/>
</analyzer>
</fieldType>
-<fieldType name="text_fi" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_fi.txt" format="snowball"/>
<filter language="Finnish" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_fr" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.ElisionFilterFactory" ignoreCase="true"
articles="lang/contractions_fr.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_fr.txt" format="snowball"/>
<filter class="solr.FrenchLightStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_ga" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.ElisionFilterFactory" ignoreCase="true"
articles="lang/contractions_ga.txt"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/hyphenations_ga.txt"/>
<filter class="solr.IrishLowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_ga.txt"/>
<filter language="Irish" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_general" class="solr.TextField" multiValued="true"
positionIncrementGap="100">
-<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
-<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.SynonymFilterFactory" ignoreCase="true"
synonyms="synonyms.txt" expand="true"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_general_rev" class="solr.TextField"
positionIncrementGap="100">
-<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ReversedWildcardFilterFactory" maxPosAsterisk="3"
maxPosQuestion="2" withOriginal="true" maxFractionAsterisk="0.33"/>
</analyzer>
-<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" ignoreCase="true"
synonyms="synonyms.txt" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_gl" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_gl.txt"/>
<filter class="solr.GalicianStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_hi" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.IndicNormalizationFilterFactory"/>
<filter class="solr.HindiNormalizationFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_hi.txt"/>
<filter class="solr.HindiStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_hu" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_hu.txt" format="snowball"/>
<filter language="Hungarian" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_hy" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_hy.txt"/>
<filter language="Armenian" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_id" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_id.txt"/>
<filter class="solr.IndonesianStemFilterFactory" stemDerivational="true"/>
</analyzer>
</fieldType>
-<fieldType name="text_it" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.ElisionFilterFactory" ignoreCase="true"
articles="lang/contractions_it.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_it.txt" format="snowball"/>
<filter class="solr.ItalianLightStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_ja" class="solr.TextField"
positionIncrementGap="100" autoGeneratePhraseQueries="false">
-<analyzer>
<tokenizer class="solr.JapaneseTokenizerFactory" mode="search"/>
<filter class="solr.JapaneseBaseFormFilterFactory"/>
<filter class="solr.JapanesePartOfSpeechStopFilterFactory"
tags="lang/stoptags_ja.txt"/>
<filter class="solr.CJKWidthFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_ja.txt"/>
<filter class="solr.JapaneseKatakanaStemFilterFactory" minimumLength="4"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_lv" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_lv.txt"/>
<filter class="solr.LatvianStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_nl" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_nl.txt" format="snowball"/>
<filter class="solr.StemmerOverrideFilterFactory" ignoreCase="false"
dictionary="lang/stemdict_nl.txt"/>
<filter language="Dutch" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_no" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_no.txt" format="snowball"/>
<filter language="Norwegian" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_pt" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_pt.txt" format="snowball"/>
<filter class="solr.PortugueseLightStemFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_ro" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_ro.txt"/>
<filter language="Romanian" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_ru" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_ru.txt" format="snowball"/>
<filter language="Russian" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_sv" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_sv.txt" format="snowball"/>
<filter language="Swedish" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_th" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.ThaiTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="lang/stopwords_th.txt"/>
</analyzer>
</fieldType>
-<fieldType name="text_tr" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.TurkishLowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="false"
words="lang/stopwords_tr.txt"/>
<filter language="Turkish" class="solr.SnowballPorterFilterFactory"/>
</analyzer>
</fieldType>
-<fieldType name="text_ws" class="solr.TextField"
positionIncrementGap="100">
-<analyzer>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
</analyzer>
</fieldType>
<fieldType name="tfloat" class="solr.TrieFloatField" precisionStep="8"
positionIncrementGap="0"/>
<fieldType name="tfloats" class="solr.TrieFloatField" multiValued="true"
precisionStep="8" positionIncrementGap="0"/>
<fieldType name="tint" class="solr.TrieIntField" precisionStep="8"
positionIncrementGap="0"/>
<fieldType name="tints" class="solr.TrieIntField" multiValued="true"
precisionStep="8" positionIncrementGap="0"/>
<fieldType name="tlong" class="solr.TrieLongField" precisionStep="8"
positionIncrementGap="0"/>
<fieldType name="tlongs" class="solr.TrieLongField" multiValued="true"
precisionStep="8" positionIncrementGap="0"/>
<field name="_text" type="text_general" multiValued="true" stored="false"
indexed="true"/>
<field name="_version_" type="long" stored="true" indexed="true"/>
<field name="aapl_keywords" type="strings"/>
<field name="author" type="strings"/>
<field name="content" type="text_general" stored="true" indexed="true"/>
<field name="content_type" type="strings"/>
<field name="cp_subject" type="strings"/>
<field name="created" type="strings"/>
<field name="creation_date" type="tdates"/>
<field name="creator" type="strings"/>
<field name="date" type="tdates"/>
<field name="dc_creator" type="strings"/>
<field name="dc_format" type="strings"/>
<field name="dc_subject" type="strings"/>
<field name="dc_title" type="strings"/>
<field name="dcterms_created" type="tdates"/>
<field name="dcterms_modified" type="tdates"/>
<field name="id" type="string" multiValued="false" stored="true"
indexed="true" required="true"/>
<field name="keywords" type="strings"/>
<field name="last_modified" type="tdates"/>
<field name="last_save_date" type="tdates"/>
<field name="meta_author" type="strings"/>
<field name="meta_creation_date" type="tdates"/>
<field name="meta_keyword" type="strings"/>
<field name="meta_save_date" type="tdates"/>
<field name="modified" type="tdates"/>
<field name="pdf_encrypted" type="booleans"/>
<field name="pdf_pdfversion" type="tdoubles"/>
<field name="producer" type="strings"/>
<field name="resourcename" type="strings"/>
<field name="stream_content_type" type="strings"/>
<field name="stream_size" type="tlongs"/>
<field name="subject" type="strings"/>
<field name="text" type="text_general" multiValued="true" stored="true"
indexed="true"/>
<field name="title" type="strings"/>
<field name="x_parsed_by" type="strings"/>
<field name="xmp_creatortool" type="strings"/>
<field name="xmptpg_npages" type="tlongs"/>
<dynamicField name="*_txt_en_split_tight" type="text_en_splitting_tight"
stored="true" indexed="true"/>
<dynamicField name="*_descendent_path" type="descendent_path" stored="true"
indexed="true"/>
<dynamicField name="*_ancestor_path" type="ancestor_path" stored="true"
indexed="true"/>
<dynamicField name="*_txt_en_split" type="text_en_splitting" stored="true"
indexed="true"/>
<dynamicField name="*_coordinate" type="tdouble" stored="false"
indexed="true"/>
<dynamicField name="ignored_*" type="ignored" multiValued="true"/>
<dynamicField name="*_txt_rev" type="text_general_rev" stored="true"
indexed="true"/>
<dynamicField name="*_phon_en" type="phonetic_en" stored="true"
indexed="true"/>
<dynamicField name="*_s_lower" type="lowercase" stored="true"
indexed="true"/>
<dynamicField name="*_txt_cjk" type="text_cjk" stored="true"
indexed="true"/>
<dynamicField name="random_*" type="random"/>
<dynamicField name="*_txt_en" type="text_en" stored="true" indexed="true"/>
<dynamicField name="*_txt_ar" type="text_ar" stored="true" indexed="true"/>
<dynamicField name="*_txt_bg" type="text_bg" stored="true" indexed="true"/>
<dynamicField name="*_txt_ca" type="text_ca" stored="true" indexed="true"/>
<dynamicField name="*_txt_cz" type="text_cz" stored="true" indexed="true"/>
<dynamicField name="*_txt_da" type="text_da" stored="true" indexed="true"/>
<dynamicField name="*_txt_de" type="text_de" stored="true" indexed="true"/>
<dynamicField name="*_txt_el" type="text_el" stored="true" indexed="true"/>
<dynamicField name="*_txt_es" type="text_es" stored="true" indexed="true"/>
<dynamicField name="*_txt_eu" type="text_eu" stored="true" indexed="true"/>
<dynamicField name="*_txt_fa" type="text_fa" stored="true" indexed="true"/>
<dynamicField name="*_txt_fi" type="text_fi" stored="true" indexed="true"/>
<dynamicField name="*_txt_fr" type="text_fr" stored="true" indexed="true"/>
<dynamicField name="*_txt_ga" type="text_ga" stored="true" indexed="true"/>
<dynamicField name="*_txt_gl" type="text_gl" stored="true" indexed="true"/>
<dynamicField name="*_txt_hi" type="text_hi" stored="true" indexed="true"/>
<dynamicField name="*_txt_hu" type="text_hu" stored="true" indexed="true"/>
<dynamicField name="*_txt_hy" type="text_hy" stored="true" indexed="true"/>
<dynamicField name="*_txt_id" type="text_id" stored="true" indexed="true"/>
<dynamicField name="*_txt_it" type="text_it" stored="true" indexed="true"/>
<dynamicField name="*_txt_ja" type="text_ja" stored="true" indexed="true"/>
<dynamicField name="*_txt_lv" type="text_lv" stored="true" indexed="true"/>
<dynamicField name="*_txt_nl" type="text_nl" stored="true" indexed="true"/>
<dynamicField name="*_txt_no" type="text_no" stored="true" indexed="true"/>
<dynamicField name="*_txt_pt" type="text_pt" stored="true" indexed="true"/>
<dynamicField name="*_txt_ro" type="text_ro" stored="true" indexed="true"/>
<dynamicField name="*_txt_ru" type="text_ru" stored="true" indexed="true"/>
<dynamicField name="*_txt_sv" type="text_sv" stored="true" indexed="true"/>
<dynamicField name="*_txt_th" type="text_th" stored="true" indexed="true"/>
<dynamicField name="*_txt_tr" type="text_tr" stored="true" indexed="true"/>
<dynamicField name="*_point" type="point" stored="true" indexed="true"/>
<dynamicField name="*_srpt" type="location_rpt" stored="true"
indexed="true"/>
<dynamicField name="*_tdts" type="tdates" stored="true" indexed="true"/>
<dynamicField name="attr_*" type="text_general" multiValued="true"
stored="true" indexed="true"/>
<dynamicField name="*_txt" type="text_general" stored="true"
indexed="true"/>
<dynamicField name="*_dts" type="date" multiValued="true" stored="true"
indexed="true"/>
<dynamicField name="*_tis" type="tints" stored="true" indexed="true"/>
<dynamicField name="*_tls" type="tlongs" stored="true" indexed="true"/>
<dynamicField name="*_tfs" type="tfloats" stored="true" indexed="true"/>
<dynamicField name="*_tds" type="tdoubles" stored="true" indexed="true"/>
<dynamicField name="*_tdt" type="tdate" stored="true" indexed="true"/>
<dynamicField name="*_is" type="ints" stored="true" indexed="true"/>
<dynamicField name="*_ss" type="strings" stored="true" indexed="true"/>
<dynamicField name="*_ls" type="longs" stored="true" indexed="true"/>
<dynamicField name="*_bs" type="booleans" stored="true" indexed="true"/>
<dynamicField name="*_fs" type="floats" stored="true" indexed="true"/>
<dynamicField name="*_ds" type="doubles" stored="true" indexed="true"/>
<dynamicField name="*_dt" type="date" stored="true" indexed="true"/>
<dynamicField name="*_ti" type="tint" stored="true" indexed="true"/>
<dynamicField name="*_tl" type="tlong" stored="true" indexed="true"/>
<dynamicField name="*_tf" type="tfloat" stored="true" indexed="true"/>
<dynamicField name="*_td" type="tdouble" stored="true" indexed="true"/>
<dynamicField name="*_ws" type="text_ws" stored="true" indexed="true"/>
<dynamicField name="*_i" type="int" stored="true" indexed="true"/>
<dynamicField name="*_s" type="string" stored="true" indexed="true"/>
<dynamicField name="*_l" type="long" stored="true" indexed="true"/>
<dynamicField name="*_t" type="text_general" stored="true" indexed="true"/>
<dynamicField name="*_b" type="boolean" stored="true" indexed="true"/>
<dynamicField name="*_f" type="float" stored="true" indexed="true"/>
<dynamicField name="*_d" type="double" stored="true" indexed="true"/>
<dynamicField name="*_p" type="location" stored="true" indexed="true"/>
<dynamicField name="*_c" type="currency" stored="true" indexed="true"/>
<copyField dest="_text" source="*"/>
</schema>
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Thanks and Regards,
Edwin
On 4 March 2015 at 07:38, Erick Erickson <er...@gmail.com> wrote:
> OK, this last query is telling:
>
> {
> "responseHeader":{
> "status":0,
> "QTime":5,
> "params":{
> "fl":"id,content",
> "q":"content:[* TO *]"}},
> "response":{"numFound":0,"start":0,"docs":[]
> }}
>
> Despite what you think, you are _not_ actually indexing anything to
> the "content" field in Solr
>
> I'm not quite sure how, doe the Solr logs show anything interesting
> when you send the doc to Solr?
>
> Best,
> Erick
>
> On Tue, Mar 3, 2015 at 6:25 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> > On 3/3/2015 12:23 AM, Zheng Lin Edwin Yeo wrote:
> >> The content field is unable to be shown during searching, even though
> the
> >> following line has been added to the schema using curl from the resource
> >> named in 'managedSchemaResourceName'.
> >>
> >> <field name="content" stored="true" type="text_general" indexed="true"/>
> >
> > Did you restart Solr (or reload the core/collection) and reindex all
> > your documents after you made the change? The contents of the existing
> > index do not change when you change your schema.
> >
> > http://wiki.apache.org/solr/HowToReindex
> >
> > Thanks,
> > Shawn
> >
>
Re: Fwd: Unable to show the indexed content in Solr 5.0
Posted by Erick Erickson <er...@gmail.com>.
OK, this last query is telling:
{
"responseHeader":{
"status":0,
"QTime":5,
"params":{
"fl":"id,content",
"q":"content:[* TO *]"}},
"response":{"numFound":0,"start":0,"docs":[]
}}
Despite what you think, you are _not_ actually indexing anything to
the "content" field in Solr
I'm not quite sure how, doe the Solr logs show anything interesting
when you send the doc to Solr?
Best,
Erick
On Tue, Mar 3, 2015 at 6:25 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 3/3/2015 12:23 AM, Zheng Lin Edwin Yeo wrote:
>> The content field is unable to be shown during searching, even though the
>> following line has been added to the schema using curl from the resource
>> named in 'managedSchemaResourceName'.
>>
>> <field name="content" stored="true" type="text_general" indexed="true"/>
>
> Did you restart Solr (or reload the core/collection) and reindex all
> your documents after you made the change? The contents of the existing
> index do not change when you change your schema.
>
> http://wiki.apache.org/solr/HowToReindex
>
> Thanks,
> Shawn
>
Re: Fwd: Unable to show the indexed content in Solr 5.0
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/3/2015 12:23 AM, Zheng Lin Edwin Yeo wrote:
> The content field is unable to be shown during searching, even though the
> following line has been added to the schema using curl from the resource
> named in 'managedSchemaResourceName'.
>
> <field name="content" stored="true" type="text_general" indexed="true"/>
Did you restart Solr (or reload the core/collection) and reindex all
your documents after you made the change? The contents of the existing
index do not change when you change your schema.
http://wiki.apache.org/solr/HowToReindex
Thanks,
Shawn
Fwd: Unable to show the indexed content in Solr 5.0
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
Hi,
The content field is unable to be shown during searching, even though the
following line has been added to the schema using curl from the resource
named in 'managedSchemaResourceName'.
<field name="content" stored="true" type="text_general" indexed="true"/>
I'm using the schema from ManagedIndexSchemaFactory.
As the ExtractRequestHandler has already been defined in solrconfig.xml by
default, and I'm using the ManagedIndexSchemaFactory. I have add the
content field line to allow the indexed content to be shown when user does
a query, as the default setting is not for the content to be shown. I added
in using curl as follows:
$ curl -X POST -H 'Content-type:application/json' --data-binary '{
"update-field" :
{ "name":"text", "type":"text_general", "stored":true, "indexed":true,
"storeOffsetsWithPositions":true}
}' http://localhost:8983/solr/collection1/schema
I have indexed the document using the following command:
java -Dc=collection1 -Dauto=true -jar example\exampledocs\post.jar
example\exampledcos\solr-word.pdf.
The document is successfully indexed, and when I does a search of any words
from the content, the search is able to return document ID and other
informations like subject, author, date, etc. However, the content of the
document is not shown.
This is what I got from the result.
If I didn't request the content field in the fl parameters, this is what I
got.
{
"responseHeader": {
"status": 0,
"QTime": 0,
"params": {
"indent": "true",
"q": "*:*",
"_": "1425362114731",
"wt": "json"
}
},
"response": {
"numFound": 2,
"start": 0,
"docs": [
{
"id": "C:\\Users\\GHI\\solr-5.0.0\\example\\exampledocs\\solr-word.pdf",
"meta_save_date": [
"2008-11-13T00:00:00Z"
],
"dc_subject": [
"solr, word, pdf"
],
"subject": [
"solr word"
],
"author": [
"Grant Ingersoll"
],
"dcterms_created": [
"2008-11-13T00:00:00Z"
],
"date": [
"2008-11-13T00:00:00Z"
],
"creator": [
"Grant Ingersoll"
],
"creation_date": [
"2008-11-13T00:00:00Z"
],
"title": [
"solr-word"
],
"meta_author": [
"Grant Ingersoll"
],
"stream_content_type": [
"application/pdf"
],
"created": [
"Thu Nov 13 13:35:51 UTC 2008"
],
"stream_size": [
21052
],
"meta_keyword": [
"solr, word, pdf"
],
"cp_subject": [
"solr word"
],
"dc_format": [
"application/pdf; version=1.3"
],
"xmp_creatortool": [
"Microsoft Word"
],
"resourcename": [
"C:\\Users\\GHI\\solr-5.0.0\\example\\exampledocs\\solr-word.pdf"
],
"keywords": [
"solr, word, pdf"
],
"last_save_date": [
"2008-11-13T00:00:00Z"
],
"dc_title": [
"solr-word"
],
"dcterms_modified": [
"2008-11-13T00:00:00Z"
],
"meta_creation_date": [
"2008-11-13T00:00:00Z"
],
"dc_creator": [
"Grant Ingersoll"
],
"pdf_pdfversion": [
1.3
],
"last_modified": [
"2008-11-13T00:00:00Z"
],
"aapl_keywords": [
"solr, word, pdf"
],
"x_parsed_by": [
"org.apache.tika.parser.DefaultParser",
"org.apache.tika.parser.pdf.PDFParser"
],
"modified": [
"2008-11-13T00:00:00Z"
],
"xmptpg_npages": [
1
],
"pdf_encrypted": [
false
],
"producer": [
"Mac OS X 10.5.5 Quartz PDFContext"
],
"content_type": [
"application/pdf"
],
"_version_": 1494155334466404300
},
{
"id": "C:\\Users\\GHI\\solr-5.0.0\\example\\exampledocs\\solr-word2.pdf",
"meta_save_date": [
"2015-02-25T00:00:00Z"
],
"author": [
"GHI"
],
"dcterms_created": [
"2015-02-25T00:00:00Z"
],
"date": [
"2015-02-25T00:00:00Z"
],
"creator": [
"GHI"
],
"creation_date": [
"2015-02-25T00:00:00Z"
],
"title": [
"This is another test of PDF extraction in Solr"
],
"meta_author": [
"GHI"
],
"stream_content_type": [
"application/pdf"
],
"created": [
"Wed Feb 25 08:32:19 UTC 2015"
],
"stream_size": [
10345
],
"dc_format": [
"application/pdf; version=1.4"
],
"xmp_creatortool": [
"PDFCreator Version 1.3.2"
],
"resourcename": [
"C:\\Users\\GHI\\solr-5.0.0\\example\\exampledocs\\solr-word2.pdf"
],
"last_save_date": [
"2015-02-25T00:00:00Z"
],
"dc_title": [
"This is another test of PDF extraction in Solr"
],
"dcterms_modified": [
"2015-02-25T00:00:00Z"
],
"meta_creation_date": [
"2015-02-25T00:00:00Z"
],
"dc_creator": [
"GHI"
],
"pdf_pdfversion": [
1.4
],
"last_modified": [
"2015-02-25T00:00:00Z"
],
"x_parsed_by": [
"org.apache.tika.parser.DefaultParser",
"org.apache.tika.parser.pdf.PDFParser"
],
"modified": [
"2015-02-25T00:00:00Z"
],
"xmptpg_npages": [
1
],
"pdf_encrypted": [
false
],
"producer": [
"GPL Ghostscript 9.05"
],
"content_type": [
"application/pdf"
],
"_version_": 1494155342991327200
}
]
}
}
If I request for the content field in the fl parameters, this is what I got.
{
"responseHeader": {
"status": 0,
"QTime": 1,
"params": {
"fl": "content",
"indent": "true",
"q": "*:*",
"_": "1425362147661",
"wt": "json"
}
},
"response": {
"numFound": 2,
"start": 0,
"docs": [
{},
{}
]
}
}
If I do a query like q=content:[* TO *]&fl=id,content
{
"responseHeader":{
"status":0,
"QTime":5,
"params":{
"fl":"id,content",
"q":"content:[* TO *]"}},
"response":{"numFound":0,"start":0,"docs":[]
}}
I'm able to get it to work in Solr 4.10.1, but it's not working in
Solr 5.0. Is there anything that I need to take note for Solr 5.0
which is different from the previous versions of Solr?
Regards,
Edwin