You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by kaveh minooie <ka...@plutoz.com> on 2012/01/27 22:37:21 UTC
problem with index-more and solr
I apologize if it is a stupid question but I have been googleing and I
haven't been able to find anything usefull. I am trying to use the
index-more plugin to get the contentlenght and date. I have put them in
nutch-site.xml and i can see that index-more gets loaded (in hadoop.log
file) but after indexing I can't see those fields in the result. both of
them are both in schema.xml file and solrindex-mapping.xml. I'll
appreciate any hint you guys can spare.
btw i am using nutch Revision: 1236417
thanks,
--
Kaveh Minooie
www.plutoz.com
Re: problem with index-more and solr
Posted by kaveh minooie <ka...@plutoz.com>.
sorry wrong solrindex-mapping in previous email. this is the correct one:
<mapping>
<!--
Simple mapping of fields created by Nutch IndexingFilters
to fields defined (and expected) in Solr schema.xml.
Any fields in NutchDocument that match a name defined
in field/@source will be renamed to the corresponding
field/@dest.
Additionally, if a field name (before mapping) matches
a copyField/@source then its values will be copied to
the corresponding copyField/@dest.
uniqueKey has the same meaning as in Solr schema.xml
and defaults to "id" if not defined.
-->
<fields>
<field dest="cache" source="cache"/>
<field dest="anchor" source="anchor"/>
<field dest="type" source="type"/>
<field dest="contentLength" source="contentLength"/>
<field dest="lastModified" source="lastModified"/>
<field dest="date" source="date"/>
<field dest="lang" source="lang"/>
<field dest="subcollection" source="subcollection"/>
<field dest="plutoz_ranking" source="plutoz_ranking"/>
<field dest="user_ranking" source="user_ranking"/>
<field dest="domain_ranking" source="domain_ranking"/>
<field dest="categories" source="categories"/>
<field dest="author" source="author"/>
<field dest="terms" source="terms"/>
<field dest="publishedDate" source="publishedDate"/>
<field dest="updatedDate" source="updatedDate"/>
<field dest="content" source="content"/>
<field dest="site" source="site"/>
<field dest="title" source="title"/>
<field dest="host" source="host"/>
<field dest="segment" source="segment"/>
<field dest="boost" source="boost"/>
<field dest="digest" source="digest"/>
<field dest="tstamp" source="tstamp"/>
<!--field dest="id" source="url"/-->
<!--copyField source="url" dest="url"/-->
</fields>
<uniqueKey>url</uniqueKey>
</mapping>
On 01/27/2012 02:08 PM, kaveh minooie wrote:
> thanks for that. so I did that and content is empty and contentlength is
> 0. I am not storing the content in the schema.xml file
>
> <field name="content" type="text" stored="false" indexed="true"
> termVectors="true"/>
> <field name="contentLength" type="long" stored="true" indexed="false"/>
>
> do I have to store content to be able to get the size?
>
> this is the result of the indexchecker btw:
>
> kaveh@index9:~/build/nutch/runtime/local$ bin/nutch indexchecker
> http://www.hafingtonpost.com
> fetching: http://www.hafingtonpost.com
> parsing: http://www.hafingtonpost.com
> contentType: text/html
> user_ranking : 25.0
> content :
> domain_ranking : 10.0
> host : www.hafingtonpost.com
> plutoz_ranking : 0.0
> terms :
> categories :
> tstamp : Fri Jan 27 13:56:39 PST 2012
> type : text/html
> type : text
> type : html
> date : Fri Jan 27 13:56:39 PST 2012
> contentLength : 0
> url : http://www.hafingtonpost.com
>
>
> also while we are at it some of the other fields here do not appear in
> the solr result as well such as user_ranking domain_ranking date
> contentlength and terms. content is not there either but no surprise
> there. I know solr do not include empty fields here for example terms,
> but I don't know why user_ranking and domain_ranking do not show up:
>
>
>
> here is my schema ans solrindex-mapping files:
>
> <schema name="nutch" version="1.1">
> <types>
> <fieldType name="string" class="solr.StrField" sortMissingLast="true"
> omitNorms="true"/>
> <fieldType name="long" class="solr.LongField" omitNorms="true"/>
> <fieldType name="float" class="solr.FloatField" omitNorms="true"/>
> <fieldType name="integer" class="solr.IntField" omitNorms="true"/>
> <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
> <analyzer type="index">
> <!--<charFilter class="solr.HTMLStripCharFilterFactory"/>-->
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords.txt"/>
> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1"
> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
> catenateAll="0" splitOnCaseChange="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
> <filter class="solr.SynonymFilterFactory" synonyms="ContentSynonyms.txt"
> ignoreCase="true" expand="true"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.WhitespaceTokenizerFactory"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords.txt"/>
> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1"
> generateNumberParts="1" catenateWords="1" catenateNumbers="1"
> catenateAll="0" splitOnCaseChange="1"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> </fieldType>
> <fieldType name="url" class="solr.StrField" positionIncrementGap="100">
> <!--<analyzer>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>-->
> </fieldType>
> <fieldType name="url2" class="solr.TextField" positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.LetterTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="true"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.LetterTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.EnglishPorterFilterFactory"/>
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="true"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> </fieldType>
> <fieldType name="url3" class="solr.TextField" positionIncrementGap="100">
> <analyzer type="index">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> </fieldType>
> <fieldType name="textSpell" class="solr.TextField"
> positionIncrementGap="100" stored="false" omitNorms="true">
> <analyzer type="index">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords.txt"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.StandardFilterFactory"/>
> </analyzer>
> <analyzer type="query">
> <tokenizer class="solr.StandardTokenizerFactory"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords.txt"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.StandardFilterFactory"/>
> </analyzer>
> </fieldType>
> </types>
> <fields>
> <field name="id" type="string" stored="true" indexed="true"/>
> <field name="segment" type="string" stored="true" indexed="true"/>
> <field name="digest" type="string" stored="true" indexed="false"/>
> <field name="boost" type="float" stored="true" indexed="false"/>
> <field name="dupid" type="string" stored="true" indexed="false"/>
> <field name="host" type="url" stored="true" indexed="true"/>
> <field name="host2" type="url2" stored="false" indexed="true"/>
> <field name="host3" type="url3" stored="false" indexed="true"/>
> <field name="site" type="string" stored="false" indexed="true"/>
> <field name="url" type="url" stored="true" indexed="true" required="true"/>
> <field name="url2" type="url3" stored="false" indexed="true"/>
> <field name="content" type="text" stored="false" indexed="true"
> termVectors="true"/>
> <field name="title" type="text" stored="false" indexed="true"
> termVectors="true"/>
> <field name="cache" type="string" stored="true" indexed="false"/>
> <field name="tstamp" type="long" stored="true" indexed="true"/>
> <field name="anchor" type="string" stored="false" indexed="true"
> multiValued="true"/>
> <field name="type" type="string" stored="true" indexed="true"
> multiValued="true"/>
> <field name="contentLength" type="long" stored="true" indexed="false"/>
> <field name="lastModified" type="long" stored="true" indexed="false"/>
> <field name="date" type="integer" stored="false" indexed="false"/>
> <field name="lang" type="string" stored="false" indexed="true"/>
> <field name="subcollection" type="string" stored="true" indexed="true"/>
> <field name="plutoz_ranking" type="float" stored="true" indexed="true"/>
> <field name="user_ranking" type="float" stored="false" indexed="true"/>
> <field name="domain_ranking" type="float" stored="false" indexed="true"/>
> <field name="categories" type="string" stored="true" indexed="true"
> multiValued="true"/>
> <field name="categories2" type="text" stored="false" indexed="true"
> multiValued="true"/>
> <field name="terms" type="text" stored="false" indexed="true"
> multiValued="true"/>
> <field name="author" type="string" stored="false" indexed="true"/>
> <field name="tag" type="string" stored="false" indexed="true"/>
> <field name="feed" type="string" stored="false" indexed="true"/>
> <field name="publishedDate" type="integer" stored="true" indexed="false"/>
> <field name="updatedDate" type="integer" stored="false" indexed="true"/>
> <field name="a_spell" type="textSpell" multiValued="true"/>
> </fields>
> <uniqueKey>url</uniqueKey>
> <defaultSearchField>title</defaultSearchField>
> <solrQueryParser defaultOperator="AND"/>
> <copyField source="url" dest="id"/>
> <copyField source="url" dest="url2"/>
> <copyField source="host" dest="host2"/>
> <copyField source="host" dest="host3"/>
> <copyField source="categories" dest="categories2"/>
> <copyField source="terms" dest="a_spell"/>
> <copyField source="title" dest="a_spell"/>
> <copyField source="content" dest="a_spell"/>
> </schema>
>
>
> solrindex-mapping:
>
>
>> <mapping>
>> <!-- Simple mapping of fields created by Nutch IndexingFilters
>> to fields defined (and expected) in Solr schema.xml.
>>
>> Any fields in NutchDocument that match a name defined
>> in field/@source will be renamed to the corresponding
>> field/@dest.
>> Additionally, if a field name (before mapping) matches
>> a copyField/@source then its values will be copied to
>> the corresponding copyField/@dest.
>>
>> uniqueKey has the same meaning as in Solr schema.xml
>> and defaults to "id" if not defined.
>> -->
>> <fields>
>> <!--field dest="cache" source="cache"/>
>> <field dest="anchor" source="anchor"/>
>> <field dest="type" source="type"/>
>> <field dest="contentLength" source="contentLength"/>
>> <field dest="lastModified" source="lastModified"/>
>> <field dest="date" source="date"/>
>> <field dest="lang" source="lang"/>
>> <field dest="subcollection" source="subcollection"/>
>> <field dest="plutoz_ranking" source="plutoz_ranking"/>
>> <field dest="user_ranking" source="user_ranking"/>
>> <field dest="domain_ranking" source="domain_ranking"/>
>> <field dest="categories" source="categories"/>
>> <field dest="author" source="author"/>
>> <field dest="terms" source="terms"/>
>> <field dest="publishedDate" source="publishedDate"/>
>> <field dest="updatedDate" source="updatedDate"/-->
>>
>> <field dest="content" source="content"/>
>> <field dest="site" source="site"/>
>> <field dest="title" source="title"/>
>> <field dest="host" source="host"/>
>> <field dest="segment" source="segment"/>
>> <field dest="boost" source="boost"/>
>> <field dest="digest" source="digest"/>
>> <field dest="tstamp" source="tstamp"/>
>> <field dest="id" source="url"/>
>> <copyField source="url" dest="url"/>
>> </fields>
>> <uniqueKey>url</uniqueKey>
>> </mapping>
>
>
>
>
>
>
> On 01/27/2012 01:41 PM, Markus Jelsma wrote:
>> Strange, the defaults work when the plugin is added to
>> plugin.includes. You
>> can use the indexchecker tool to make sure all is well on Nutch' side.
>>
>>> I apologize if it is a stupid question but I have been googleing and I
>>> haven't been able to find anything usefull. I am trying to use the
>>> index-more plugin to get the contentlenght and date. I have put them in
>>> nutch-site.xml and i can see that index-more gets loaded (in hadoop.log
>>> file) but after indexing I can't see those fields in the result. both of
>>> them are both in schema.xml file and solrindex-mapping.xml. I'll
>>> appreciate any hint you guys can spare.
>>>
>>> btw i am using nutch Revision: 1236417
>>>
>>> thanks,
>
--
Kaveh Minooie
www.plutoz.com
Re: problem with index-more and solr
Posted by kaveh minooie <ka...@plutoz.com>.
thanks for that. so I did that and content is empty and contentlength is
0. I am not storing the content in the schema.xml file
<field name="content" type="text" stored="false" indexed="true"
termVectors="true"/>
<field name="contentLength" type="long" stored="true" indexed="false"/>
do I have to store content to be able to get the size?
this is the result of the indexchecker btw:
kaveh@index9:~/build/nutch/runtime/local$ bin/nutch indexchecker
http://www.hafingtonpost.com
fetching: http://www.hafingtonpost.com
parsing: http://www.hafingtonpost.com
contentType: text/html
user_ranking : 25.0
content :
domain_ranking : 10.0
host : www.hafingtonpost.com
plutoz_ranking : 0.0
terms :
categories :
tstamp : Fri Jan 27 13:56:39 PST 2012
type : text/html
type : text
type : html
date : Fri Jan 27 13:56:39 PST 2012
contentLength : 0
url : http://www.hafingtonpost.com
also while we are at it some of the other fields here do not appear in
the solr result as well such as user_ranking domain_ranking date
contentlength and terms. content is not there either but no surprise
there. I know solr do not include empty fields here for example terms,
but I don't know why user_ranking and domain_ranking do not show up:
here is my schema ans solrindex-mapping files:
<schema name="nutch" version="1.1">
<types>
<fieldType name="string" class="solr.StrField"
sortMissingLast="true" omitNorms="true"/>
<fieldType name="long" class="solr.LongField" omitNorms="true"/>
<fieldType name="float" class="solr.FloatField" omitNorms="true"/>
<fieldType name="integer" class="solr.IntField" omitNorms="true"/>
<fieldType name="text" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<!--<charFilter class="solr.HTMLStripCharFilterFactory"/>-->
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/>
<filter class="solr.SynonymFilterFactory"
synonyms="ContentSynonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
<fieldType name="url" class="solr.StrField"
positionIncrementGap="100">
<!--<analyzer>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>-->
</fieldType>
<fieldType name="url2" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.LetterTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.SynonymFilterFactory"
synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.LetterTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"/>
<filter class="solr.SynonymFilterFactory"
synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
<fieldType name="url3" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
<fieldType name="textSpell" class="solr.TextField"
positionIncrementGap="100" stored="false" omitNorms="true">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StandardFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StandardFilterFactory"/>
</analyzer>
</fieldType>
</types>
<fields>
<field name="id" type="string" stored="true" indexed="true"/>
<field name="segment" type="string" stored="true" indexed="true"/>
<field name="digest" type="string" stored="true" indexed="false"/>
<field name="boost" type="float" stored="true" indexed="false"/>
<field name="dupid" type="string" stored="true" indexed="false"/>
<field name="host" type="url" stored="true" indexed="true"/>
<field name="host2" type="url2" stored="false" indexed="true"/>
<field name="host3" type="url3" stored="false" indexed="true"/>
<field name="site" type="string" stored="false" indexed="true"/>
<field name="url" type="url" stored="true" indexed="true"
required="true"/>
<field name="url2" type="url3" stored="false" indexed="true"/>
<field name="content" type="text" stored="false" indexed="true"
termVectors="true"/>
<field name="title" type="text" stored="false" indexed="true"
termVectors="true"/>
<field name="cache" type="string" stored="true" indexed="false"/>
<field name="tstamp" type="long" stored="true" indexed="true"/>
<field name="anchor" type="string" stored="false" indexed="true"
multiValued="true"/>
<field name="type" type="string" stored="true" indexed="true"
multiValued="true"/>
<field name="contentLength" type="long" stored="true" indexed="false"/>
<field name="lastModified" type="long" stored="true" indexed="false"/>
<field name="date" type="integer" stored="false" indexed="false"/>
<field name="lang" type="string" stored="false" indexed="true"/>
<field name="subcollection" type="string" stored="true" indexed="true"/>
<field name="plutoz_ranking" type="float" stored="true" indexed="true"/>
<field name="user_ranking" type="float" stored="false" indexed="true"/>
<field name="domain_ranking" type="float" stored="false" indexed="true"/>
<field name="categories" type="string" stored="true" indexed="true"
multiValued="true"/>
<field name="categories2" type="text" stored="false" indexed="true"
multiValued="true"/>
<field name="terms" type="text" stored="false" indexed="true"
multiValued="true"/>
<field name="author" type="string" stored="false" indexed="true"/>
<field name="tag" type="string" stored="false" indexed="true"/>
<field name="feed" type="string" stored="false" indexed="true"/>
<field name="publishedDate" type="integer" stored="true" indexed="false"/>
<field name="updatedDate" type="integer" stored="false" indexed="true"/>
<field name="a_spell" type="textSpell" multiValued="true"/>
</fields>
<uniqueKey>url</uniqueKey>
<defaultSearchField>title</defaultSearchField>
<solrQueryParser defaultOperator="AND"/>
<copyField source="url" dest="id"/>
<copyField source="url" dest="url2"/>
<copyField source="host" dest="host2"/>
<copyField source="host" dest="host3"/>
<copyField source="categories" dest="categories2"/>
<copyField source="terms" dest="a_spell"/>
<copyField source="title" dest="a_spell"/>
<copyField source="content" dest="a_spell"/>
</schema>
solrindex-mapping:
> <mapping>
> <!-- Simple mapping of fields created by Nutch IndexingFilters
> to fields defined (and expected) in Solr schema.xml.
>
> Any fields in NutchDocument that match a name defined
> in field/@source will be renamed to the corresponding
> field/@dest.
> Additionally, if a field name (before mapping) matches
> a copyField/@source then its values will be copied to
> the corresponding copyField/@dest.
>
> uniqueKey has the same meaning as in Solr schema.xml
> and defaults to "id" if not defined.
> -->
> <fields>
> <!--field dest="cache" source="cache"/>
> <field dest="anchor" source="anchor"/>
> <field dest="type" source="type"/>
> <field dest="contentLength" source="contentLength"/>
> <field dest="lastModified" source="lastModified"/>
> <field dest="date" source="date"/>
> <field dest="lang" source="lang"/>
> <field dest="subcollection" source="subcollection"/>
> <field dest="plutoz_ranking" source="plutoz_ranking"/>
> <field dest="user_ranking" source="user_ranking"/>
> <field dest="domain_ranking" source="domain_ranking"/>
> <field dest="categories" source="categories"/>
> <field dest="author" source="author"/>
> <field dest="terms" source="terms"/>
> <field dest="publishedDate" source="publishedDate"/>
> <field dest="updatedDate" source="updatedDate"/-->
>
> <field dest="content" source="content"/>
> <field dest="site" source="site"/>
> <field dest="title" source="title"/>
> <field dest="host" source="host"/>
> <field dest="segment" source="segment"/>
> <field dest="boost" source="boost"/>
> <field dest="digest" source="digest"/>
> <field dest="tstamp" source="tstamp"/>
> <field dest="id" source="url"/>
> <copyField source="url" dest="url"/>
> </fields>
> <uniqueKey>url</uniqueKey>
> </mapping>
On 01/27/2012 01:41 PM, Markus Jelsma wrote:
> Strange, the defaults work when the plugin is added to plugin.includes. You
> can use the indexchecker tool to make sure all is well on Nutch' side.
>
>> I apologize if it is a stupid question but I have been googleing and I
>> haven't been able to find anything usefull. I am trying to use the
>> index-more plugin to get the contentlenght and date. I have put them in
>> nutch-site.xml and i can see that index-more gets loaded (in hadoop.log
>> file) but after indexing I can't see those fields in the result. both of
>> them are both in schema.xml file and solrindex-mapping.xml. I'll
>> appreciate any hint you guys can spare.
>>
>> btw i am using nutch Revision: 1236417
>>
>> thanks,
--
Kaveh Minooie
www.plutoz.com
Re: problem with index-more and solr
Posted by Markus Jelsma <ma...@openindex.io>.
Strange, the defaults work when the plugin is added to plugin.includes. You
can use the indexchecker tool to make sure all is well on Nutch' side.
> I apologize if it is a stupid question but I have been googleing and I
> haven't been able to find anything usefull. I am trying to use the
> index-more plugin to get the contentlenght and date. I have put them in
> nutch-site.xml and i can see that index-more gets loaded (in hadoop.log
> file) but after indexing I can't see those fields in the result. both of
> them are both in schema.xml file and solrindex-mapping.xml. I'll
> appreciate any hint you guys can spare.
>
> btw i am using nutch Revision: 1236417
>
> thanks,