[jira] [Comment Edited] (STANBOL-858) Provide SolrCore configuration for the Entityhub SolrYard with smartcn fieldType configuration for Chinese (zh)

["Rupert Westenthaler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STANBOL-858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538877#comment-13538877 ] 

Rupert Westenthaler edited comment on STANBOL-858 at 12/22/12 6:46 PM:
-----------------------------------------------------------------------

When the bundle

    <bundle>
      <groupId>org.apache.stanbol</groupId>
      <artifactId>org.apache.stanbol.commons.solr.extras.smartcn</artifactId>
      <version>0.10.0-SNAPSHOT</version>
    </bundle>

is present the following fieldType can be used to index Chinese text

<fieldType name="text_zh" class="solr.TextField" positionIncrementGap="100">
      <analyzer type="index">
        <tokenizer class="solr.SmartChineseSentenceTokenizerFactory"/>
        <filter class="solr.SmartChineseWordTokenFilterFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.SmartChineseSentenceTokenizerFactory"/>
        <filter class="solr.SmartChineseWordTokenFilterFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
        <filter class="solr.PositionFilterFactory" />
      </analyzer>
    </fieldType>

To apply this fieldType to Chinese labels one need also to add

    <!--
     Dynamic field for Chinese languages.
     -->
    <dynamicField name="@zh*"  type="text_zh" indexed="true" stored="true" multiValued="true" omitNorms="false"/>
                
      was (Author: rwesten):
    When the bundle


is present the following fieldType can be used to index Chinese text

<fieldType name="text_zh" class="solr.TextField" positionIncrementGap="100">
      <analyzer type="index">
        <tokenizer class="solr.SmartChineseSentenceTokenizerFactory"/>
        <filter class="solr.SmartChineseWordTokenFilterFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.SmartChineseSentenceTokenizerFactory"/>
        <filter class="solr.SmartChineseWordTokenFilterFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
        <filter class="solr.PositionFilterFactory" />
      </analyzer>
    </fieldType>

To apply this fieldType to Chinese labels one need also to add

    <!--
     Dynamic field for Chinese languages.
     -->
    <dynamicField name="@zh*"  type="text_zh" indexed="true" stored="true" multiValued="true" omitNorms="false"/>
                  
> Provide SolrCore configuration for the Entityhub SolrYard with smartcn fieldType configuration for Chinese (zh)
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: STANBOL-858
>                 URL: https://issues.apache.org/jira/browse/STANBOL-858
>             Project: Stanbol
>          Issue Type: Sub-task
>            Reporter: Rupert Westenthaler
>            Assignee: Rupert Westenthaler
>
> As the title says this will provide an Entityhub SolrYard compatible schema.xml file that defines a fieldType for indexing Chinese text as well as a dynamic field configuration that matches plain literal values with the language 'zh'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira