You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Vitaliy Avdeev <va...@sistyma.net> on 2010/06/18 15:41:41 UTC
Comma delemitered words shawn in terms like one word.
Hello.
In indexing text I have such string John,Mark,Sam. Then I looks at it in
TermVectorComponent it looks like this johnmarksam.
I am using this type for storing data
<fieldType name="textTight2" class="solr.TextField"
positionIncrementGap="100" >
<analyzer>
<tokenizer class="solr.HTMLStripWhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="false"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="0" generateNumberParts="0" catenateWords="1"
catenateNumbers="1" catenateAll="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
What filter I need to use to get John Mark Sam as different words?
Re: Comma delemitered words shawn in terms like one word.
Posted by abhay kumar <ab...@gmail.com>.
Thanks I was looking exactly for this.
I needed to spli tokens based on comma.
On Fri, Jun 18, 2010 at 10:12 PM, Joe Calderon <ca...@gmail.com>wrote:
> set generateWordParts=1 on wordDelimiter or use
> PatternTokenizerFactory to split on commas
>
>
> http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.PatternTokenizerFactory
>
>
> you can use the analysis page to see what your filter chains are going
> to do before you index
>
> /admin/analysis.jsp
>
> On Fri, Jun 18, 2010 at 6:41 AM, Vitaliy Avdeev <va...@sistyma.net>
> wrote:
> > Hello.
> > In indexing text I have such string John,Mark,Sam. Then I looks at it in
> > TermVectorComponent it looks like this johnmarksam.
> >
> > I am using this type for storing data
> >
> > <fieldType name="textTight2" class="solr.TextField"
> > positionIncrementGap="100" >
> > <analyzer>
> > <tokenizer class="solr.HTMLStripWhitespaceTokenizerFactory"/>
> > <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> > ignoreCase="true" expand="false"/>
> > <filter class="solr.StopFilterFactory" ignoreCase="true"
> > words="stopwords.txt"/>
> > <filter class="solr.WordDelimiterFilterFactory"
> > generateWordParts="0" generateNumberParts="0" catenateWords="1"
> > catenateNumbers="1" catenateAll="0"/>
> > <filter class="solr.LowerCaseFilterFactory"/>
> > <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> > </analyzer>
> > </fieldType>
> >
> > What filter I need to use to get John Mark Sam as different words?
> >
>
--
Thanks and Regards
Abhay Kumar Singh
Re: Comma delemitered words shawn in terms like one word.
Posted by Joe Calderon <ca...@gmail.com>.
set generateWordParts=1 on wordDelimiter or use
PatternTokenizerFactory to split on commas
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.PatternTokenizerFactory
you can use the analysis page to see what your filter chains are going
to do before you index
/admin/analysis.jsp
On Fri, Jun 18, 2010 at 6:41 AM, Vitaliy Avdeev <va...@sistyma.net> wrote:
> Hello.
> In indexing text I have such string John,Mark,Sam. Then I looks at it in
> TermVectorComponent it looks like this johnmarksam.
>
> I am using this type for storing data
>
> <fieldType name="textTight2" class="solr.TextField"
> positionIncrementGap="100" >
> <analyzer>
> <tokenizer class="solr.HTMLStripWhitespaceTokenizerFactory"/>
> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="false"/>
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords.txt"/>
> <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="0" generateNumberParts="0" catenateWords="1"
> catenateNumbers="1" catenateAll="0"/>
> <filter class="solr.LowerCaseFilterFactory"/>
> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
> </analyzer>
> </fieldType>
>
> What filter I need to use to get John Mark Sam as different words?
>