You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Kiwi de coder <ki...@gmail.com> on 2011/07/13 16:40:25 UTC

about standardAnaylzer in solr

hi,

I using solr 3.3 which in schema.xml contain this :

    <fieldType name="text_standard" class="solr.TextField">
      <analyzer class="org.apache.lucene.
analysis.standard.StandardAnalyzer"/>
    </fieldType>

i use the sentences as example "XY&Z Corporation - xyz@example.com"

however, when I try on /analysis.jsp, it show difference result compare to
using Lucene.

using solr I got result below when using "text_standard" and "text_general"
(is both the same ?)

XY    Z    Corporation    xyz    example.com (which all belong to <ALPHANUM>
)

when using Lucene, i got this

  StandardAnalyzer:

1: [xy&z:0->4:<COMPANY>]
2: [corporation:5->16:<ALPHANUM>]
3: [xyz@example.com:19->34:<EMAIL>]


so my question is, how to make it analysis like in Lucene ?

regards,
kiwi

Re: about standardAnaylzer in solr

Posted by Kiwi de coder <ki...@gmail.com>.

ok, work now ! thx :)

kiwi

On Wed, Jul 13, 2011 at 11:06 PM, Edoardo Tosca <e....@sourcesense.com>wrote:

> Try to change from StandardTolkenizerFactory to ClassicTokenizerFactory or
> create your own fieldType
>
> <fieldType name="text_general" class="solr.TextField"
> positionIncrementGap="100">
>      <analyzer type="index">
>        *<tokenizer class="solr.ClassicTokenizerFactory"/>*
>        ...
>    </fieldType>
>
>
> Edo
>
> On Wed, Jul 13, 2011 at 3:40 PM, Kiwi de coder <ki...@gmail.com> wrote:
>
> > hi,
> >
> > I using solr 3.3 which in schema.xml contain this :
> >
> >    <fieldType name="text_standard" class="solr.TextField">
> >      <analyzer class="org.apache.lucene.
> > analysis.standard.StandardAnalyzer"/>
> >    </fieldType>
> >
> > i use the sentences as example "XY&Z Corporation - xyz@example.com"
> >
> > however, when I try on /analysis.jsp, it show difference result compare
> to
> > using Lucene.
> >
> > using solr I got result below when using "text_standard" and
> "text_general"
> > (is both the same ?)
> >
> > XY    Z    Corporation    xyz    example.com (which all belong to
> > <ALPHANUM>
> > )
> >
> > when using Lucene, i got this
> >
> >  StandardAnalyzer:
> >
> > 1: [xy&z:0->4:<COMPANY>]
> > 2: [corporation:5->16:<ALPHANUM>]
> > 3: [xyz@example.com:19->34:<EMAIL>]
> >
> >
> > so my question is, how to make it analysis like in Lucene ?
> >
> > regards,
> > kiwi
> >
>
>
>
> --
> Edoardo Tosca
> Sourcesense - making sense of Open Source: http://www.sourcesense.com
>

Re: about standardAnaylzer in solr

Posted by Edoardo Tosca <e....@sourcesense.com>.

Try to change from StandardTolkenizerFactory to ClassicTokenizerFactory or
create your own fieldType

<fieldType name="text_general" class="solr.TextField"
positionIncrementGap="100">
      <analyzer type="index">
        *<tokenizer class="solr.ClassicTokenizerFactory"/>*
        ...
    </fieldType>


Edo

On Wed, Jul 13, 2011 at 3:40 PM, Kiwi de coder <ki...@gmail.com> wrote:

> hi,
>
> I using solr 3.3 which in schema.xml contain this :
>
>    <fieldType name="text_standard" class="solr.TextField">
>      <analyzer class="org.apache.lucene.
> analysis.standard.StandardAnalyzer"/>
>    </fieldType>
>
> i use the sentences as example "XY&Z Corporation - xyz@example.com"
>
> however, when I try on /analysis.jsp, it show difference result compare to
> using Lucene.
>
> using solr I got result below when using "text_standard" and "text_general"
> (is both the same ?)
>
> XY    Z    Corporation    xyz    example.com (which all belong to
> <ALPHANUM>
> )
>
> when using Lucene, i got this
>
>  StandardAnalyzer:
>
> 1: [xy&z:0->4:<COMPANY>]
> 2: [corporation:5->16:<ALPHANUM>]
> 3: [xyz@example.com:19->34:<EMAIL>]
>
>
> so my question is, how to make it analysis like in Lucene ?
>
> regards,
> kiwi
>



-- 
Edoardo Tosca
Sourcesense - making sense of Open Source: http://www.sourcesense.com

Re: about standardAnaylzer in solr

Posted by Chris Hostetter <ho...@fucit.org>.

: I using solr 3.3 which in schema.xml contain this :
	...
: however, when I try on /analysis.jsp, it show difference result compare to
: using Lucene.
	... 
: so my question is, how to make it analysis like in Lucene ?

To be certain of the differences, you need to be more explicit about what 
your low level lucene code looks like.

My guess is that the "Version" param used when constructing 
StandardAnalyzer is different then the <luceneMatchVersion/> in your 
solrconfig.xml (you can override this for a single analyzer using 
<analyzer luceneMatchVersion="..." .../>)


-Hoss