MappingCharFilterFactory weird behaviour

[Lukas Brune <lu...@comintelli.com>]

Hello!

I'm having some troubles with using MappingCharFilterFactory in my schema.
We're using it to replace some escaped html entities
so HTMLStripCharFilterFactory can take care of those.

When testing this out in Analysis it works perfectly, however, when adding
elements to Solr, the mapping doesn't seem to apply.

We're currently copying some other fields into the field with the replaces,
so it's a MultiValued field. (Don't know if that makes a difference)

    <fieldtype name="field" class="solr.TextField"
positionIncrementGap="100">
        <analyzer>
          <charFilter class="solr.MappingCharFilterFactory"
mapping="mapping.txt"/>
          <charFilter class="solr.HTMLStripCharFilterFactory"/>
           // other stuff
        </analyzer>
      </fieldtype>




   <field name="field_name"   type="field"   indexed="true" stored="true"
multiValued="true" required="false" termVectors="true" termPositions="true"
termOffsets="true"/>


Best Regards,
*Lukas Brune* | Machine Learning Engineer & Web Developer | Comintelli AB
lukas.brune@comintelli.com| Mobile:+46(0)706229823 |
www.intelligence2day.com

<https://intelligence2day.com/try/>

Re: MappingCharFilterFactory weird behaviour

[Alexandre Rafalovitch <ar...@gmail.com>]

How do you know it does not apply?

My Doh moment is often forgetting that stored version of the field is not
affected by analyzers. One has to look in schema Admin UI to check indexed
values.

Regards,
   Alex

On Mon., Oct. 5, 2020, 6:01 a.m. Lukas Brune, <lu...@comintelli.com>
wrote:

> Hello!
>
> I'm having some troubles with using MappingCharFilterFactory in my schema.
> We're using it to replace some escaped html entities
> so HTMLStripCharFilterFactory can take care of those.
>
> When testing this out in Analysis it works perfectly, however, when adding
> elements to Solr, the mapping doesn't seem to apply.
>
> We're currently copying some other fields into the field with the replaces,
> so it's a MultiValued field. (Don't know if that makes a difference)
>
>     <fieldtype name="field" class="solr.TextField"
> positionIncrementGap="100">
>         <analyzer>
>           <charFilter class="solr.MappingCharFilterFactory"
> mapping="mapping.txt"/>
>           <charFilter class="solr.HTMLStripCharFilterFactory"/>
>            // other stuff
>         </analyzer>
>       </fieldtype>
>
>
>
>
>    <field name="field_name"   type="field"   indexed="true" stored="true"
> multiValued="true" required="false" termVectors="true" termPositions="true"
> termOffsets="true"/>
>
>
> Best Regards,
> *Lukas Brune* | Machine Learning Engineer & Web Developer | Comintelli AB
> lukas.brune@comintelli.com| Mobile:+46(0)706229823 |
> www.intelligence2day.com
>
> <https://intelligence2day.com/try/>
>