You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jamie Johnson <je...@gmail.com> on 2012/07/06 07:40:15 UTC

Regression of JIRA 1826?

I just upgraded to trunk to try to fix an issue I was having with the
highlighter described in JIRA 1826, but it appears that this issue
still exists on trunk.  I'm running the following query

subject:ztest*

subject is a text field (not multivalued) and the return in highlighting is

<em>ZTest</em>For<em>ZTestForJamie</em>

the actual stored value is "ZTestForJamie".  Is anyone else experiencing this?

Re: Regression of JIRA 1826?

Posted by Jamie Johnson <je...@gmail.com>.
I have not seen any recent activity on this, has anyone else ran into this
or been able to duplicate this problem?
On Jul 9, 2012 3:09 PM, "Jamie Johnson" <je...@gmail.com> wrote:

> Thanks Lance, attached is a trimmed down version of my schema and a
> print out of the object that exhibits the issue.  Again if I put
> splitOnCaseChange = 0 on the text field I don't see the same issue.
>
> sample doc
>
> SolrInputDocument[key=1, datetime_dt=Mon Jul 09 15:07:32 EDT 2012,
> type=1, subject_txt=PowerShot.com]
>
> code
>
>                 SolrInputDocument doc = new SolrInputDocument();
>                 doc.addField("key", "1");
>                 doc.addField("datetime_dt", new Date());
>                 doc.addField("type", "1");
>                 doc.addField("subject_txt", "PowerShot.com");
>
> schema.xml
> <?xml version="1.0" encoding="UTF-8" ?>
>
> <schema name="example" version="1.3">
>  <fields>
>
>    <field name="_version_" type="long" indexed="true" stored="true"
> required="false" />
>    <field name="key" type="string" indexed="true" stored="true"
> required="true" />
>    <field name="type" type="string" indexed="true" stored="true"
> required="true" />
>    <field name="subject_txt" type="text" stored="true" indexed="true"
> required="true" termVectors="true" termPositions="true"
> termOffsets="true" default=" Unknown"/>
>    <field name="datetime_dt" type="date" indexed="true" stored="true"
> required="true"/>
>
>    <dynamicField name="*_txt" type="text"    indexed="true"  stored="true"
> />
>    <dynamicField name="*_dt" type="date"    indexed="true"  stored="true"/>
>  </fields>
>
>  <uniqueKey>key</uniqueKey>
>
>  <defaultSearchField>content_mvtxt</defaultSearchField>
>
>   <types>
>     <fieldType name="long" class="solr.TrieLongField"
> precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
>     <fieldType name="string" class="solr.StrField"
> sortMissingLast="true" omitNorms="true"/>
>     <fieldType name="date" class="solr.TrieDateField" omitNorms="true"
> precisionStep="0" positionIncrementGap="0"/>
>
>     <fieldType name="text" class="solr.TextField"
> positionIncrementGap="100" autoGeneratePhraseQueries="true">
>       <analyzer type="index">
>         <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>         <filter class="solr.StopFilterFactory"
>                 ignoreCase="true"
>                 words="stopwords.txt"
>                 enablePositionIncrements="true"
>                 />
>         <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
>         <filter class="solr.LowerCaseFilterFactory"/>
>         <filter class="solr.KeywordMarkerFilterFactory"
> protected="protwords.txt"/>
>         <filter class="solr.PorterStemFilterFactory"/>
>       </analyzer>
>       <analyzer type="query">
>         <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>         <filter class="solr.SynonymFilterFactory"
> synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
>         <filter class="solr.StopFilterFactory"
>                 ignoreCase="true"
>                 words="stopwords.txt"
>                 enablePositionIncrements="true"
>                 />
>         <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="1" generateNumberParts="1" catenateWords="0"
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
>         <filter class="solr.LowerCaseFilterFactory"/>
>         <filter class="solr.KeywordMarkerFilterFactory"
> protected="protwords.txt"/>
>         <filter class="solr.PorterStemFilterFactory"/>
>       </analyzer>
>     </fieldType>
>
>  </types>
> </schema>
>
>
>
> On Sun, Jul 8, 2012 at 11:57 PM, Lance Norskog <go...@gmail.com> wrote:
> > Please post a trimmed-down version of your schema.xml and a sample
> document.
> >
> > On Sun, Jul 8, 2012 at 11:54 AM, Jamie Johnson <je...@gmail.com>
> wrote:
> >> Is there any more information that folks need to dig into this?  I
> >> have been unable to this point to figure out what specifically it is
> >> happening, so would appreciate any help.
> >>
> >> On Fri, Jul 6, 2012 at 2:13 PM, Jamie Johnson <je...@gmail.com>
> wrote:
> >>> A little more information on this.
> >>>
> >>> I tinkered a bit with the schema and it appears to be related to
> >>> WordDelimiterFilterFactory and splitOnCaseChange being true, or at
> >>> least this setting being set exhibits the issue.
> >>>
> >>> Also I am using the edismax query parser.  Again any ideas/help would
> >>> be greatly appreciated.
> >>>
> >>> On Fri, Jul 6, 2012 at 1:40 AM, Jamie Johnson <je...@gmail.com>
> wrote:
> >>>> I just upgraded to trunk to try to fix an issue I was having with the
> >>>> highlighter described in JIRA 1826, but it appears that this issue
> >>>> still exists on trunk.  I'm running the following query
> >>>>
> >>>> subject:ztest*
> >>>>
> >>>> subject is a text field (not multivalued) and the return in
> highlighting is
> >>>>
> >>>> <em>ZTest</em>For<em>ZTestForJamie</em>
> >>>>
> >>>> the actual stored value is "ZTestForJamie".  Is anyone else
> experiencing this?
> >
> >
> >
> > --
> > Lance Norskog
> > goksron@gmail.com
>

Re: Regression of JIRA 1826?

Posted by Jamie Johnson <je...@gmail.com>.
Thanks Lance, attached is a trimmed down version of my schema and a
print out of the object that exhibits the issue.  Again if I put
splitOnCaseChange = 0 on the text field I don't see the same issue.

sample doc

SolrInputDocument[key=1, datetime_dt=Mon Jul 09 15:07:32 EDT 2012,
type=1, subject_txt=PowerShot.com]

code

		SolrInputDocument doc = new SolrInputDocument();
		doc.addField("key", "1");
		doc.addField("datetime_dt", new Date());
		doc.addField("type", "1");
		doc.addField("subject_txt", "PowerShot.com");

schema.xml
<?xml version="1.0" encoding="UTF-8" ?>

<schema name="example" version="1.3">
 <fields>

   <field name="_version_" type="long" indexed="true" stored="true"
required="false" />
   <field name="key" type="string" indexed="true" stored="true"
required="true" />
   <field name="type" type="string" indexed="true" stored="true"
required="true" />
   <field name="subject_txt" type="text" stored="true" indexed="true"
required="true" termVectors="true" termPositions="true"
termOffsets="true" default=" Unknown"/>
   <field name="datetime_dt" type="date" indexed="true" stored="true"
required="true"/>

   <dynamicField name="*_txt" type="text"    indexed="true"  stored="true" />
   <dynamicField name="*_dt" type="date"    indexed="true"  stored="true"/>
 </fields>

 <uniqueKey>key</uniqueKey>

 <defaultSearchField>content_mvtxt</defaultSearchField>

  <types>
    <fieldType name="long" class="solr.TrieLongField"
precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
    <fieldType name="string" class="solr.StrField"
sortMissingLast="true" omitNorms="true"/>
    <fieldType name="date" class="solr.TrieDateField" omitNorms="true"
precisionStep="0" positionIncrementGap="0"/>

    <fieldType name="text" class="solr.TextField"
positionIncrementGap="100" autoGeneratePhraseQueries="true">
      <analyzer type="index">
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <filter class="solr.StopFilterFactory"
                ignoreCase="true"
                words="stopwords.txt"
                enablePositionIncrements="true"
                />
        <filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.KeywordMarkerFilterFactory"
protected="protwords.txt"/>
        <filter class="solr.PorterStemFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <filter class="solr.SynonymFilterFactory"
synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
        <filter class="solr.StopFilterFactory"
                ignoreCase="true"
                words="stopwords.txt"
                enablePositionIncrements="true"
                />
        <filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="0"
catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.KeywordMarkerFilterFactory"
protected="protwords.txt"/>
        <filter class="solr.PorterStemFilterFactory"/>
      </analyzer>
    </fieldType>

 </types>
</schema>



On Sun, Jul 8, 2012 at 11:57 PM, Lance Norskog <go...@gmail.com> wrote:
> Please post a trimmed-down version of your schema.xml and a sample document.
>
> On Sun, Jul 8, 2012 at 11:54 AM, Jamie Johnson <je...@gmail.com> wrote:
>> Is there any more information that folks need to dig into this?  I
>> have been unable to this point to figure out what specifically it is
>> happening, so would appreciate any help.
>>
>> On Fri, Jul 6, 2012 at 2:13 PM, Jamie Johnson <je...@gmail.com> wrote:
>>> A little more information on this.
>>>
>>> I tinkered a bit with the schema and it appears to be related to
>>> WordDelimiterFilterFactory and splitOnCaseChange being true, or at
>>> least this setting being set exhibits the issue.
>>>
>>> Also I am using the edismax query parser.  Again any ideas/help would
>>> be greatly appreciated.
>>>
>>> On Fri, Jul 6, 2012 at 1:40 AM, Jamie Johnson <je...@gmail.com> wrote:
>>>> I just upgraded to trunk to try to fix an issue I was having with the
>>>> highlighter described in JIRA 1826, but it appears that this issue
>>>> still exists on trunk.  I'm running the following query
>>>>
>>>> subject:ztest*
>>>>
>>>> subject is a text field (not multivalued) and the return in highlighting is
>>>>
>>>> <em>ZTest</em>For<em>ZTestForJamie</em>
>>>>
>>>> the actual stored value is "ZTestForJamie".  Is anyone else experiencing this?
>
>
>
> --
> Lance Norskog
> goksron@gmail.com

Re: Regression of JIRA 1826?

Posted by Lance Norskog <go...@gmail.com>.
Please post a trimmed-down version of your schema.xml and a sample document.

On Sun, Jul 8, 2012 at 11:54 AM, Jamie Johnson <je...@gmail.com> wrote:
> Is there any more information that folks need to dig into this?  I
> have been unable to this point to figure out what specifically it is
> happening, so would appreciate any help.
>
> On Fri, Jul 6, 2012 at 2:13 PM, Jamie Johnson <je...@gmail.com> wrote:
>> A little more information on this.
>>
>> I tinkered a bit with the schema and it appears to be related to
>> WordDelimiterFilterFactory and splitOnCaseChange being true, or at
>> least this setting being set exhibits the issue.
>>
>> Also I am using the edismax query parser.  Again any ideas/help would
>> be greatly appreciated.
>>
>> On Fri, Jul 6, 2012 at 1:40 AM, Jamie Johnson <je...@gmail.com> wrote:
>>> I just upgraded to trunk to try to fix an issue I was having with the
>>> highlighter described in JIRA 1826, but it appears that this issue
>>> still exists on trunk.  I'm running the following query
>>>
>>> subject:ztest*
>>>
>>> subject is a text field (not multivalued) and the return in highlighting is
>>>
>>> <em>ZTest</em>For<em>ZTestForJamie</em>
>>>
>>> the actual stored value is "ZTestForJamie".  Is anyone else experiencing this?



-- 
Lance Norskog
goksron@gmail.com

Re: Regression of JIRA 1826?

Posted by Jamie Johnson <je...@gmail.com>.
Is there any more information that folks need to dig into this?  I
have been unable to this point to figure out what specifically it is
happening, so would appreciate any help.

On Fri, Jul 6, 2012 at 2:13 PM, Jamie Johnson <je...@gmail.com> wrote:
> A little more information on this.
>
> I tinkered a bit with the schema and it appears to be related to
> WordDelimiterFilterFactory and splitOnCaseChange being true, or at
> least this setting being set exhibits the issue.
>
> Also I am using the edismax query parser.  Again any ideas/help would
> be greatly appreciated.
>
> On Fri, Jul 6, 2012 at 1:40 AM, Jamie Johnson <je...@gmail.com> wrote:
>> I just upgraded to trunk to try to fix an issue I was having with the
>> highlighter described in JIRA 1826, but it appears that this issue
>> still exists on trunk.  I'm running the following query
>>
>> subject:ztest*
>>
>> subject is a text field (not multivalued) and the return in highlighting is
>>
>> <em>ZTest</em>For<em>ZTestForJamie</em>
>>
>> the actual stored value is "ZTestForJamie".  Is anyone else experiencing this?

Re: Regression of JIRA 1826?

Posted by Jamie Johnson <je...@gmail.com>.
A little more information on this.

I tinkered a bit with the schema and it appears to be related to
WordDelimiterFilterFactory and splitOnCaseChange being true, or at
least this setting being set exhibits the issue.

Also I am using the edismax query parser.  Again any ideas/help would
be greatly appreciated.

On Fri, Jul 6, 2012 at 1:40 AM, Jamie Johnson <je...@gmail.com> wrote:
> I just upgraded to trunk to try to fix an issue I was having with the
> highlighter described in JIRA 1826, but it appears that this issue
> still exists on trunk.  I'm running the following query
>
> subject:ztest*
>
> subject is a text field (not multivalued) and the return in highlighting is
>
> <em>ZTest</em>For<em>ZTestForJamie</em>
>
> the actual stored value is "ZTestForJamie".  Is anyone else experiencing this?