You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Isan Fulia <is...@germinait.com> on 2011/10/04 07:41:01 UTC
Re: Query failing because of omitTermFreqAndPositions
Hi Mike,
Thanks for the information.But why is it that once omiited positions in the
past , it will always omit positions
even if omitPositions is made false.
Thanks,
Isan Fulia.
On 29 September 2011 17:49, Michael McCandless <lu...@mikemccandless.com>wrote:
> Once a given field has omitted positions in the past, even for just
> one document, it "sticks" and that field will forever omit positions.
>
> Try creating a new index, never omitting positions from that field?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Thu, Sep 29, 2011 at 1:14 AM, Isan Fulia <is...@germinait.com>
> wrote:
> > Hi All,
> >
> > My schema consisted of field textForQuery which was defined as
> > <field name="textForQuery" type="text" indexed="true" stored="false"
> > multiValued="true"/>
> >
> > After indexing 10 lakhs of documents I changed the field to
> > <field name="textForQuery" type="text" indexed="true" stored="false"
> > multiValued="true" *omitTermFreqAndPositions="true"*/>
> >
> > So documents that were indexed after that omiited the position
> information
> > of the terms.
> > As a result I was not able to search the text which rely on position
> > information for eg. "coke studio at mtv" even though its present in some
> > documents.
> >
> > So I again changed the field textForQuery to
> > <field name="textForQuery" type="text" indexed="true" stored="false"
> > multiValued="true"/>
> >
> > But now even for new documents added the query requiring positon
> > information is still failing.
> > For example i reindexed certain documents that consisted of "coke studio
> at
> > mtv" but still the query is not returning any documents when searched for
> > *textForQuery:"coke studio at mtv"*
> >
> > Can anyone please help me out why this is happening
> >
> >
> > --
> > Thanks & Regards,
> > Isan Fulia.
> >
>
--
Thanks & Regards,
Isan Fulia.
Re: Query failing because of omitTermFreqAndPositions
Posted by Michael McCandless <lu...@mikemccandless.com>.
This is because, within one segment only 1 value (omitP or not) is
possible, for all the docs in that segment.
This then means, on merging segments with different values for omitP,
Lucene must "reconcile" the different values, and that reconciliation
will favor omitting positions (if it went the other way, Lucene would
have to make up fake positions, which seems very dangerous).
Even if you delete all documents containing that field, and optimize
down to one segment, this "omitPositions" bit will still stick,
because of how Lucene stores the metadata per field.
omitNorms also behaves this way: once omitted, always omitted.
Mike McCandless
http://blog.mikemccandless.com
On Tue, Oct 4, 2011 at 1:41 AM, Isan Fulia <is...@germinait.com> wrote:
> Hi Mike,
>
> Thanks for the information.But why is it that once omiited positions in the
> past , it will always omit positions
> even if omitPositions is made false.
>
> Thanks,
> Isan Fulia.
>
> On 29 September 2011 17:49, Michael McCandless <lu...@mikemccandless.com>wrote:
>
>> Once a given field has omitted positions in the past, even for just
>> one document, it "sticks" and that field will forever omit positions.
>>
>> Try creating a new index, never omitting positions from that field?
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Thu, Sep 29, 2011 at 1:14 AM, Isan Fulia <is...@germinait.com>
>> wrote:
>> > Hi All,
>> >
>> > My schema consisted of field textForQuery which was defined as
>> > <field name="textForQuery" type="text" indexed="true" stored="false"
>> > multiValued="true"/>
>> >
>> > After indexing 10 lakhs of documents I changed the field to
>> > <field name="textForQuery" type="text" indexed="true" stored="false"
>> > multiValued="true" *omitTermFreqAndPositions="true"*/>
>> >
>> > So documents that were indexed after that omiited the position
>> information
>> > of the terms.
>> > As a result I was not able to search the text which rely on position
>> > information for eg. "coke studio at mtv" even though its present in some
>> > documents.
>> >
>> > So I again changed the field textForQuery to
>> > <field name="textForQuery" type="text" indexed="true" stored="false"
>> > multiValued="true"/>
>> >
>> > But now even for new documents added the query requiring positon
>> > information is still failing.
>> > For example i reindexed certain documents that consisted of "coke studio
>> at
>> > mtv" but still the query is not returning any documents when searched for
>> > *textForQuery:"coke studio at mtv"*
>> >
>> > Can anyone please help me out why this is happening
>> >
>> >
>> > --
>> > Thanks & Regards,
>> > Isan Fulia.
>> >
>>
>
>
>
> --
> Thanks & Regards,
> Isan Fulia.
>