You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Utkarsh Sengar <ut...@gmail.com> on 2013/05/02 23:13:36 UTC

Not able to see newly added copyField in the response (indexing is 80% complete)

Hello,

I updated my schema to use a copyField and have triggered a reindex, 80% of
the reindexing is complete. Although when I query the data, I don't see
"myNewCopyFieldName" being returned with the documents.

Is there something wrong with my schema or I need to wait for the indexing
to complete to see the new copyField?



This is my schema (retracted the actual names):

 <fields>
    <field name="key"  type="string" indexed="true"  stored="true"/>
    <field name="1"  type="string" indexed="true"  stored="true"/>
    <field name="2"  type="string" indexed="true"  stored="true"/>
    <field name="3"  type="string" indexed="false"  stored="true"/>
    <field name="4"  type="string" indexed="true"  stored="true"/>
    <field name="5"  type="string" indexed="true"  stored="true"/>
    <field name="6"  type="custom_type" indexed="true"  stored="true"/>
    <field name="7"  type="text_general" indexed="true"  stored="true"/>
    <field name="8"  type="string" indexed="true"  stored="true"/>
    <field name="9"  type="text_general" indexed="true"  stored="true"/>
    <field name="10"  type="text_general" indexed="true"  stored="true"/>
    <field name="11"  type="string" indexed="true"  stored="true"/>
    <field name="12"  type="string" indexed="true"  stored="true"/>
    <field name="13"  type="string" indexed="true"  stored="true"/>
   <field name="myNewCopyFieldName" type="text_general" indexed="true"
stored="true" multiValued="true"/>
 </fields>
<defaultSearchField>4</defaultSearchField><uniqueKey>key</uniqueKey>

<copyField source="1" dest="myNewCopyFieldName"/><copyField source="2"
dest="myNewCopyFieldName"/><copyField source="3"
dest="myNewCopyFieldName"/><copyField source="4"
dest="myNewCopyFieldName"/><copyField source="6"
dest="myNewCopyFieldName"/>

Where:
<fieldType name="custom_type" class="solr.TextField" positionIncrementGap="100">
   <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer></fieldType>


and


<fieldType name="text_general" class="solr.TextField"
positionIncrementGap="100">
   <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer></fieldType>


-- 
Thanks,
-Utkarsh

Re: Not able to see newly added copyField in the response (indexing is 80% complete)

Posted by Utkarsh Sengar <ut...@gmail.com>.

Thanks Shawn. Find my answers below.


On Thu, May 2, 2013 at 2:34 PM, Shawn Heisey <so...@elyograg.org> wrote:

> On 5/2/2013 3:13 PM, Utkarsh Sengar wrote:
> > Hello,
> >
> > I updated my schema to use a copyField and have triggered a reindex, 80%
> of
> > the reindexing is complete. Although when I query the data, I don't see
> > "myNewCopyFieldName" being returned with the documents.
> >
> > Is there something wrong with my schema or I need to wait for the
> indexing
> > to complete to see the new copyField?
>
> After making sure that you restarted Solr (or reloaded the core) after
> changing your schema, there are two things to mention:
>

Yes, I restarted solr and also did a reload.


> 1) Using stored="true" with a copyField doesn't make any sense, because
> you already have the individual values stored with the source fields.  I
> haven't done any testing, but Solr might ignore stored="true" on
> copyField fields.
>

Ah I see, didn't know about this. If its not stored then it makes sense.
Need to verify this though.


>
> 2) If I'm wrong about how Solr behaves with stored="true" on a
> copyField, then a soft commit (4.x and later) or a hard commit with
> openSearcher=true would be required to see changes from indexing.  Have
> you committed your updates yet?
>

I am using Solr 4.x and soft commit is enabled. So I assume commit happened.
I see this in my solr admin:

   - lastModified:less than a minute ago
   - version:453962
   - numDocs:26413743
   - maxDoc: 28322675
   - current:
   - indexing: yes

So, lastModified = less than minute means the change was committed right?


> Thanks,
> Shawn
>
>


-- 
Thanks,
-Utkarsh

Re: Not able to see newly added copyField in the response (indexing is 80% complete)

Posted by Shawn Heisey <so...@elyograg.org>.

On 5/2/2013 3:13 PM, Utkarsh Sengar wrote:
> Hello,
> 
> I updated my schema to use a copyField and have triggered a reindex, 80% of
> the reindexing is complete. Although when I query the data, I don't see
> "myNewCopyFieldName" being returned with the documents.
> 
> Is there something wrong with my schema or I need to wait for the indexing
> to complete to see the new copyField?

After making sure that you restarted Solr (or reloaded the core) after
changing your schema, there are two things to mention:

1) Using stored="true" with a copyField doesn't make any sense, because
you already have the individual values stored with the source fields.  I
haven't done any testing, but Solr might ignore stored="true" on
copyField fields.

2) If I'm wrong about how Solr behaves with stored="true" on a
copyField, then a soft commit (4.x and later) or a hard commit with
openSearcher=true would be required to see changes from indexing.  Have
you committed your updates yet?

Thanks,
Shawn