You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by tedsolr <ts...@sciquest.com> on 2016/10/27 21:33:25 UTC
Reindex after schema change options
Not all my fields use docValues. This is going to be a problem in the future.
Once I change the schema.xml to use docValues for these certain field types,
how do I reindex the data in place - without starting from the source?
I'm aware of lucene's IndexUpgrader but that will only ensure a correct
lucene match version. I'm not changing that. Could I use SolrJ to walk
through the documents and "touch" each one and do an atomic update on the
fields that have changed? (all the fields I care about are stored)
Thanks, Ted
v5.2.1
--
View this message in context: http://lucene.472066.n3.nabble.com/Reindex-after-schema-change-options-tp4303395.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Reindex after schema change options
Posted by Damien Kamerman <da...@gmail.com>.
I'm in a similar situation where I'm
using org.apache.lucene.index.IndexUpgrader to upgrade an index from solr 4
to solr 6, and want to add docValues to the schema.
All my fields are stored so I assume I could use the DataImportHandler
SolrEntityProcessor to copy the collection to a new collection and pick up
the docValues that way.
Will this work and is there a better (command line) way?
Regards,
Damien
On 29 October 2016 at 09:50, Erick Erickson <er...@gmail.com> wrote:
> This is a little contradictory:
>
> > how do I reindex the data in place - without starting from the source?
>
> > then ran my reindex SolrJ code.
>
> So it looks like you _were_ able to re-index from scratch?
>
> BTW, to be absolutely safe I'd re-index to a _new_ collection and
> then, perhaps, use
> collection aliasing to switch seamlessly. I've seen situations where
> when some segments think fieldX has docValues and some don't it can be
> a problem.
>
> OTOH, if you define a _new_ field with docValues, that's no problem.
>
> And if it worked for you, it worked.....
>
> Best,
> Erick
>
> On Fri, Oct 28, 2016 at 8:09 AM, tedsolr <ts...@sciquest.com> wrote:
> > So I ran a quick test of my idea and it worked. I modified the schema.xml
> > file - uploaded it to ZK - reloaded the collection - then ran my reindex
> > SolrJ code. After it completed the schema browser in the admin console
> shows
> > that the field uses docValues. I tried a streaming expression on it to
> using
> > the /export request handler and that was good - no errors.
> >
> > Still would love to hear from anyone who has done this differently.
> >
> >
> > tedsolr wrote
> >> Not all my fields use docValues. This is going to be a problem in the
> >> future. Once I change the schema.xml to use docValues for these certain
> >> field types, how do I reindex the data in place - without starting from
> >> the source?
> >>
> >> I'm aware of lucene's IndexUpgrader but that will only ensure a correct
> >> lucene match version. I'm not changing that. Could I use SolrJ to walk
> >> through the documents and "touch" each one and do an atomic update on
> the
> >> fields that have changed? (all the fields I care about are stored)
> >>
> >> Thanks, Ted
> >> v5.2.1
> >
> >
> >
> >
> >
> > --
> > View this message in context: http://lucene.472066.n3.
> nabble.com/Reindex-after-schema-change-options-tp4303395p4303510.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Reindex after schema change options
Posted by Erick Erickson <er...@gmail.com>.
This is a little contradictory:
> how do I reindex the data in place - without starting from the source?
> then ran my reindex SolrJ code.
So it looks like you _were_ able to re-index from scratch?
BTW, to be absolutely safe I'd re-index to a _new_ collection and
then, perhaps, use
collection aliasing to switch seamlessly. I've seen situations where
when some segments think fieldX has docValues and some don't it can be
a problem.
OTOH, if you define a _new_ field with docValues, that's no problem.
And if it worked for you, it worked.....
Best,
Erick
On Fri, Oct 28, 2016 at 8:09 AM, tedsolr <ts...@sciquest.com> wrote:
> So I ran a quick test of my idea and it worked. I modified the schema.xml
> file - uploaded it to ZK - reloaded the collection - then ran my reindex
> SolrJ code. After it completed the schema browser in the admin console shows
> that the field uses docValues. I tried a streaming expression on it to using
> the /export request handler and that was good - no errors.
>
> Still would love to hear from anyone who has done this differently.
>
>
> tedsolr wrote
>> Not all my fields use docValues. This is going to be a problem in the
>> future. Once I change the schema.xml to use docValues for these certain
>> field types, how do I reindex the data in place - without starting from
>> the source?
>>
>> I'm aware of lucene's IndexUpgrader but that will only ensure a correct
>> lucene match version. I'm not changing that. Could I use SolrJ to walk
>> through the documents and "touch" each one and do an atomic update on the
>> fields that have changed? (all the fields I care about are stored)
>>
>> Thanks, Ted
>> v5.2.1
>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Reindex-after-schema-change-options-tp4303395p4303510.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Reindex after schema change options
Posted by tedsolr <ts...@sciquest.com>.
So I ran a quick test of my idea and it worked. I modified the schema.xml
file - uploaded it to ZK - reloaded the collection - then ran my reindex
SolrJ code. After it completed the schema browser in the admin console shows
that the field uses docValues. I tried a streaming expression on it to using
the /export request handler and that was good - no errors.
Still would love to hear from anyone who has done this differently.
tedsolr wrote
> Not all my fields use docValues. This is going to be a problem in the
> future. Once I change the schema.xml to use docValues for these certain
> field types, how do I reindex the data in place - without starting from
> the source?
>
> I'm aware of lucene's IndexUpgrader but that will only ensure a correct
> lucene match version. I'm not changing that. Could I use SolrJ to walk
> through the documents and "touch" each one and do an atomic update on the
> fields that have changed? (all the fields I care about are stored)
>
> Thanks, Ted
> v5.2.1
--
View this message in context: http://lucene.472066.n3.nabble.com/Reindex-after-schema-change-options-tp4303395p4303510.html
Sent from the Solr - User mailing list archive at Nabble.com.