Re: hot deploy of newer version of solr schema in production

[roz dev <ro...@gmail.com>]

Thanks Jan for your inputs.

I am keen to know about the way people keep running live sites while there
is a breaking change which calls for complete re-indexing.
we want to build a new index , with new schema (it may take couple of
hours) without impacting live e-commerce site.

any thoughts are welcome

Thanks
Saroj


On Tue, Jan 24, 2012 at 12:21 AM, Jan Høydahl <ja...@cominvent.com> wrote:

> Hi,
>
> To be able to do a true hot deploy of newer schema without reindexing, you
> must carefully see to that none of your changes are breaking changes. So
> you should test the process on your development machine and make sure it
> works. Adding and deleting fields would work, but not changing the
> field-type or analysis of an existing field. Depending on from/to version,
> you may want to keep the old schema-version number.
>
> The process is:
> 1. Deploy the new schema, including all dependencies such as dictionaries
> 2. Do a RELOAD CORE http://wiki.apache.org/solr/CoreAdmin#RELOAD
>
> My preference is to do a more thorough upgrade of schema including new
> functionality and breaking changes, and then do a full reindex. The
> exception is if my index is huge and the reason for Solr upgrade or schema
> change is to fix a bug, not to use new functionality.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> Solr Training - www.solrtraining.com
>
> On 24. jan. 2012, at 01:51, roz dev wrote:
>
> > Hi All,
> >
> > I need community's feedback about deploying newer versions of solr schema
> > into production while existing (older) schema is in use by applications.
> >
> > How do people perform these things? What has been the learning of people
> > about this.
> >
> > Any thoughts are welcome.
> >
> > Thanks
> > Saroj
>
>

Re: hot deploy of newer version of solr schema in production

[Jan Høydahl <ja...@cominvent.com>]

Hi,

It depends. If you have a small install with 1-2 servers, the easiest is to spin up new servers, upgrade features, reindex, and then make the switch in the frontends.

In very large installs, it's often better to spin up parallell Tomcat instances on new ports (make sure you got enough ram) and do the switch in much the same way as above.

A third option is to create a new core for the new incompatible schema, and when the new core is tested and working, you do a SWAP core command which (in theory) should give you a non-interruptive switch.

However, often you schedule feature and frontend changes to take place at the same time as a major search engine upgrade, so it's not always as easy as switching the backend, you need to synchronize things.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com

On 1. feb. 2012, at 08:30, roz dev wrote:

> Thanks Jan for your inputs.
> 
> I am keen to know about the way people keep running live sites while there
> is a breaking change which calls for complete re-indexing.
> we want to build a new index , with new schema (it may take couple of
> hours) without impacting live e-commerce site.
> 
> any thoughts are welcome
> 
> Thanks
> Saroj
> 
> 
> On Tue, Jan 24, 2012 at 12:21 AM, Jan Høydahl <ja...@cominvent.com> wrote:
> 
>> Hi,
>> 
>> To be able to do a true hot deploy of newer schema without reindexing, you
>> must carefully see to that none of your changes are breaking changes. So
>> you should test the process on your development machine and make sure it
>> works. Adding and deleting fields would work, but not changing the
>> field-type or analysis of an existing field. Depending on from/to version,
>> you may want to keep the old schema-version number.
>> 
>> The process is:
>> 1. Deploy the new schema, including all dependencies such as dictionaries
>> 2. Do a RELOAD CORE http://wiki.apache.org/solr/CoreAdmin#RELOAD
>> 
>> My preference is to do a more thorough upgrade of schema including new
>> functionality and breaking changes, and then do a full reindex. The
>> exception is if my index is huge and the reason for Solr upgrade or schema
>> change is to fix a bug, not to use new functionality.
>> 
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> Solr Training - www.solrtraining.com
>> 
>> On 24. jan. 2012, at 01:51, roz dev wrote:
>> 
>>> Hi All,
>>> 
>>> I need community's feedback about deploying newer versions of solr schema
>>> into production while existing (older) schema is in use by applications.
>>> 
>>> How do people perform these things? What has been the learning of people
>>> about this.
>>> 
>>> Any thoughts are welcome.
>>> 
>>> Thanks
>>> Saroj
>> 
>>