Updating schema.xml without deleting index?

[Jeff Lerman <je...@blip.tv>]

I've tried searching for this answer all over but have found no results
thus far.  I am trying to add a new field to my schema.xml with a
default value of 0.  I have a ton of data indexed right now and it would
be very hard to retrieve all of the original sources to rebuild my
index.  So my question is...is there any way to send a command to SOLR
that tells it to re-index everything it has and include the new field I
added?

 

Thanks,

 

Jeff

RE: Updating schema.xml without deleting index?

["Nguyen, Joe" <jn...@automotive.com>]

Don't know whether this would work... Just speculate :-)

A.  You'll need to create a new schema with the new field or you could
use dynamic field in your current schema (assume you already config the
default value to 0).
B.  Add a couple of new documents
C.  Run optimize script.  Since optimize will consolidate all segments
into a single segment.  At the end, you'll have a single segment which
include the new field. 

Would that work?

-----Original Message-----
From: Jeff Lerman [mailto:jeff@blip.tv] 
Sent: Monday, November 17, 2008 12:45 Joe
To: solr-user@lucene.apache.org
Subject: Updating schema.xml without deleting index?

I've tried searching for this answer all over but have found no results
thus far.  I am trying to add a new field to my schema.xml with a
default value of 0.  I have a ton of data indexed right now and it would
be very hard to retrieve all of the original sources to rebuild my
index.  So my question is...is there any way to send a command to SOLR
that tells it to re-index everything it has and include the new field I
added?

 

Thanks,

 

Jeff

RE: Updating schema.xml without deleting index?

[Chris Hostetter <ho...@fucit.org>]

: of myfield as the same result.  I wish there was an option to just
: completely reindex all data..i suppose optimize may do that a little
: bit?

"optimize" is just a low level lucene call to purge all deleted docs and 
merge all index segments into a single segment.

and there is an option to reindex all data: take whatever you used to 
index in the data the first time, and do it again. :)

seriously though, if you use something like DateImportHandler this is 
fairly easy, if you don't use something like DIH, it's a matter of 
designing whatever system you do use so that it's easy do reindex later as 
needed (unless you're certain that your schema is perfect and never needs 
to change)

The way you solved your use case (exclude things that don't have a value) 
is exactly how i go about deal with situations like this routinely.



-Hoss

RE: Updating schema.xml without deleting index?

[Jeff Lerman <je...@blip.tv>]

Yep seems this is the case.  I found a work-around in case anyone wants
to know.  I adjusted my query to include a fq of '-myfield:1'.  So
pretty much, all my data that was in there already would have had a
value of '0' for myfield so I can treat either myfield:0 or the absence
of myfield as the same result.  I wish there was an option to just
completely reindex all data..i suppose optimize may do that a little
bit?

--Jeff

-----Original Message-----
From: Chris Hostetter [mailto:hossman_lucene@fucit.org] 
Sent: Friday, November 21, 2008 4:47 PM
To: solr-user@lucene.apache.org
Subject: Re: Updating schema.xml without deleting index?


: I've tried searching for this answer all over but have found no
results
: thus far.  I am trying to add a new field to my schema.xml with a
: default value of 0.  I have a ton of data indexed right now and it
would
: be very hard to retrieve all of the original sources to rebuild my
: index.  So my question is...is there any way to send a command to SOLR
: that tells it to re-index everything it has and include the new field
I
: added?

default values only affect documents as they are indexed.  you can add 
that field, without reindexing anything, and solr will work, but those 
docs still won't have that value.  you have to manually re-index 
everything for them to get it.



-Hoss

Re: Updating schema.xml without deleting index?

[Chris Hostetter <ho...@fucit.org>]

: I've tried searching for this answer all over but have found no results
: thus far.  I am trying to add a new field to my schema.xml with a
: default value of 0.  I have a ton of data indexed right now and it would
: be very hard to retrieve all of the original sources to rebuild my
: index.  So my question is...is there any way to send a command to SOLR
: that tells it to re-index everything it has and include the new field I
: added?

default values only affect documents as they are indexed.  you can add 
that field, without reindexing anything, and solr will work, but those 
docs still won't have that value.  you have to manually re-index 
everything for them to get it.



-Hoss