How to reindex in solr

[Kashif Khan <up...@gmail.com>]

Hi all,

I have my solr indexed completely and now i have added a new field in the
schema which is a copyfield of another field. Please suggest me how can i
reindex solr without going through formal process which i did for the first
time because there are some fields whose data is really time consuming to
obtain. I have been trying to reindex from solrj and indexes well except for
those fields which have been mentioned as stored=false. Now on the
production servers these indexing is getting failed because of the out of
memory swap space. Please suggest some good method to reindex using lucene
indexes with even stored=false.

--
View this message in context: http://lucene.472066.n3.nabble.com/How-to-reindex-in-solr-tp3550871p3550871.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to reindex in solr

[Erick Erickson <er...@gmail.com>]

> Is there a formal manner to transfer the data to a database or file-format> from which it can be reloaded?>> I would say an export to a CSV file (which could become huge) and the reload> it from that?
Not quite sure what you mean by that. The data is not *in* the
solr index if it's not stored, so if you're talking about exporting
from Solr, this won't work either.

Solr presumes that the system-of-record is somewhere
else, it was never intended as a raw data store. You could
store all your fields to accomplish this, but then you have
to be very, very, very sure your index is preserved
appropriately.

Otherwise, you could store Solr documents on disk. You could
put it in an RDBMS. You could....

Best
Erick


On Sat, Dec 3, 2011 at 6:06 AM, RT <rw...@gmail.com> wrote:
> Hi,
>
> production servers these indexing is getting failed because of the out of
>>>
>>> memory swap space. Please suggest some good method to reindex using
>>> lucene
>>> indexes with even stored=false.
>>
>>
>> It is not possible to reindex fields which were not stored. You will
>> need to go back to the original data source.
>
>
> This is a major contraint in some cases. In my case a lot of the data is
> user-generated and does not have another source to index from.
>
> Is there a formal manner to transfer the data to a database or file-format
> from which it can be reloaded?
>
> I would say an export to a CSV file (which could become huge) and the reload
> it from that?
>
> Roalnd
>
>

Re: How to reindex in solr

[RT <rw...@gmail.com>]

Hi,
production servers these indexing is getting failed because of the out of
>> memory swap space. Please suggest some good method to reindex using lucene
>> indexes with even stored=false.
> 
> It is not possible to reindex fields which were not stored. You will
> need to go back to the original data source.

This is a major contraint in some cases. In my case a lot of the data is 
user-generated and does not have another source to index from.

Is there a formal manner to transfer the data to a database or 
file-format from which it can be reloaded?

I would say an export to a CSV file (which could become huge) and the 
reload it from that?

Roalnd

Re: How to reindex in solr

[Gora Mohanty <go...@mimirtech.com>]

On Thu, Dec 1, 2011 at 2:00 PM, Kashif Khan <up...@gmail.com> wrote:
> Hi all,
>
> I have my solr indexed completely and now i have added a new field in the
> schema which is a copyfield of another field. Please suggest me how can i
> reindex solr without going through formal process which i did for the first
> time because there are some fields whose data is really time consuming to
> obtain. I have been trying to reindex from solrj and indexes well except for
> those fields which have been mentioned as stored=false. Now on the
> production servers these indexing is getting failed because of the out of
> memory swap space. Please suggest some good method to reindex using lucene
> indexes with even stored=false.

It is not possible to reindex fields which were not stored. You will
need to go back to the original data source.

Regards,
Gora