You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Mike <mz...@gmail.com> on 2022/07/11 11:28:57 UTC
Transfer to a new server
Hello!
How can I transfer a 500 GB Solr index to a new server?
Thanks
Re: Transfer to a new server
Posted by Dave <ha...@gmail.com>.
You could put it on a hard drive and mail it. Or just, over the internet using replication
> On Jul 11, 2022, at 7:45 AM, Thomas Corthals <th...@klascement.net> wrote:
>
> Hello Mike,
>
> If possible, just rebuild it from the original source on the new server.
>
> Regards,
>
> Thomas
>
> Op ma 11 jul. 2022 om 13:28 schreef Mike <mz...@gmail.com>:
>
>> Hello!
>>
>> How can I transfer a 500 GB Solr index to a new server?
>>
>> Thanks
>>
Re: Transfer to a new server
Posted by Gus Heck <gu...@gmail.com>.
Plenty of folks rebuild multiple TB indexes in under a week. Pain mostly
comes from not having understood the need to support such an activity early
enough in the process.
On Mon, Jul 11, 2022 at 8:25 AM Dave <ha...@gmail.com> wrote:
> Ideally yes, but I feel that pain of trying to rebuild a 500gb index and
> another 500 of stored full text along side of it, plus another couple
> hundred gigs of a supporting index, and all from different sources.
>
> But yes it should be able to be done in a week or two. I never upgrade
> the solr servers without a full reindex.
>
> > On Jul 11, 2022, at 8:20 AM, Gus Heck <gu...@gmail.com> wrote:
> >
> > After you complete this move you should address this. It's not good to
> run
> > a server that can't be re-indexed within a reasonable amount of time.
> Such
> > a situation means you will never be able to take advantage of new index
> > features, never be able to change the way existing fields are analyzed,
> and
> > any corruption (very rare but not zero chance). a serious human error,
> or a
> > malicious attack could mean weeks of downtime. Ability to re-index within
> > an acceptable time period is a key part of managing a search system.
> >
> >> On Mon, Jul 11, 2022 at 7:47 AM Mike <mz...@gmail.com> wrote:
> >>
> >> Hello Thomas,
> >>
> >> that would take weeks.
> >>
> >> Mike
> >>
> >> Am Mo., 11. Juli 2022 um 13:45 Uhr schrieb Thomas Corthals <
> >> thomas@klascement.net>:
> >>
> >>> Hello Mike,
> >>>
> >>> If possible, just rebuild it from the original source on the new
> server.
> >>>
> >>> Regards,
> >>>
> >>> Thomas
> >>>
> >>> Op ma 11 jul. 2022 om 13:28 schreef Mike <mz...@gmail.com>:
> >>>
> >>>> Hello!
> >>>>
> >>>> How can I transfer a 500 GB Solr index to a new server?
> >>>>
> >>>> Thanks
> >>>>
> >>>
> >>
> >
> >
> > --
> > http://www.needhamsoftware.com (work)
> > http://www.the111shift.com (play)
>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Transfer to a new server
Posted by Dave <ha...@gmail.com>.
Ideally yes, but I feel that pain of trying to rebuild a 500gb index and another 500 of stored full text along side of it, plus another couple hundred gigs of a supporting index, and all from different sources.
But yes it should be able to be done in a week or two. I never upgrade the solr servers without a full reindex.
> On Jul 11, 2022, at 8:20 AM, Gus Heck <gu...@gmail.com> wrote:
>
> After you complete this move you should address this. It's not good to run
> a server that can't be re-indexed within a reasonable amount of time. Such
> a situation means you will never be able to take advantage of new index
> features, never be able to change the way existing fields are analyzed, and
> any corruption (very rare but not zero chance). a serious human error, or a
> malicious attack could mean weeks of downtime. Ability to re-index within
> an acceptable time period is a key part of managing a search system.
>
>> On Mon, Jul 11, 2022 at 7:47 AM Mike <mz...@gmail.com> wrote:
>>
>> Hello Thomas,
>>
>> that would take weeks.
>>
>> Mike
>>
>> Am Mo., 11. Juli 2022 um 13:45 Uhr schrieb Thomas Corthals <
>> thomas@klascement.net>:
>>
>>> Hello Mike,
>>>
>>> If possible, just rebuild it from the original source on the new server.
>>>
>>> Regards,
>>>
>>> Thomas
>>>
>>> Op ma 11 jul. 2022 om 13:28 schreef Mike <mz...@gmail.com>:
>>>
>>>> Hello!
>>>>
>>>> How can I transfer a 500 GB Solr index to a new server?
>>>>
>>>> Thanks
>>>>
>>>
>>
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
Re: Transfer to a new server
Posted by Gus Heck <gu...@gmail.com>.
After you complete this move you should address this. It's not good to run
a server that can't be re-indexed within a reasonable amount of time. Such
a situation means you will never be able to take advantage of new index
features, never be able to change the way existing fields are analyzed, and
any corruption (very rare but not zero chance). a serious human error, or a
malicious attack could mean weeks of downtime. Ability to re-index within
an acceptable time period is a key part of managing a search system.
On Mon, Jul 11, 2022 at 7:47 AM Mike <mz...@gmail.com> wrote:
> Hello Thomas,
>
> that would take weeks.
>
> Mike
>
> Am Mo., 11. Juli 2022 um 13:45 Uhr schrieb Thomas Corthals <
> thomas@klascement.net>:
>
> > Hello Mike,
> >
> > If possible, just rebuild it from the original source on the new server.
> >
> > Regards,
> >
> > Thomas
> >
> > Op ma 11 jul. 2022 om 13:28 schreef Mike <mz...@gmail.com>:
> >
> > > Hello!
> > >
> > > How can I transfer a 500 GB Solr index to a new server?
> > >
> > > Thanks
> > >
> >
>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Transfer to a new server
Posted by Mike <mz...@gmail.com>.
Hello Thomas,
that would take weeks.
Mike
Am Mo., 11. Juli 2022 um 13:45 Uhr schrieb Thomas Corthals <
thomas@klascement.net>:
> Hello Mike,
>
> If possible, just rebuild it from the original source on the new server.
>
> Regards,
>
> Thomas
>
> Op ma 11 jul. 2022 om 13:28 schreef Mike <mz...@gmail.com>:
>
> > Hello!
> >
> > How can I transfer a 500 GB Solr index to a new server?
> >
> > Thanks
> >
>
Re: Transfer to a new server
Posted by Thomas Corthals <th...@klascement.net>.
Hello Mike,
If possible, just rebuild it from the original source on the new server.
Regards,
Thomas
Op ma 11 jul. 2022 om 13:28 schreef Mike <mz...@gmail.com>:
> Hello!
>
> How can I transfer a 500 GB Solr index to a new server?
>
> Thanks
>
Re: Transfer to a new server
Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/11/22 05:28, Mike wrote:
> How can I transfer a 500 GB Solr index to a new server?
If it's the same Solr version, just copy the index directory to the new
server. Ideally you will either shut Solr down during the copy, or make
VERY sure that there are no index changes happening during the copy. An
alternate solution to that would be to shut Solr down or turn indexing
off, make a hardlink copy of the index directory, resume normal
operations, and then duplicate the hardlink copy. Once the transfer is
done, delete the hardlink copy. Making or deleting a hardlink copy
should happen almost instantaneously.
If the target Solr is a different version than the source, then it is
best to completely reindex, even if the existing index can be read by
the new version.
Thanks,
Shawn