Solr index segment level merge

[jeba earnest <er...@gmail.com>]

I have a scenario that I need to merge the solr indexes online. I have a
primary solr index of 100 Gb and it is serving the end users and it can't
go offline for a moment. Everyday new lucene indexes(2 GB) are generated
separately.

I have tried coreadmin
https://cwiki.apache.org/confluence/display/solr/Merging+Indexes

And it will create a new core or new folder. which means it will copy 100Gb
every time to a new folder.

Is there a way I can do a segment level merging?

Jeba

Re: Solr index segment level merge

[Tomás Fernández Löbbe <to...@gmail.com>]

Would collection aliases be an option (assuming you are using SolrCloud
mode)?


https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api4

On Tue, Dec 29, 2015 at 9:21 PM, Erick Erickson <er...@gmail.com>
wrote:

> Could you simply add the new documents to the current index?
>
> That aside, merging does not need to create a new core or a new
> folder. The form:
>
>
> mergeindexes&core=core0&indexDir=/opt/solr/core1/data/index&indexDir=/opt/solr/core2/data/index
>
> Should merge the indexes from the two directories into the pre-existing
> core's index.
>
> Best,
> Erick
>
> On Tue, Dec 29, 2015 at 9:00 PM, Walter Underwood <wu...@wunderwood.org>
> wrote:
> > You probably do not NEED to merge your indexes. Have you tried not
> merging the indexes?
> >
> > wunder
> > Walter Underwood
> > wunder@wunderwood.org
> > http://observer.wunderwood.org/  (my blog)
> >
> >> On Dec 29, 2015, at 7:31 PM, jeba earnest <er...@gmail.com>
> wrote:
> >>
> >> I have a scenario that I need to merge the solr indexes online. I have a
> >> primary solr index of 100 Gb and it is serving the end users and it
> can't
> >> go offline for a moment. Everyday new lucene indexes(2 GB) are generated
> >> separately.
> >>
> >> I have tried coreadmin
> >> https://cwiki.apache.org/confluence/display/solr/Merging+Indexes
> >>
> >> And it will create a new core or new folder. which means it will copy
> 100Gb
> >> every time to a new folder.
> >>
> >> Is there a way I can do a segment level merging?
> >>
> >> Jeba
> >
>

Re: Solr index segment level merge

[Erick Erickson <er...@gmail.com>]

Could you simply add the new documents to the current index?

That aside, merging does not need to create a new core or a new
folder. The form:

mergeindexes&core=core0&indexDir=/opt/solr/core1/data/index&indexDir=/opt/solr/core2/data/index

Should merge the indexes from the two directories into the pre-existing
core's index.

Best,
Erick

On Tue, Dec 29, 2015 at 9:00 PM, Walter Underwood <wu...@wunderwood.org> wrote:
> You probably do not NEED to merge your indexes. Have you tried not merging the indexes?
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>> On Dec 29, 2015, at 7:31 PM, jeba earnest <er...@gmail.com> wrote:
>>
>> I have a scenario that I need to merge the solr indexes online. I have a
>> primary solr index of 100 Gb and it is serving the end users and it can't
>> go offline for a moment. Everyday new lucene indexes(2 GB) are generated
>> separately.
>>
>> I have tried coreadmin
>> https://cwiki.apache.org/confluence/display/solr/Merging+Indexes
>>
>> And it will create a new core or new folder. which means it will copy 100Gb
>> every time to a new folder.
>>
>> Is there a way I can do a segment level merging?
>>
>> Jeba
>

Re: Solr index segment level merge

[Walter Underwood <wu...@wunderwood.org>]

You probably do not NEED to merge your indexes. Have you tried not merging the indexes?

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Dec 29, 2015, at 7:31 PM, jeba earnest <er...@gmail.com> wrote:
> 
> I have a scenario that I need to merge the solr indexes online. I have a
> primary solr index of 100 Gb and it is serving the end users and it can't
> go offline for a moment. Everyday new lucene indexes(2 GB) are generated
> separately.
> 
> I have tried coreadmin
> https://cwiki.apache.org/confluence/display/solr/Merging+Indexes
> 
> And it will create a new core or new folder. which means it will copy 100Gb
> every time to a new folder.
> 
> Is there a way I can do a segment level merging?
> 
> Jeba