Changing merge policy config on production

[alexpusch <al...@getjaco.com>]

Hi,
Is it safe to change the mergePolicyFactory config on production servers?
Specifically maxMergeAtOnce and segmentsPerTier. How will solr reconcile the
current state of the segments with the new config? In case of setting
segmentsPerTier to a lower number - will subsequent merges be particulary
heavy and might cause performance issues?

Thanks,
Alex.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Changing merge policy config on production

[alexpusch <al...@getjaco.com>]

Thanks Erick, good point on maxMergedSegmentMB, many of my segments really
are max out.
My index isn't 800G, but it's not far from it -  it's about 250G per server.
I have high confidence in Solr and my EC2 i3-2xl instances, so far I got
pretty good results.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Changing merge policy config on production

[Erick Erickson <er...@gmail.com>]

So I'm guessing you have something on the order of an 800G index?

The max segment size is roughly 5G (by default) and assuming all your segments
are close to the max size I get 160 * 5G = 800G, but that may be off.

I think you're barking up the wrong tree if these numbers are close to
correct. This is
a _very_ large index. segments per tier and merge at once aren't going
to help you
here.

You could possibly set
maxMergedSegmentMB
to something other than the default 5000, but I seriously doubt your problem is
the number of segments. You need more shards/hardware if my numbers
are in the right ballpark to have a serious impact on performance.

Best,
Erick

On Sat, Dec 16, 2017 at 12:35 AM, alexpusch <al...@getjaco.com> wrote:
> To be clear - I'm talking about query performance, not indexing performance.
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Changing merge policy config on production

[alexpusch <al...@getjaco.com>]

To be clear - I'm talking about query performance, not indexing performance.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Changing merge policy config on production

[alexpusch <al...@getjaco.com>]

Thanks for the quick answer Erick,

I'm hoping to improve performance by reducing the number of segments.

Currently I have ~160 segments. Am I wrong thinking it might improve
performance?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Changing merge policy config on production

[Erick Erickson <er...@gmail.com>]

The merge rate will be limited by the number of merge threads. You'll merge
more often though so the load will change. That said, I wouldn't be
concerned unless you have a very high indexing rate.

Why do you want to change anyway? Unless you've tried the new settings in a
Dev environment, the biggest risk seems to me to be whether the new
settings are just plain bad in your situation rather than what the short
term effects are.

On Dec 15, 2017 4:20 PM, "alexpusch" <al...@getjaco.com> wrote:

> Hi,
> Is it safe to change the mergePolicyFactory config on production servers?
> Specifically maxMergeAtOnce and segmentsPerTier. How will solr reconcile
> the
> current state of the segments with the new config? In case of setting
> segmentsPerTier to a lower number - will subsequent merges be particulary
> heavy and might cause performance issues?
>
> Thanks,
> Alex.
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>