Tuning Merge Settings for Solr Cloud

[Saksham Gupta <sa...@indiamart.com.INVALID>]

Hi team,
We use a solr cloud where more than 4 million search requests are served
and more than 50 million documents are updated daily.
We want to tune the merge configuration of solr to improve searching and
indexing performance.

1. Do we need to perform full indexing in order to bring the changes in
merge setting(segmentsPerTier) to action.
2. How to decide the ideal merge settings for my use case?
*Current merge settings:*
<mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory">
            <int name="maxMergeAtOnce">5</int>
            <int name="segmentsPerTier">3</int>
        </mergePolicyFactory>

Re: Tuning Merge Settings for Solr Cloud

[Shawn Heisey <ap...@elyograg.org>]

On 5/16/23 00:35, Saksham Gupta wrote:
> 1. Do we need to perform full indexing in order to bring the changes in
> merge setting(segmentsPerTier) to action.

Changing the settings will affect any new merges.  Depending on the 
change made, that might result in significant merge activity right away, 
or less merge activity until later.

> 2. How to decide the ideal merge settings for my use case?
> *Current merge settings:*
> <mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory">
>              <int name="maxMergeAtOnce">5</int>
>              <int name="segmentsPerTier">3</int>
>          </mergePolicyFactory>

There is no "one size fits all" setting.  A smaller number of segments 
will generally perform a little bit better than lots of segments, but 
the difference in current Solr versions is not as drastic as it once was.

How to set the mergePolicy is highly dependent on the particulars of 
your setup and usually requires experimentation.  Personally I would 
leave the settings alone unless there is a noticeable problem related to 
segment merges.

Thanks,
Shawn

Re: Tuning Merge Settings for Solr Cloud

[Wei <we...@gmail.com>]

Hi Saksham,

You don't have to perform full indexing for the merge policy setting
change.  For the ideal settings, it depends on what you are trying to
optimize.  Generally more merges and smaller number of segments help to
improved search performance.

Thanks,
Wei

On Thu, May 18, 2023 at 11:39 AM Saksham Gupta
<sa...@indiamart.com.invalid> wrote:

> Hi All,
> Any help regarding this problem?
>
> On Tue, May 16, 2023 at 12:05 PM Saksham Gupta <
> saksham.gupta@indiamart.com>
> wrote:
>
> > Hi team,
> > We use a solr cloud where more than 4 million search requests are served
> > and more than 50 million documents are updated daily.
> > We want to tune the merge configuration of solr to improve searching and
> > indexing performance.
> >
> > 1. Do we need to perform full indexing in order to bring the changes in
> > merge setting(segmentsPerTier) to action.
> > 2. How to decide the ideal merge settings for my use case?
> > *Current merge settings:*
> > <mergePolicyFactory
> class="org.apache.solr.index.TieredMergePolicyFactory">
> >             <int name="maxMergeAtOnce">5</int>
> >             <int name="segmentsPerTier">3</int>
> >         </mergePolicyFactory>
> >
>

Re: Tuning Merge Settings for Solr Cloud

[Saksham Gupta <sa...@indiamart.com.INVALID>]

Hi All,
Any help regarding this problem?

On Tue, May 16, 2023 at 12:05 PM Saksham Gupta <sa...@indiamart.com>
wrote:

> Hi team,
> We use a solr cloud where more than 4 million search requests are served
> and more than 50 million documents are updated daily.
> We want to tune the merge configuration of solr to improve searching and
> indexing performance.
>
> 1. Do we need to perform full indexing in order to bring the changes in
> merge setting(segmentsPerTier) to action.
> 2. How to decide the ideal merge settings for my use case?
> *Current merge settings:*
> <mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory">
>             <int name="maxMergeAtOnce">5</int>
>             <int name="segmentsPerTier">3</int>
>         </mergePolicyFactory>
>