autoSoftCommit vs commitWithin

[Matthew Lapointe <ml...@alpha-sense.com.INVALID>]

Hello,

I have been reading the documentation on UpdateHandlers and Near Real Time
Searching, and I'm wondering if there is any practical difference between
these two configurations:

   - autoSoftCommit enabled, with maxTime=10000 and commitWithin not
   specified
   - autoSoftCommit disabled, commitWithin=10000 set on each update request

My assumption is that the functionality would be the same, but just wanted
to make sure I'm not missing something. Currently we are using
autoSoftCommit, but using commitWithin seems potentially better if it would
enable using a larger commitWithin for some lower priority updates.

Thanks,
Matthew

Re: autoSoftCommit vs commitWithin

[Shawn Heisey <ap...@elyograg.org>]

On 3/1/2022 9:55 AM, Matthew Lapointe wrote:
> I have been reading the documentation on UpdateHandlers and Near Real Time
> Searching, and I'm wondering if there is any practical difference between
> these two configurations:
> 
>     - autoSoftCommit enabled, with maxTime=10000 and commitWithin not
>     specified
>     - autoSoftCommit disabled, commitWithin=10000 set on each update request
> 
> My assumption is that the functionality would be the same, but just wanted
> to make sure I'm not missing something. Currently we are using
> autoSoftCommit, but using commitWithin seems potentially better if it would
> enable using a larger commitWithin for some lower priority updates.

The effective outcome should be the same for those scenarios if a 
significant majority of index updates have the commitWithin specified. 
Assuming you're on a pretty new release.  The default for commitWithin 
is to do a soft commit in recent Solr versions.  I don't know if that 
has always been its behavior or if that default was changed from hard to 
soft at some point.

For this scenario, I personally would set up autoSoftCommit as well as 
using commitWithin.  Set the automatic setting to have a larger value 
than the commitWithin, so it acts as a safety valve that catches any 
updates that were missing the commitWithin.  Maybe 60000?

If your commits end up taking longer than 10 seconds, then you'll want 
to consider making the interval longer, so that you're not running 
multiple commits at the same time.

Thanks,
Shawn