You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Benjamin Mellish <be...@gmail.com> on 2019/07/18 15:37:34 UTC

Does commitWithin override autoSoftCommit?

I have a solrconfig.xml file as follows:

<updateHandler class="solr.DirectUpdateHandler2">
    <autoSoftCommit>
        <maxTime>2000</maxTime>
    </autoSoftCommit>
    <autoCommit>
        <maxTime>20000</maxTime>
        <openSearcher>false</openSearcher>
    </autoCommit>
    <updateLog>
        <str name="dir">${solr.ulog.dir:}</str>
    </updateLog>
</updateHandler>

But I also submit records with a 'commitWithin' of 10 seconds. It seems
that my documents are not searchable with the soft commit every second, but
rather with the commitWithin or the hard commit. Is the 'commitWithin'
setting taking precedence over the autoSoftCommit timer?

Re: Does commitWithin override autoSoftCommit?

Posted by Benjamin Mellish <be...@gmail.com>.
Thank you for the reply  We are using Solr to maintain a state and kick off
certain processing steps on the file. I am trying to tighten up this
timing.  I understand this is not a great system but it's sort of what I'm
stuck with. I've been reading over the NRT documents and adjusting my
commitWithin time down to about 1 second. I'm getting pretty good results
this way.  Is there another suggestion about how I might get a single field
with a state out of a doc in NRT without adjusting these fields?  I forgot
to mention I'm using 4.10.

On Thu, Jul 18, 2019, 8:50 AM Shawn Heisey <ap...@elyograg.org> wrote:

> On 7/18/2019 9:37 AM, Benjamin Mellish wrote:
> > I have a solrconfig.xml file as follows:
> >
> > <updateHandler class="solr.DirectUpdateHandler2">
> >      <autoSoftCommit>
> >          <maxTime>2000</maxTime>
> >      </autoSoftCommit>
> >      <autoCommit>
> >          <maxTime>20000</maxTime>
> >          <openSearcher>false</openSearcher>
> >      </autoCommit>
> >      <updateLog>
> >          <str name="dir">${solr.ulog.dir:}</str>
> >      </updateLog>
> > </updateHandler>
> >
> > But I also submit records with a 'commitWithin' of 10 seconds. It seems
> > that my documents are not searchable with the soft commit every second,
> but
> > rather with the commitWithin or the hard commit. Is the 'commitWithin'
> > setting taking precedence over the autoSoftCommit timer?
>
> With this config and a 10 second commitWithin, the autoSoftCommit is
> what will fire first -- two seconds after the first update completes.
>
> It could be that the commit takes so long to complete that it LOOKS like
> it's the larger timeframe.  So if a commit that opens a new searcher
> takes 10 seconds or longer to complete, it will start after two seconds
> (the autoSoftCommit value), but it will take at least 12 seconds total
> for users to actually see the change.
>
> Reducing or eliminating cache warming can help commits to complete
> faster.  Sometimes it is not possible to speed up the soft commit.
> Having low values for autoSoftCommit like two seconds is not recommended.
>
> Thanks,
> Shawn
>

Re: Does commitWithin override autoSoftCommit?

Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/18/2019 9:37 AM, Benjamin Mellish wrote:
> I have a solrconfig.xml file as follows:
> 
> <updateHandler class="solr.DirectUpdateHandler2">
>      <autoSoftCommit>
>          <maxTime>2000</maxTime>
>      </autoSoftCommit>
>      <autoCommit>
>          <maxTime>20000</maxTime>
>          <openSearcher>false</openSearcher>
>      </autoCommit>
>      <updateLog>
>          <str name="dir">${solr.ulog.dir:}</str>
>      </updateLog>
> </updateHandler>
> 
> But I also submit records with a 'commitWithin' of 10 seconds. It seems
> that my documents are not searchable with the soft commit every second, but
> rather with the commitWithin or the hard commit. Is the 'commitWithin'
> setting taking precedence over the autoSoftCommit timer?

With this config and a 10 second commitWithin, the autoSoftCommit is 
what will fire first -- two seconds after the first update completes.

It could be that the commit takes so long to complete that it LOOKS like 
it's the larger timeframe.  So if a commit that opens a new searcher 
takes 10 seconds or longer to complete, it will start after two seconds 
(the autoSoftCommit value), but it will take at least 12 seconds total 
for users to actually see the change.

Reducing or eliminating cache warming can help commits to complete 
faster.  Sometimes it is not possible to speed up the soft commit. 
Having low values for autoSoftCommit like two seconds is not recommended.

Thanks,
Shawn