You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Rahul Goswami <ra...@gmail.com> on 2019/03/08 02:12:49 UTC
Delay searches till log replay finishes
Hello Solr gurus,
I am using Solr 7.2.1 (non-SolrCloud). I have a situation where Solr got
killed before it could commit updates to the disk resulting in log replay
on startup. During this interval, I observe that a searcher is opened even
before log replay has finished, resulting in some stale results, which in
turn has a cascading effect on other parts of the application. Is there a
setting in Solr which would prevent Solr from serving search requests
before log replay has finished?
Thanks,
Rahul
Re: Delay searches till log replay finishes
Posted by Rahul Goswami <ra...@gmail.com>.
Eric,Shawn,
Apologies for the late update on this thread and thank you for your inputs.
My assumption about the number of segments increasing was out of incomplete
understanding of the TieredMergePolicy, but I get it now. Another concern
was slowing indexing rate due to constant merges. This is from reading the
documentation:
"Choosing the best merge factors is generally a trade-off of indexing speed
vs. searching speed. Having fewer segments in the index generally
accelerates searches, because there are fewer places to look. It also can
also result in fewer physical files on disk. But to keep the number of
segments low, merges will occur more often, which can add load to the
system and slow down updates to the index"
Taking your suggestions, we have reduced hard commit interval
(openSearcher=false) from 10 mins to 1 min to begin with. Also, our servers
are on Windows so that could also be a cause of the service getting killed
before being able to gracefully shutdown. The cascading effect is stale
results while tlogs are being played on startup. I understand that although
not foolproof, reducing the autoCommit interval should help mitigate the
problem and we'll continue to monitor this for now.
Thanks,
Rahul
On Fri, Mar 8, 2019 at 2:14 PM Erick Erickson <er...@gmail.com>
wrote:
> (1) no, and Shawn’s comments are well taken.
>
> (2) bq. is the number of segments would drastically increase
>
> Not true. First of all, TieredMergePolicy will take care of merging “like
> sized” segments for you. You’ll have the same number (or close) no matter
> how short the autocommit interval. Second, new segments are created
> whenever the internal indexing buffer is filled up, default 100M anyway so
> just because you have a long autocommit interval doesn’t say much about the
> number of segments that are created.
>
> This is really not something you should be concerned about, certainly not
> something you should accept other problems because. Solr runs quite well
> with 15 second autocommit and very high indexing rates, why do you think
> your situation is different? Do you have any evidence that would be a
> problem at all?
>
> Best,
> Erick
>
>
> > On Mar 8, 2019, at 11:05 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> >
> > On 3/8/2019 10:44 AM, Rahul Goswami wrote:
> >> 1) Is there currently a configuration setting in Solr that will trigger
> the
> >> first option you mentioned ? Which is to not serve any searches until
> tlogs
> >> are played. If not, since instances shutting down abruptly is not very
> >> uncommon, would a JIRA to implement this configuration be warranted?
> >
> > In what setup is an abrupt shutdown *expected*? If that's really
> common, then your setup is, in my opinion, very broken. It is our intent
> that abrupt death of the Solr process should be quite rare. We do still
> have a problem on Windows where the wait for clean shutdown is only five
> seconds -- nowhere near enough. The Windows script still needs a lot of
> work, but most of us are not adept at Windows scripting.
> >
> > There is an issue for the timeout interval in bin\solr.cmd on Windows:
> >
> > https://issues.apache.org/jira/browse/SOLR-9698
> >
> >> 2) We have a setup with moderate indexing rate and moderate search rate.
> >> Currently the auto commit interval is 10 mins. What should be a
> recommended
> >> hard commit interval for such a setup? Our concern with going too low on
> >> that autoCommit interval (with openSearcher=false) is the number of
> >> segments that would drastically increase, eventually causing
> merges,slower
> >> searches etc.
> >
> > Solr has shipped with a 15 second autoCommit, where openSearcher is set
> to false, for a while now. This is a setting that works quite well. As
> long as you're not opening a new searcher, commits are quite fast. I
> personally would use 60 seconds, but 15 seconds does work well. It is
> usually autoSoftCommit where you need to be concerned about short
> intervals, because a soft commit opens a searcher.
> >
> > Thanks,
> > Shawn
>
>
Re: Delay searches till log replay finishes
Posted by Erick Erickson <er...@gmail.com>.
(1) no, and Shawn’s comments are well taken.
(2) bq. is the number of segments would drastically increase
Not true. First of all, TieredMergePolicy will take care of merging “like sized” segments for you. You’ll have the same number (or close) no matter how short the autocommit interval. Second, new segments are created whenever the internal indexing buffer is filled up, default 100M anyway so just because you have a long autocommit interval doesn’t say much about the number of segments that are created.
This is really not something you should be concerned about, certainly not something you should accept other problems because. Solr runs quite well with 15 second autocommit and very high indexing rates, why do you think your situation is different? Do you have any evidence that would be a problem at all?
Best,
Erick
> On Mar 8, 2019, at 11:05 AM, Shawn Heisey <ap...@elyograg.org> wrote:
>
> On 3/8/2019 10:44 AM, Rahul Goswami wrote:
>> 1) Is there currently a configuration setting in Solr that will trigger the
>> first option you mentioned ? Which is to not serve any searches until tlogs
>> are played. If not, since instances shutting down abruptly is not very
>> uncommon, would a JIRA to implement this configuration be warranted?
>
> In what setup is an abrupt shutdown *expected*? If that's really common, then your setup is, in my opinion, very broken. It is our intent that abrupt death of the Solr process should be quite rare. We do still have a problem on Windows where the wait for clean shutdown is only five seconds -- nowhere near enough. The Windows script still needs a lot of work, but most of us are not adept at Windows scripting.
>
> There is an issue for the timeout interval in bin\solr.cmd on Windows:
>
> https://issues.apache.org/jira/browse/SOLR-9698
>
>> 2) We have a setup with moderate indexing rate and moderate search rate.
>> Currently the auto commit interval is 10 mins. What should be a recommended
>> hard commit interval for such a setup? Our concern with going too low on
>> that autoCommit interval (with openSearcher=false) is the number of
>> segments that would drastically increase, eventually causing merges,slower
>> searches etc.
>
> Solr has shipped with a 15 second autoCommit, where openSearcher is set to false, for a while now. This is a setting that works quite well. As long as you're not opening a new searcher, commits are quite fast. I personally would use 60 seconds, but 15 seconds does work well. It is usually autoSoftCommit where you need to be concerned about short intervals, because a soft commit opens a searcher.
>
> Thanks,
> Shawn
Re: Delay searches till log replay finishes
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/8/2019 10:44 AM, Rahul Goswami wrote:
> 1) Is there currently a configuration setting in Solr that will trigger the
> first option you mentioned ? Which is to not serve any searches until tlogs
> are played. If not, since instances shutting down abruptly is not very
> uncommon, would a JIRA to implement this configuration be warranted?
In what setup is an abrupt shutdown *expected*? If that's really
common, then your setup is, in my opinion, very broken. It is our
intent that abrupt death of the Solr process should be quite rare. We
do still have a problem on Windows where the wait for clean shutdown is
only five seconds -- nowhere near enough. The Windows script still
needs a lot of work, but most of us are not adept at Windows scripting.
There is an issue for the timeout interval in bin\solr.cmd on Windows:
https://issues.apache.org/jira/browse/SOLR-9698
> 2) We have a setup with moderate indexing rate and moderate search rate.
> Currently the auto commit interval is 10 mins. What should be a recommended
> hard commit interval for such a setup? Our concern with going too low on
> that autoCommit interval (with openSearcher=false) is the number of
> segments that would drastically increase, eventually causing merges,slower
> searches etc.
Solr has shipped with a 15 second autoCommit, where openSearcher is set
to false, for a while now. This is a setting that works quite well. As
long as you're not opening a new searcher, commits are quite fast. I
personally would use 60 seconds, but 15 seconds does work well. It is
usually autoSoftCommit where you need to be concerned about short
intervals, because a soft commit opens a searcher.
Thanks,
Shawn
Re: Delay searches till log replay finishes
Posted by Rahul Goswami <ra...@gmail.com>.
Eric,
Thanks for the detailed response...I have two follow up questions :
1) Is there currently a configuration setting in Solr that will trigger the
first option you mentioned ? Which is to not serve any searches until tlogs
are played. If not, since instances shutting down abruptly is not very
uncommon, would a JIRA to implement this configuration be warranted?
2) We have a setup with moderate indexing rate and moderate search rate.
Currently the auto commit interval is 10 mins. What should be a recommended
hard commit interval for such a setup? Our concern with going too low on
that autoCommit interval (with openSearcher=false) is the number of
segments that would drastically increase, eventually causing merges,slower
searches etc.
Thanks,
Rahul
On Fri, Mar 8, 2019 at 12:08 PM Erick Erickson <er...@gmail.com>
wrote:
> Yes, you’ll get stale values. There’s no way I know of to change that,
> it’s a fundamental result of Lucene’s design.
>
> There’s a “segment_n” file that contains pointers to the current valid
> segments. When a commit happens, segments are closed and the very last
> operation is to update that file.
>
> In the abnormal termination case, that file has not been updated with docs
> that are in the tlog. So when Solr opens a searcher, it has no record at
> all of any new segments created since the last hard commit. So there are
> only two choices:
>
> 1> refuse to seve any searches at all
> 2> allow searches on the last snapshot of the index while the tlog replays
>
> The latter is the choice we’ve made and I agree with it. While
> theoretically you could refuse to open a searcher while the tlog was
> replaying, I’d rather get some results than none at all.
>
> Especially when this only happens when Solr is abnormally terminated.
>
> You can mitigate the time frame here by setting your hard commit interval
> to, say, 15 seconds, which should be the upper bound of getting stale docs
> when the tlog is replayed.
>
> It’s also good practice to have the autocommit interval relatively short
> for a variety of reasons, not the least of which is that it’ll grow
> infinitely until a hard commit happens.
>
> Best,
> Erick
>
> > On Mar 8, 2019, at 8:48 AM, Rahul Goswami <ra...@gmail.com> wrote:
> >
> > What I am observing is that Solr is fully started up even before it has
> > finished playing the tlog. In the logs I see that a searcher is
> registered
> > first and the "Log replay finished" appears later. During that time if I
> > search, I do get stale values. Below are the log lines that I captured :
> >
> > WARN - 2019-03-08 16:33:42.126; [ x:techproducts]
> > org.apache.solr.update.UpdateLog$LogReplayer; Starting log replay
> >
> tlog{file=C:\Work\Solr\solr-7.2.1\Installation\solr-7.2.1\example\techproducts\solr\techproducts\data\tlog\tlog.0000000000000000009
> > refcount=2} active=false starting pos=0 inSortedOrder=false
> > INFO - 2019-03-08 16:33:42.141; [ x:techproducts]
> > org.apache.solr.core.SolrCore; [techproducts] webapp=null path=null
> >
> params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher}
> > hits=3 status=0 QTime=37
> > INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> > org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
> > INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> >
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener;
> > Loading spell index for spellchecker: default
> > INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> >
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener;
> > Loading spell index for spellchecker: wordbreak
> > INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> > org.apache.solr.core.SolrCore; [techproducts] Registered new searcher
> > Searcher@63c5631[techproducts]
> >
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> > Uninverting(_5(7.2.1):C1)))}
> > INFO - 2019-03-08 16:34:07.373; [ x:techproducts]
> > org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> > params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=2
> > INFO - 2019-03-08 16:34:08.818; [ x:techproducts]
> > org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> > params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
> > INFO - 2019-03-08 16:34:14.422; [ x:techproducts]
> > org.apache.solr.update.DirectUpdateHandler2; start
> >
> commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > INFO - 2019-03-08 16:34:16.353; [ x:techproducts]
> > org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> > params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
> > INFO - 2019-03-08 16:34:18.948; [ x:techproducts]
> > org.apache.solr.update.SolrIndexWriter; Calling setCommitData with
> > IW:org.apache.solr.update.SolrIndexWriter@266e1192
> commitCommandVersion:0
> > INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> > org.apache.solr.search.SolrIndexSearcher; Opening
> > [Searcher@5c6044f1[techproducts]
> > main]
> > INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> > org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
> > INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> > org.apache.solr.core.QuerySenderListener; QuerySenderListener sending
> > requests to Searcher@5c6044f1[techproducts]
> >
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> > Uninverting(_6(7.2.1):C1)))}
> > INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> > org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
> > INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> > org.apache.solr.core.SolrCore; [techproducts] Registered new searcher
> > Searcher@5c6044f1[techproducts]
> >
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> > Uninverting(_6(7.2.1):C1)))}
> > INFO - 2019-03-08 16:34:19.056; [ x:techproducts]
> >
> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor;
> > [techproducts] {add=[SP2514N (1627455755076501504)]} 0 36923
> > WARN - 2019-03-08 16:34:19.056; [ x:techproducts]
> > org.apache.solr.update.UpdateLog$LogReplayer; Log replay finished.
> > recoveryInfo=RecoveryInfo{adds=1 deletes=0 deleteByQuery=0 errors=0
> > positionOfStart=0}
> > INFO - 2019-03-08 16:34:23.523; [ x:techproducts]
> > org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> > params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
> >
> > On Thu, Mar 7, 2019 at 11:36 PM Zheng Lin Edwin Yeo <
> edwinyeozl@gmail.com>
> > wrote:
> >
> >> Hi,
> >>
> >> Do you mean that when you startup Solr, it will automatically do the
> search
> >> request even before the Solr is fully started up?
> >>
> >> Regards,
> >> Edwin
> >>
> >>
> >> On Fri, 8 Mar 2019 at 10:13, Rahul Goswami <ra...@gmail.com>
> wrote:
> >>
> >>> Hello Solr gurus,
> >>>
> >>> I am using Solr 7.2.1 (non-SolrCloud). I have a situation where Solr
> got
> >>> killed before it could commit updates to the disk resulting in log
> replay
> >>> on startup. During this interval, I observe that a searcher is opened
> >> even
> >>> before log replay has finished, resulting in some stale results, which
> in
> >>> turn has a cascading effect on other parts of the application. Is
> there a
> >>> setting in Solr which would prevent Solr from serving search requests
> >>> before log replay has finished?
> >>>
> >>> Thanks,
> >>> Rahul
> >>>
> >>
>
>
Re: Delay searches till log replay finishes
Posted by Erick Erickson <er...@gmail.com>.
Yes, you’ll get stale values. There’s no way I know of to change that, it’s a fundamental result of Lucene’s design.
There’s a “segment_n” file that contains pointers to the current valid segments. When a commit happens, segments are closed and the very last operation is to update that file.
In the abnormal termination case, that file has not been updated with docs that are in the tlog. So when Solr opens a searcher, it has no record at all of any new segments created since the last hard commit. So there are only two choices:
1> refuse to seve any searches at all
2> allow searches on the last snapshot of the index while the tlog replays
The latter is the choice we’ve made and I agree with it. While theoretically you could refuse to open a searcher while the tlog was replaying, I’d rather get some results than none at all.
Especially when this only happens when Solr is abnormally terminated.
You can mitigate the time frame here by setting your hard commit interval to, say, 15 seconds, which should be the upper bound of getting stale docs when the tlog is replayed.
It’s also good practice to have the autocommit interval relatively short for a variety of reasons, not the least of which is that it’ll grow infinitely until a hard commit happens.
Best,
Erick
> On Mar 8, 2019, at 8:48 AM, Rahul Goswami <ra...@gmail.com> wrote:
>
> What I am observing is that Solr is fully started up even before it has
> finished playing the tlog. In the logs I see that a searcher is registered
> first and the "Log replay finished" appears later. During that time if I
> search, I do get stale values. Below are the log lines that I captured :
>
> WARN - 2019-03-08 16:33:42.126; [ x:techproducts]
> org.apache.solr.update.UpdateLog$LogReplayer; Starting log replay
> tlog{file=C:\Work\Solr\solr-7.2.1\Installation\solr-7.2.1\example\techproducts\solr\techproducts\data\tlog\tlog.0000000000000000009
> refcount=2} active=false starting pos=0 inSortedOrder=false
> INFO - 2019-03-08 16:33:42.141; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=null path=null
> params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher}
> hits=3 status=0 QTime=37
> INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
> INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener;
> Loading spell index for spellchecker: default
> INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener;
> Loading spell index for spellchecker: wordbreak
> INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] Registered new searcher
> Searcher@63c5631[techproducts]
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> Uninverting(_5(7.2.1):C1)))}
> INFO - 2019-03-08 16:34:07.373; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=2
> INFO - 2019-03-08 16:34:08.818; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
> INFO - 2019-03-08 16:34:14.422; [ x:techproducts]
> org.apache.solr.update.DirectUpdateHandler2; start
> commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> INFO - 2019-03-08 16:34:16.353; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
> INFO - 2019-03-08 16:34:18.948; [ x:techproducts]
> org.apache.solr.update.SolrIndexWriter; Calling setCommitData with
> IW:org.apache.solr.update.SolrIndexWriter@266e1192 commitCommandVersion:0
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.search.SolrIndexSearcher; Opening
> [Searcher@5c6044f1[techproducts]
> main]
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.core.QuerySenderListener; QuerySenderListener sending
> requests to Searcher@5c6044f1[techproducts]
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> Uninverting(_6(7.2.1):C1)))}
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] Registered new searcher
> Searcher@5c6044f1[techproducts]
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> Uninverting(_6(7.2.1):C1)))}
> INFO - 2019-03-08 16:34:19.056; [ x:techproducts]
> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor;
> [techproducts] {add=[SP2514N (1627455755076501504)]} 0 36923
> WARN - 2019-03-08 16:34:19.056; [ x:techproducts]
> org.apache.solr.update.UpdateLog$LogReplayer; Log replay finished.
> recoveryInfo=RecoveryInfo{adds=1 deletes=0 deleteByQuery=0 errors=0
> positionOfStart=0}
> INFO - 2019-03-08 16:34:23.523; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
>
> On Thu, Mar 7, 2019 at 11:36 PM Zheng Lin Edwin Yeo <ed...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Do you mean that when you startup Solr, it will automatically do the search
>> request even before the Solr is fully started up?
>>
>> Regards,
>> Edwin
>>
>>
>> On Fri, 8 Mar 2019 at 10:13, Rahul Goswami <ra...@gmail.com> wrote:
>>
>>> Hello Solr gurus,
>>>
>>> I am using Solr 7.2.1 (non-SolrCloud). I have a situation where Solr got
>>> killed before it could commit updates to the disk resulting in log replay
>>> on startup. During this interval, I observe that a searcher is opened
>> even
>>> before log replay has finished, resulting in some stale results, which in
>>> turn has a cascading effect on other parts of the application. Is there a
>>> setting in Solr which would prevent Solr from serving search requests
>>> before log replay has finished?
>>>
>>> Thanks,
>>> Rahul
>>>
>>
Re: Delay searches till log replay finishes
Posted by Rahul Goswami <ra...@gmail.com>.
What I am observing is that Solr is fully started up even before it has
finished playing the tlog. In the logs I see that a searcher is registered
first and the "Log replay finished" appears later. During that time if I
search, I do get stale values. Below are the log lines that I captured :
WARN - 2019-03-08 16:33:42.126; [ x:techproducts]
org.apache.solr.update.UpdateLog$LogReplayer; Starting log replay
tlog{file=C:\Work\Solr\solr-7.2.1\Installation\solr-7.2.1\example\techproducts\solr\techproducts\data\tlog\tlog.0000000000000000009
refcount=2} active=false starting pos=0 inSortedOrder=false
INFO - 2019-03-08 16:33:42.141; [ x:techproducts]
org.apache.solr.core.SolrCore; [techproducts] webapp=null path=null
params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher}
hits=3 status=0 QTime=37
INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener;
Loading spell index for spellchecker: default
INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener;
Loading spell index for spellchecker: wordbreak
INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
org.apache.solr.core.SolrCore; [techproducts] Registered new searcher
Searcher@63c5631[techproducts]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
Uninverting(_5(7.2.1):C1)))}
INFO - 2019-03-08 16:34:07.373; [ x:techproducts]
org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=2
INFO - 2019-03-08 16:34:08.818; [ x:techproducts]
org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
INFO - 2019-03-08 16:34:14.422; [ x:techproducts]
org.apache.solr.update.DirectUpdateHandler2; start
commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
INFO - 2019-03-08 16:34:16.353; [ x:techproducts]
org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
INFO - 2019-03-08 16:34:18.948; [ x:techproducts]
org.apache.solr.update.SolrIndexWriter; Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@266e1192 commitCommandVersion:0
INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
org.apache.solr.search.SolrIndexSearcher; Opening
[Searcher@5c6044f1[techproducts]
main]
INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
org.apache.solr.core.QuerySenderListener; QuerySenderListener sending
requests to Searcher@5c6044f1[techproducts]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
Uninverting(_6(7.2.1):C1)))}
INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
org.apache.solr.core.SolrCore; [techproducts] Registered new searcher
Searcher@5c6044f1[techproducts]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
Uninverting(_6(7.2.1):C1)))}
INFO - 2019-03-08 16:34:19.056; [ x:techproducts]
org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor;
[techproducts] {add=[SP2514N (1627455755076501504)]} 0 36923
WARN - 2019-03-08 16:34:19.056; [ x:techproducts]
org.apache.solr.update.UpdateLog$LogReplayer; Log replay finished.
recoveryInfo=RecoveryInfo{adds=1 deletes=0 deleteByQuery=0 errors=0
positionOfStart=0}
INFO - 2019-03-08 16:34:23.523; [ x:techproducts]
org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
On Thu, Mar 7, 2019 at 11:36 PM Zheng Lin Edwin Yeo <ed...@gmail.com>
wrote:
> Hi,
>
> Do you mean that when you startup Solr, it will automatically do the search
> request even before the Solr is fully started up?
>
> Regards,
> Edwin
>
>
> On Fri, 8 Mar 2019 at 10:13, Rahul Goswami <ra...@gmail.com> wrote:
>
> > Hello Solr gurus,
> >
> > I am using Solr 7.2.1 (non-SolrCloud). I have a situation where Solr got
> > killed before it could commit updates to the disk resulting in log replay
> > on startup. During this interval, I observe that a searcher is opened
> even
> > before log replay has finished, resulting in some stale results, which in
> > turn has a cascading effect on other parts of the application. Is there a
> > setting in Solr which would prevent Solr from serving search requests
> > before log replay has finished?
> >
> > Thanks,
> > Rahul
> >
>
Re: Delay searches till log replay finishes
Posted by Zheng Lin Edwin Yeo <ed...@gmail.com>.
Hi,
Do you mean that when you startup Solr, it will automatically do the search
request even before the Solr is fully started up?
Regards,
Edwin
On Fri, 8 Mar 2019 at 10:13, Rahul Goswami <ra...@gmail.com> wrote:
> Hello Solr gurus,
>
> I am using Solr 7.2.1 (non-SolrCloud). I have a situation where Solr got
> killed before it could commit updates to the disk resulting in log replay
> on startup. During this interval, I observe that a searcher is opened even
> before log replay has finished, resulting in some stale results, which in
> turn has a cascading effect on other parts of the application. Is there a
> setting in Solr which would prevent Solr from serving search requests
> before log replay has finished?
>
> Thanks,
> Rahul
>