You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Novin Novin <to...@gmail.com> on 2016/01/05 13:14:05 UTC
solr 5.2.0 need to build high query response
Hi guys,
I'm having trouble to figure what would be idle solr config for where:
I'm doing hard commit in every minute for very few number of users
because I have to show those docs in search results quickly when user save
the changes.
It is causing the response in around 2 secs to show even I am getting only
10 records.
Could you able to give some idea where to look at.
Thanks in advance,
Novin
Re: solr 5.2.0 need to build high query response
Posted by Erick Erickson <er...@gmail.com>.
Right, the point of the newSearcher and firstSearcher events
is to smooth out spikes in response time, and it looks like
that's happening now.
>From here, you're in to query tuning. Without knowing your queries,
your schema, how many docs on a server, memory allocation
and all that it's impossible to say much.
Here are some places to start looking:
https://wiki.apache.org/solr/SolrPerformanceProblems
Start with adding &debug=timing to see where Solr thinks it's
spending time, you'll see breakdowns for how much time
is spent searching, highlighting, faceting, whatever. Likely the
time is in search, but you never know.
Best,
Erick
On Wed, Jan 6, 2016 at 2:15 AM, Novin Novin <to...@gmail.com> wrote:
> Thanks Erick, this listener doing quite a good job. But not what I needed.
> Do the solr has any other things that I can look into to make it faster.
> FYI speed goes to 1 sec to 1.2 sec. I actually needed around 500 ms.
>
> On Tue, 5 Jan 2016 at 18:24 Erick Erickson <er...@gmail.com> wrote:
>
>> Yep. Do note what's happening here. You're executing a query
>> that potentially takes 10 seconds to execute (based on your
>> earlier post). But you may be opening a new searcher every
>> 2 seconds. You may start to see "too many on deck searchers"
>> in your log. If you do do _not_ try to "fix" this by upping the
>> maxWarmingSearchers in solrconfig.xml, that's really an
>> anti-pattern.
>>
>> Really, I'd consider relaxing this 2 second limit. I've often found
>> it easier to tell users "it may take up to 30 seconds for newly-added
>> docs to appear in search results" than try to satisfy overly-tight
>> requirements.
>>
>> As a former co-worker often said, "Users are much more comfortable
>> with predictable delays than unpredictable ones". It's surprising how
>> often it's the case.
>>
>> Best,
>> Erick
>>
>> P.S. What's the difference between newSearcher and firstSearcher?
>> newSearcher is fired every time a commit (soft or hard with
>> openSearcher=true)
>> where firstSearcher is fired up only when Solr starts. This is to
>> accommodate
>> the fact that the autowarm counts on things like filterCacher aren't
>> available when Solr starts. In practice, though, many (most?) people
>> put the same query in both.
>>
>> On Tue, Jan 5, 2016 at 9:17 AM, Novin Novin <to...@gmail.com> wrote:
>> > If I'm correct, you are talking about this
>> >
>> > <listener event="newSearcher" class="solr.QuerySenderListener">
>> > <arr name="queries">
>> > <!--
>> >
>> > *And how I can mention my spatial search field here.*
>> >
>> > <lst><str name="q">solr</str><str name="sort">price
>> > asc</str></lst>
>> > <lst><str name="q">rocks</str><str name="sort">weight
>> > asc</str></lst>
>> > -->
>> > </arr>
>> > </listener>
>> > <listener event="firstSearcher" class="solr.QuerySenderListener">
>> > <arr name="queries">
>> > <lst>
>> >
>> > *or may be here too.*
>> >
>> > <str name="q">static firstSearcher warming in
>> > solrconfig.xml</str>
>> > </lst>
>> > </arr>
>> > </listener>
>> >
>> > Thanks,
>> > Novin
>> >
>> > On Tue, 5 Jan 2016 at 16:22 Erick Erickson <er...@gmail.com>
>> wrote:
>> >
>> >> It sounds like you're not doing proper autowarming,
>> >> which you'd need to do either with hard or
>> >> soft commits that open new searchers.
>> >>
>> >> see:
>> >> https://wiki.apache.org/solr/SolrCaching#Cache_Warming_and_Autowarming
>> >>
>> >> In particular, you should have a newSearcher event
>> >> that facets on the fields you expect to need.
>> >>
>> >> Best,
>> >> Erick
>> >>
>> >> On Tue, Jan 5, 2016 at 8:17 AM, Novin Novin <to...@gmail.com>
>> wrote:
>> >> > Thanks David. It is quite good to use for NRT.
>> >> >
>> >> > Apologies, I didn't mention that facet search is really slow.
>> >> >
>> >> > I found the below reason which could be the reason because I am using
>> >> facet
>> >> > spatial search which is getting slow.
>> >> >
>> >> > To know more about solr hard and soft commits, have a look at this
>> blog :
>> >> >
>> >>
>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >> >
>> >> > In this article, "soft commits are that they will make documents
>> visible,
>> >> > but at some cost. In particular the “top level” caches, which include
>> >> what
>> >> > you configure in solrconfig.xml (filterCache, queryResultCache, etc)
>> will
>> >> > be invalidated! Autowarming will be performed on your top level caches
>> >> > (e.g. filterCache, queryResultCache), and any newSearcher queries
>> will be
>> >> > executed. Also, the FieldValueCache is invalidated, so facet queries
>> will
>> >> > have to wait until the cache is refreshed."
>> >> >
>> >> > Do you have any idea what could possible be do about this?
>> >> >
>> >> >
>> >> >
>> >> > On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
>> >> > davidphilipcherian@gmail.com> wrote:
>> >> >
>> >> >> You should use solr softcommit for this use case. So, by setting
>> >> softcommit
>> >> >> to 5 seconds and autoCommit to minute with openSearcher=false should
>> do
>> >> the
>> >> >> work.
>> >> >>
>> >> >> <autoCommit>
>> >> >> <maxTime>60000</maxTime>
>> >> >> <openSearcher>false</openSearcher>
>> >> >> </autoCommit>
>> >> >>
>> >> >> <autoSoftCommit>
>> >> >> <maxTime>2000</maxTime>
>> >> >> </autoSoftCommit>
>> >> >>
>> >> >> Reference link-
>> >> >>
>> >>
>> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
>> >> >>
>> >> >> To know more about solr hard and soft commits, have a look at this
>> blog
>> >> :
>> >> >>
>> >> >>
>> >>
>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >> >>
>> >> >> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <to...@gmail.com>
>> >> wrote:
>> >> >>
>> >> >> > Hi guys,
>> >> >> >
>> >> >> > I'm having trouble to figure what would be idle solr config for
>> where:
>> >> >> >
>> >> >> > I'm doing hard commit in every minute for very few number of
>> users
>> >> >> > because I have to show those docs in search results quickly when
>> user
>> >> >> save
>> >> >> > the changes.
>> >> >> >
>> >> >> > It is causing the response in around 2 secs to show even I am
>> getting
>> >> >> only
>> >> >> > 10 records.
>> >> >> >
>> >> >> > Could you able to give some idea where to look at.
>> >> >> >
>> >> >> >
>> >> >> > Thanks in advance,
>> >> >> > Novin
>> >> >> >
>> >> >>
>> >>
>>
Re: solr 5.2.0 need to build high query response
Posted by Novin Novin <to...@gmail.com>.
Thanks Erick, this listener doing quite a good job. But not what I needed.
Do the solr has any other things that I can look into to make it faster.
FYI speed goes to 1 sec to 1.2 sec. I actually needed around 500 ms.
On Tue, 5 Jan 2016 at 18:24 Erick Erickson <er...@gmail.com> wrote:
> Yep. Do note what's happening here. You're executing a query
> that potentially takes 10 seconds to execute (based on your
> earlier post). But you may be opening a new searcher every
> 2 seconds. You may start to see "too many on deck searchers"
> in your log. If you do do _not_ try to "fix" this by upping the
> maxWarmingSearchers in solrconfig.xml, that's really an
> anti-pattern.
>
> Really, I'd consider relaxing this 2 second limit. I've often found
> it easier to tell users "it may take up to 30 seconds for newly-added
> docs to appear in search results" than try to satisfy overly-tight
> requirements.
>
> As a former co-worker often said, "Users are much more comfortable
> with predictable delays than unpredictable ones". It's surprising how
> often it's the case.
>
> Best,
> Erick
>
> P.S. What's the difference between newSearcher and firstSearcher?
> newSearcher is fired every time a commit (soft or hard with
> openSearcher=true)
> where firstSearcher is fired up only when Solr starts. This is to
> accommodate
> the fact that the autowarm counts on things like filterCacher aren't
> available when Solr starts. In practice, though, many (most?) people
> put the same query in both.
>
> On Tue, Jan 5, 2016 at 9:17 AM, Novin Novin <to...@gmail.com> wrote:
> > If I'm correct, you are talking about this
> >
> > <listener event="newSearcher" class="solr.QuerySenderListener">
> > <arr name="queries">
> > <!--
> >
> > *And how I can mention my spatial search field here.*
> >
> > <lst><str name="q">solr</str><str name="sort">price
> > asc</str></lst>
> > <lst><str name="q">rocks</str><str name="sort">weight
> > asc</str></lst>
> > -->
> > </arr>
> > </listener>
> > <listener event="firstSearcher" class="solr.QuerySenderListener">
> > <arr name="queries">
> > <lst>
> >
> > *or may be here too.*
> >
> > <str name="q">static firstSearcher warming in
> > solrconfig.xml</str>
> > </lst>
> > </arr>
> > </listener>
> >
> > Thanks,
> > Novin
> >
> > On Tue, 5 Jan 2016 at 16:22 Erick Erickson <er...@gmail.com>
> wrote:
> >
> >> It sounds like you're not doing proper autowarming,
> >> which you'd need to do either with hard or
> >> soft commits that open new searchers.
> >>
> >> see:
> >> https://wiki.apache.org/solr/SolrCaching#Cache_Warming_and_Autowarming
> >>
> >> In particular, you should have a newSearcher event
> >> that facets on the fields you expect to need.
> >>
> >> Best,
> >> Erick
> >>
> >> On Tue, Jan 5, 2016 at 8:17 AM, Novin Novin <to...@gmail.com>
> wrote:
> >> > Thanks David. It is quite good to use for NRT.
> >> >
> >> > Apologies, I didn't mention that facet search is really slow.
> >> >
> >> > I found the below reason which could be the reason because I am using
> >> facet
> >> > spatial search which is getting slow.
> >> >
> >> > To know more about solr hard and soft commits, have a look at this
> blog :
> >> >
> >>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
> >> >
> >> > In this article, "soft commits are that they will make documents
> visible,
> >> > but at some cost. In particular the “top level” caches, which include
> >> what
> >> > you configure in solrconfig.xml (filterCache, queryResultCache, etc)
> will
> >> > be invalidated! Autowarming will be performed on your top level caches
> >> > (e.g. filterCache, queryResultCache), and any newSearcher queries
> will be
> >> > executed. Also, the FieldValueCache is invalidated, so facet queries
> will
> >> > have to wait until the cache is refreshed."
> >> >
> >> > Do you have any idea what could possible be do about this?
> >> >
> >> >
> >> >
> >> > On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
> >> > davidphilipcherian@gmail.com> wrote:
> >> >
> >> >> You should use solr softcommit for this use case. So, by setting
> >> softcommit
> >> >> to 5 seconds and autoCommit to minute with openSearcher=false should
> do
> >> the
> >> >> work.
> >> >>
> >> >> <autoCommit>
> >> >> <maxTime>60000</maxTime>
> >> >> <openSearcher>false</openSearcher>
> >> >> </autoCommit>
> >> >>
> >> >> <autoSoftCommit>
> >> >> <maxTime>2000</maxTime>
> >> >> </autoSoftCommit>
> >> >>
> >> >> Reference link-
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
> >> >>
> >> >> To know more about solr hard and soft commits, have a look at this
> blog
> >> :
> >> >>
> >> >>
> >>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
> >> >>
> >> >> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <to...@gmail.com>
> >> wrote:
> >> >>
> >> >> > Hi guys,
> >> >> >
> >> >> > I'm having trouble to figure what would be idle solr config for
> where:
> >> >> >
> >> >> > I'm doing hard commit in every minute for very few number of
> users
> >> >> > because I have to show those docs in search results quickly when
> user
> >> >> save
> >> >> > the changes.
> >> >> >
> >> >> > It is causing the response in around 2 secs to show even I am
> getting
> >> >> only
> >> >> > 10 records.
> >> >> >
> >> >> > Could you able to give some idea where to look at.
> >> >> >
> >> >> >
> >> >> > Thanks in advance,
> >> >> > Novin
> >> >> >
> >> >>
> >>
>
Re: solr 5.2.0 need to build high query response
Posted by Erick Erickson <er...@gmail.com>.
Yep. Do note what's happening here. You're executing a query
that potentially takes 10 seconds to execute (based on your
earlier post). But you may be opening a new searcher every
2 seconds. You may start to see "too many on deck searchers"
in your log. If you do do _not_ try to "fix" this by upping the
maxWarmingSearchers in solrconfig.xml, that's really an
anti-pattern.
Really, I'd consider relaxing this 2 second limit. I've often found
it easier to tell users "it may take up to 30 seconds for newly-added
docs to appear in search results" than try to satisfy overly-tight
requirements.
As a former co-worker often said, "Users are much more comfortable
with predictable delays than unpredictable ones". It's surprising how
often it's the case.
Best,
Erick
P.S. What's the difference between newSearcher and firstSearcher?
newSearcher is fired every time a commit (soft or hard with openSearcher=true)
where firstSearcher is fired up only when Solr starts. This is to accommodate
the fact that the autowarm counts on things like filterCacher aren't
available when Solr starts. In practice, though, many (most?) people
put the same query in both.
On Tue, Jan 5, 2016 at 9:17 AM, Novin Novin <to...@gmail.com> wrote:
> If I'm correct, you are talking about this
>
> <listener event="newSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <!--
>
> *And how I can mention my spatial search field here.*
>
> <lst><str name="q">solr</str><str name="sort">price
> asc</str></lst>
> <lst><str name="q">rocks</str><str name="sort">weight
> asc</str></lst>
> -->
> </arr>
> </listener>
> <listener event="firstSearcher" class="solr.QuerySenderListener">
> <arr name="queries">
> <lst>
>
> *or may be here too.*
>
> <str name="q">static firstSearcher warming in
> solrconfig.xml</str>
> </lst>
> </arr>
> </listener>
>
> Thanks,
> Novin
>
> On Tue, 5 Jan 2016 at 16:22 Erick Erickson <er...@gmail.com> wrote:
>
>> It sounds like you're not doing proper autowarming,
>> which you'd need to do either with hard or
>> soft commits that open new searchers.
>>
>> see:
>> https://wiki.apache.org/solr/SolrCaching#Cache_Warming_and_Autowarming
>>
>> In particular, you should have a newSearcher event
>> that facets on the fields you expect to need.
>>
>> Best,
>> Erick
>>
>> On Tue, Jan 5, 2016 at 8:17 AM, Novin Novin <to...@gmail.com> wrote:
>> > Thanks David. It is quite good to use for NRT.
>> >
>> > Apologies, I didn't mention that facet search is really slow.
>> >
>> > I found the below reason which could be the reason because I am using
>> facet
>> > spatial search which is getting slow.
>> >
>> > To know more about solr hard and soft commits, have a look at this blog :
>> >
>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >
>> > In this article, "soft commits are that they will make documents visible,
>> > but at some cost. In particular the “top level” caches, which include
>> what
>> > you configure in solrconfig.xml (filterCache, queryResultCache, etc) will
>> > be invalidated! Autowarming will be performed on your top level caches
>> > (e.g. filterCache, queryResultCache), and any newSearcher queries will be
>> > executed. Also, the FieldValueCache is invalidated, so facet queries will
>> > have to wait until the cache is refreshed."
>> >
>> > Do you have any idea what could possible be do about this?
>> >
>> >
>> >
>> > On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
>> > davidphilipcherian@gmail.com> wrote:
>> >
>> >> You should use solr softcommit for this use case. So, by setting
>> softcommit
>> >> to 5 seconds and autoCommit to minute with openSearcher=false should do
>> the
>> >> work.
>> >>
>> >> <autoCommit>
>> >> <maxTime>60000</maxTime>
>> >> <openSearcher>false</openSearcher>
>> >> </autoCommit>
>> >>
>> >> <autoSoftCommit>
>> >> <maxTime>2000</maxTime>
>> >> </autoSoftCommit>
>> >>
>> >> Reference link-
>> >>
>> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
>> >>
>> >> To know more about solr hard and soft commits, have a look at this blog
>> :
>> >>
>> >>
>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >>
>> >> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <to...@gmail.com>
>> wrote:
>> >>
>> >> > Hi guys,
>> >> >
>> >> > I'm having trouble to figure what would be idle solr config for where:
>> >> >
>> >> > I'm doing hard commit in every minute for very few number of users
>> >> > because I have to show those docs in search results quickly when user
>> >> save
>> >> > the changes.
>> >> >
>> >> > It is causing the response in around 2 secs to show even I am getting
>> >> only
>> >> > 10 records.
>> >> >
>> >> > Could you able to give some idea where to look at.
>> >> >
>> >> >
>> >> > Thanks in advance,
>> >> > Novin
>> >> >
>> >>
>>
Re: solr 5.2.0 need to build high query response
Posted by Novin Novin <to...@gmail.com>.
If I'm correct, you are talking about this
<listener event="newSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<!--
*And how I can mention my spatial search field here.*
<lst><str name="q">solr</str><str name="sort">price
asc</str></lst>
<lst><str name="q">rocks</str><str name="sort">weight
asc</str></lst>
-->
</arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst>
*or may be here too.*
<str name="q">static firstSearcher warming in
solrconfig.xml</str>
</lst>
</arr>
</listener>
Thanks,
Novin
On Tue, 5 Jan 2016 at 16:22 Erick Erickson <er...@gmail.com> wrote:
> It sounds like you're not doing proper autowarming,
> which you'd need to do either with hard or
> soft commits that open new searchers.
>
> see:
> https://wiki.apache.org/solr/SolrCaching#Cache_Warming_and_Autowarming
>
> In particular, you should have a newSearcher event
> that facets on the fields you expect to need.
>
> Best,
> Erick
>
> On Tue, Jan 5, 2016 at 8:17 AM, Novin Novin <to...@gmail.com> wrote:
> > Thanks David. It is quite good to use for NRT.
> >
> > Apologies, I didn't mention that facet search is really slow.
> >
> > I found the below reason which could be the reason because I am using
> facet
> > spatial search which is getting slow.
> >
> > To know more about solr hard and soft commits, have a look at this blog :
> >
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
> >
> > In this article, "soft commits are that they will make documents visible,
> > but at some cost. In particular the “top level” caches, which include
> what
> > you configure in solrconfig.xml (filterCache, queryResultCache, etc) will
> > be invalidated! Autowarming will be performed on your top level caches
> > (e.g. filterCache, queryResultCache), and any newSearcher queries will be
> > executed. Also, the FieldValueCache is invalidated, so facet queries will
> > have to wait until the cache is refreshed."
> >
> > Do you have any idea what could possible be do about this?
> >
> >
> >
> > On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
> > davidphilipcherian@gmail.com> wrote:
> >
> >> You should use solr softcommit for this use case. So, by setting
> softcommit
> >> to 5 seconds and autoCommit to minute with openSearcher=false should do
> the
> >> work.
> >>
> >> <autoCommit>
> >> <maxTime>60000</maxTime>
> >> <openSearcher>false</openSearcher>
> >> </autoCommit>
> >>
> >> <autoSoftCommit>
> >> <maxTime>2000</maxTime>
> >> </autoSoftCommit>
> >>
> >> Reference link-
> >>
> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
> >>
> >> To know more about solr hard and soft commits, have a look at this blog
> :
> >>
> >>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
> >>
> >> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <to...@gmail.com>
> wrote:
> >>
> >> > Hi guys,
> >> >
> >> > I'm having trouble to figure what would be idle solr config for where:
> >> >
> >> > I'm doing hard commit in every minute for very few number of users
> >> > because I have to show those docs in search results quickly when user
> >> save
> >> > the changes.
> >> >
> >> > It is causing the response in around 2 secs to show even I am getting
> >> only
> >> > 10 records.
> >> >
> >> > Could you able to give some idea where to look at.
> >> >
> >> >
> >> > Thanks in advance,
> >> > Novin
> >> >
> >>
>
Re: solr 5.2.0 need to build high query response
Posted by Erick Erickson <er...@gmail.com>.
It sounds like you're not doing proper autowarming,
which you'd need to do either with hard or
soft commits that open new searchers.
see: https://wiki.apache.org/solr/SolrCaching#Cache_Warming_and_Autowarming
In particular, you should have a newSearcher event
that facets on the fields you expect to need.
Best,
Erick
On Tue, Jan 5, 2016 at 8:17 AM, Novin Novin <to...@gmail.com> wrote:
> Thanks David. It is quite good to use for NRT.
>
> Apologies, I didn't mention that facet search is really slow.
>
> I found the below reason which could be the reason because I am using facet
> spatial search which is getting slow.
>
> To know more about solr hard and soft commits, have a look at this blog :
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> In this article, "soft commits are that they will make documents visible,
> but at some cost. In particular the “top level” caches, which include what
> you configure in solrconfig.xml (filterCache, queryResultCache, etc) will
> be invalidated! Autowarming will be performed on your top level caches
> (e.g. filterCache, queryResultCache), and any newSearcher queries will be
> executed. Also, the FieldValueCache is invalidated, so facet queries will
> have to wait until the cache is refreshed."
>
> Do you have any idea what could possible be do about this?
>
>
>
> On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
> davidphilipcherian@gmail.com> wrote:
>
>> You should use solr softcommit for this use case. So, by setting softcommit
>> to 5 seconds and autoCommit to minute with openSearcher=false should do the
>> work.
>>
>> <autoCommit>
>> <maxTime>60000</maxTime>
>> <openSearcher>false</openSearcher>
>> </autoCommit>
>>
>> <autoSoftCommit>
>> <maxTime>2000</maxTime>
>> </autoSoftCommit>
>>
>> Reference link-
>> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
>>
>> To know more about solr hard and soft commits, have a look at this blog :
>>
>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>>
>> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <to...@gmail.com> wrote:
>>
>> > Hi guys,
>> >
>> > I'm having trouble to figure what would be idle solr config for where:
>> >
>> > I'm doing hard commit in every minute for very few number of users
>> > because I have to show those docs in search results quickly when user
>> save
>> > the changes.
>> >
>> > It is causing the response in around 2 secs to show even I am getting
>> only
>> > 10 records.
>> >
>> > Could you able to give some idea where to look at.
>> >
>> >
>> > Thanks in advance,
>> > Novin
>> >
>>
Re: solr 5.2.0 need to build high query response
Posted by Novin Novin <to...@gmail.com>.
Thanks David. It is quite good to use for NRT.
Apologies, I didn't mention that facet search is really slow.
I found the below reason which could be the reason because I am using facet
spatial search which is getting slow.
To know more about solr hard and soft commits, have a look at this blog :
https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
In this article, "soft commits are that they will make documents visible,
but at some cost. In particular the “top level” caches, which include what
you configure in solrconfig.xml (filterCache, queryResultCache, etc) will
be invalidated! Autowarming will be performed on your top level caches
(e.g. filterCache, queryResultCache), and any newSearcher queries will be
executed. Also, the FieldValueCache is invalidated, so facet queries will
have to wait until the cache is refreshed."
Do you have any idea what could possible be do about this?
On Tue, 5 Jan 2016 at 12:31 davidphilip cherian <
davidphilipcherian@gmail.com> wrote:
> You should use solr softcommit for this use case. So, by setting softcommit
> to 5 seconds and autoCommit to minute with openSearcher=false should do the
> work.
>
> <autoCommit>
> <maxTime>60000</maxTime>
> <openSearcher>false</openSearcher>
> </autoCommit>
>
> <autoSoftCommit>
> <maxTime>2000</maxTime>
> </autoSoftCommit>
>
> Reference link-
> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
>
> To know more about solr hard and soft commits, have a look at this blog :
>
> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <to...@gmail.com> wrote:
>
> > Hi guys,
> >
> > I'm having trouble to figure what would be idle solr config for where:
> >
> > I'm doing hard commit in every minute for very few number of users
> > because I have to show those docs in search results quickly when user
> save
> > the changes.
> >
> > It is causing the response in around 2 secs to show even I am getting
> only
> > 10 records.
> >
> > Could you able to give some idea where to look at.
> >
> >
> > Thanks in advance,
> > Novin
> >
>
Re: solr 5.2.0 need to build high query response
Posted by davidphilip cherian <da...@gmail.com>.
You should use solr softcommit for this use case. So, by setting softcommit
to 5 seconds and autoCommit to minute with openSearcher=false should do the
work.
<autoCommit>
<maxTime>60000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>2000</maxTime>
</autoSoftCommit>
Reference link-
https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
To know more about solr hard and soft commits, have a look at this blog :
https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
On Tue, Jan 5, 2016 at 5:44 PM, Novin Novin <to...@gmail.com> wrote:
> Hi guys,
>
> I'm having trouble to figure what would be idle solr config for where:
>
> I'm doing hard commit in every minute for very few number of users
> because I have to show those docs in search results quickly when user save
> the changes.
>
> It is causing the response in around 2 secs to show even I am getting only
> 10 records.
>
> Could you able to give some idea where to look at.
>
>
> Thanks in advance,
> Novin
>