You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Lasitha Wattaladeniya <wa...@gmail.com> on 2016/12/18 09:35:41 UTC
Soft commit and reading data just after the commit
Hello devs,
I'm here with another problem i'm facing. I'm trying to do a commit (soft
commit) through solrj and just after the commit, retrieve the data from
solr (requirement is to get updated data list).
I'm using soft commit instead of the hard commit, is previously I got an
error "Exceeded limit of maxWarmingSearchers=2, try again later" because of
too many commit requests. Now I have removed the explicit commit and has
let the solr to do the commit using autoSoftCommit *(1 mili second)* and
autoCommit *(30 seconds)* configurations. Now I'm not getting any errors
when i'm committing frequently.
The problem i'm facing now is, I'm not getting the updated data when I
fetch from solr just after the soft commit. So in this case what are the
best practices to use ? to wait 1 mili second before retrieving data after
soft commit ? I don't feel like waiting from client side is a good option.
Please give me some help from your expert knowledge
Best regards,
Lasitha Wattaladeniya
Software Engineer
Mobile : +6593896893
Blog : techreadme.blogspot.com
Re: Soft commit and reading data just after the commit
Posted by Walter Underwood <wu...@wunderwood.org>.
You probably need a database instead of a search engine.
What requirement makes you want to do this with a search engine?
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Dec 19, 2016, at 6:34 PM, Lasitha Wattaladeniya <wa...@gmail.com> wrote:
>
> Hi Hendrik,
>
> Thanks for your input. Previously I was using the hard commit
> (SolrClient.commit()) but then I got some error when there are concurrent
> real time index requests from my app. The error was "Exceeded limit of
> maxWarmingSearchers=2, try again later", then i changed the code to use
> only solrserver.add(docs) method and configured autoSoftCommit timeout and
> autoCommit timeout in solrConfig.
>
> I think, i'll get the same error when I use the method you described
> (SolrClient.commit(String
> collection, boolean waitFlush, boolean waitSearcher, boolean softCommit)),
> Anyway i'll have a look at that method.
>
> Best regards,
> Lasitha
>
> Lasitha Wattaladeniya
> Software Engineer
>
> Mobile : +6593896893 <+65%209389%206893>
> Blog : techreadme.blogspot.com
>
> On Tue, Dec 20, 2016 at 3:31 AM, Hendrik Haddorp <he...@gmx.net>
> wrote:
>
>> Hi,
>>
>> the SolrJ API has this method: SolrClient.commit(String collection,
>> boolean waitFlush, boolean waitSearcher, boolean softCommit).
>> My assumption so far was that when you set waitSearcher to true that the
>> method call only returns once a search would find the new data, which
>> sounds what you want. I used this already and it seemed to work just fine.
>>
>> regards,
>> Hendrik
>>
>>
>> On 19.12.2016 04:09, Lasitha Wattaladeniya wrote:
>>
>>> Hi all,
>>>
>>> Thanks for your replies,
>>>
>>> @dorian : the requirement is, we are showing a list of entries on a page.
>>> For each user there's a read / unread flag. The data for listing is
>>> fetched from solr. And you can see the entry was previously read or not.
>>> So
>>> when a user views an entry by clicking. We are updating the database flag
>>> to READ and use real time indexing to update solr entry. So when the user
>>> close the full view of the entry and go back to entry listing page, the
>>> data fetched from solr should be updated to READ. That's the use case we
>>> are trying to fix.
>>>
>>> @eric : thanks for the lengthy reply. So let's say I increase the
>>> autosoftcommit time out to may be 100 ms. In that case do I have to wait
>>> much that time from client side before calling search ?. What's the
>>> correct way of achieving this?
>>>
>>> Regards,
>>> Lasitha
>>>
>>> On 18 Dec 2016 23:52, "Erick Erickson" <er...@gmail.com> wrote:
>>>
>>> 1 ms autocommit is far too frequent. And it's not
>>>> helping you anyway.
>>>>
>>>> There is some lag between when a commit happens
>>>> and when the docs are really available. The sequence is:
>>>> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
>>>> 2> a new searcher is opened and autowarming starts
>>>> 3> until the new searcher is opened, queries continue to be served by
>>>> the old searcher
>>>> 4> the new searcher is fully opened
>>>> 5> _new_ requests are served by the new searcher.
>>>> 6> the last request is finished by the old searcher and it's closed.
>>>>
>>>> So what's probably happening is that you send docs and then send a
>>>> query and Solr is still in step <3>. You can look at your admin UI
>>>> pluginst/stats page or your log to see how long it takes for a
>>>> searcher to open and adjust your expectations accordingly.
>>>>
>>>> If you want to fetch only the document (not try to get it by a
>>>> search), Real Time Get is designed to insure that you always get the
>>>> most recent copy whether it's searchable or not.
>>>>
>>>> All that said, Solr wasn't designed for autocommits that are that
>>>> frequent. That's why the documentation talks about _Near_ Real Time.
>>>> You may need to adjust your expectations.
>>>>
>>>> Best,
>>>> Erick
>>>>
>>>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <do...@gmail.com>
>>>> wrote:
>>>>
>>>>> There's a very high probability that you're using the wrong tool for the
>>>>> job if you need 1ms softCommit time. Especially when you always need it
>>>>>
>>>> (ex
>>>>
>>>>> there are apps where you need commit-after-insert very rarely).
>>>>>
>>>>> So explain what you're using it for ?
>>>>>
>>>>> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
>>>>>
>>>> wattale@gmail.com>
>>>>
>>>>> wrote:
>>>>>
>>>>> Hi Furkan,
>>>>>>
>>>>>> Thanks for the links. I had read the first one but not the second one.
>>>>>> I
>>>>>> did read it after you sent. So in my current solrconfig.xml settings
>>>>>>
>>>>> below
>>>>
>>>>> are the configurations,
>>>>>>
>>>>>> <autoSoftCommit>
>>>>>> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
>>>>>> </autoSoftCommit>
>>>>>>
>>>>>>
>>>>>> <autoCommit>
>>>>>> <maxTime>15000</maxTime>
>>>>>> <openSearcher>false</openSearcher>
>>>>>> </autoCommit>
>>>>>>
>>>>>> The problem i'm facing is, just after adding the documents to solr
>>>>>> using
>>>>>> solrj, when I retrieve data from solr I am not getting the updated
>>>>>>
>>>>> results.
>>>>
>>>>> This happens time to time. Most of the time I get the correct data but
>>>>>>
>>>>> in
>>>>
>>>>> some occasions I get wrong results. so as you suggest, what the best
>>>>>> practice to use here ? , should I wait 1 mili second before calling for
>>>>>> updated results ?
>>>>>>
>>>>>> Regards,
>>>>>> Lasitha
>>>>>>
>>>>>> Lasitha Wattaladeniya
>>>>>> Software Engineer
>>>>>>
>>>>>> Mobile : +6593896893
>>>>>> Blog : techreadme.blogspot.com
>>>>>>
>>>>>> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankamaci@gmail.com
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> Hi Lasitha,
>>>>>>>
>>>>>>> First of all, did you check these:
>>>>>>>
>>>>>>> https://cwiki.apache.org/confluence/display/solr/Near+
>>>>>>>
>>>>>> Real+Time+Searching
>>>>>>
>>>>>>> https://lucidworks.com/blog/2013/08/23/understanding-
>>>>>>> transaction-logs-softcommit-and-commit-in-sorlcloud/
>>>>>>>
>>>>>>> after that, if you cannot adjust your configuration you can give more
>>>>>>> information and we can find a solution.
>>>>>>>
>>>>>>> Kind Regards,
>>>>>>> Furkan KAMACI
>>>>>>>
>>>>>>> On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
>>>>>>>
>>>>>> wattale@gmail.com>
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi furkan,
>>>>>>>>
>>>>>>>> Thanks for your reply, it is generally a query heavy system. We are
>>>>>>>>
>>>>>>> using
>>>>>>
>>>>>>> realtime indexing for editing the available data
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Lasitha
>>>>>>>>
>>>>>>>> Lasitha Wattaladeniya
>>>>>>>> Software Engineer
>>>>>>>>
>>>>>>>> Mobile : +6593896893 <+65%209389%206893>
>>>>>>>> Blog : techreadme.blogspot.com
>>>>>>>>
>>>>>>>> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <
>>>>>>>>
>>>>>>> furkankamaci@gmail.com>
>>>>
>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Lasitha,
>>>>>>>>>
>>>>>>>>> What is your indexing / querying requirements. Do you have an index
>>>>>>>>> heavy/light - query heavy/light system?
>>>>>>>>>
>>>>>>>>> Kind Regards,
>>>>>>>>> Furkan KAMACI
>>>>>>>>>
>>>>>>>>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>>>>>>>>> wattale@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hello devs,
>>>>>>>>>>
>>>>>>>>>> I'm here with another problem i'm facing. I'm trying to do a
>>>>>>>>>>
>>>>>>>>> commit
>>>>
>>>>> (soft
>>>>>>>>>
>>>>>>>>>> commit) through solrj and just after the commit, retrieve the data
>>>>>>>>>>
>>>>>>>>> from
>>>>>>
>>>>>>> solr (requirement is to get updated data list).
>>>>>>>>>>
>>>>>>>>>> I'm using soft commit instead of the hard commit, is previously I
>>>>>>>>>>
>>>>>>>>> got
>>>>
>>>>> an
>>>>>>>>>
>>>>>>>>>> error "Exceeded limit of maxWarmingSearchers=2, try again later"
>>>>>>>>>>
>>>>>>>>> because of
>>>>>>>>>
>>>>>>>>>> too many commit requests. Now I have removed the explicit commit
>>>>>>>>>>
>>>>>>>>> and
>>>>
>>>>> has
>>>>>>>>>
>>>>>>>>>> let the solr to do the commit using autoSoftCommit *(1 mili
>>>>>>>>>>
>>>>>>>>> second)*
>>>>
>>>>> and
>>>>>>>>>
>>>>>>>>>> autoCommit *(30 seconds)* configurations. Now I'm not getting any
>>>>>>>>>>
>>>>>>>>> errors
>>>>>>>>>
>>>>>>>>>> when i'm committing frequently.
>>>>>>>>>>
>>>>>>>>>> The problem i'm facing now is, I'm not getting the updated data
>>>>>>>>>>
>>>>>>>>> when
>>>>
>>>>> I
>>>>>>
>>>>>>> fetch from solr just after the soft commit. So in this case what
>>>>>>>>>>
>>>>>>>>> are
>>>>
>>>>> the
>>>>>>>>>
>>>>>>>>>> best practices to use ? to wait 1 mili second before retrieving
>>>>>>>>>>
>>>>>>>>> data
>>>>
>>>>> after
>>>>>>>>>
>>>>>>>>>> soft commit ? I don't feel like waiting from client side is a good
>>>>>>>>>>
>>>>>>>>> option.
>>>>>>>>>
>>>>>>>>>> Please give me some help from your expert knowledge
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Lasitha Wattaladeniya
>>>>>>>>>> Software Engineer
>>>>>>>>>>
>>>>>>>>>> Mobile : +6593896893
>>>>>>>>>> Blog : techreadme.blogspot.com
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>
Re: Soft commit and reading data just after the commit
Posted by Lasitha Wattaladeniya <wa...@gmail.com>.
Hi Hendrik,
Thanks for your input. Previously I was using the hard commit
(SolrClient.commit()) but then I got some error when there are concurrent
real time index requests from my app. The error was "Exceeded limit of
maxWarmingSearchers=2, try again later", then i changed the code to use
only solrserver.add(docs) method and configured autoSoftCommit timeout and
autoCommit timeout in solrConfig.
I think, i'll get the same error when I use the method you described
(SolrClient.commit(String
collection, boolean waitFlush, boolean waitSearcher, boolean softCommit)),
Anyway i'll have a look at that method.
Best regards,
Lasitha
Lasitha Wattaladeniya
Software Engineer
Mobile : +6593896893 <+65%209389%206893>
Blog : techreadme.blogspot.com
On Tue, Dec 20, 2016 at 3:31 AM, Hendrik Haddorp <he...@gmx.net>
wrote:
> Hi,
>
> the SolrJ API has this method: SolrClient.commit(String collection,
> boolean waitFlush, boolean waitSearcher, boolean softCommit).
> My assumption so far was that when you set waitSearcher to true that the
> method call only returns once a search would find the new data, which
> sounds what you want. I used this already and it seemed to work just fine.
>
> regards,
> Hendrik
>
>
> On 19.12.2016 04:09, Lasitha Wattaladeniya wrote:
>
>> Hi all,
>>
>> Thanks for your replies,
>>
>> @dorian : the requirement is, we are showing a list of entries on a page.
>> For each user there's a read / unread flag. The data for listing is
>> fetched from solr. And you can see the entry was previously read or not.
>> So
>> when a user views an entry by clicking. We are updating the database flag
>> to READ and use real time indexing to update solr entry. So when the user
>> close the full view of the entry and go back to entry listing page, the
>> data fetched from solr should be updated to READ. That's the use case we
>> are trying to fix.
>>
>> @eric : thanks for the lengthy reply. So let's say I increase the
>> autosoftcommit time out to may be 100 ms. In that case do I have to wait
>> much that time from client side before calling search ?. What's the
>> correct way of achieving this?
>>
>> Regards,
>> Lasitha
>>
>> On 18 Dec 2016 23:52, "Erick Erickson" <er...@gmail.com> wrote:
>>
>> 1 ms autocommit is far too frequent. And it's not
>>> helping you anyway.
>>>
>>> There is some lag between when a commit happens
>>> and when the docs are really available. The sequence is:
>>> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
>>> 2> a new searcher is opened and autowarming starts
>>> 3> until the new searcher is opened, queries continue to be served by
>>> the old searcher
>>> 4> the new searcher is fully opened
>>> 5> _new_ requests are served by the new searcher.
>>> 6> the last request is finished by the old searcher and it's closed.
>>>
>>> So what's probably happening is that you send docs and then send a
>>> query and Solr is still in step <3>. You can look at your admin UI
>>> pluginst/stats page or your log to see how long it takes for a
>>> searcher to open and adjust your expectations accordingly.
>>>
>>> If you want to fetch only the document (not try to get it by a
>>> search), Real Time Get is designed to insure that you always get the
>>> most recent copy whether it's searchable or not.
>>>
>>> All that said, Solr wasn't designed for autocommits that are that
>>> frequent. That's why the documentation talks about _Near_ Real Time.
>>> You may need to adjust your expectations.
>>>
>>> Best,
>>> Erick
>>>
>>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <do...@gmail.com>
>>> wrote:
>>>
>>>> There's a very high probability that you're using the wrong tool for the
>>>> job if you need 1ms softCommit time. Especially when you always need it
>>>>
>>> (ex
>>>
>>>> there are apps where you need commit-after-insert very rarely).
>>>>
>>>> So explain what you're using it for ?
>>>>
>>>> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
>>>>
>>> wattale@gmail.com>
>>>
>>>> wrote:
>>>>
>>>> Hi Furkan,
>>>>>
>>>>> Thanks for the links. I had read the first one but not the second one.
>>>>> I
>>>>> did read it after you sent. So in my current solrconfig.xml settings
>>>>>
>>>> below
>>>
>>>> are the configurations,
>>>>>
>>>>> <autoSoftCommit>
>>>>> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
>>>>> </autoSoftCommit>
>>>>>
>>>>>
>>>>> <autoCommit>
>>>>> <maxTime>15000</maxTime>
>>>>> <openSearcher>false</openSearcher>
>>>>> </autoCommit>
>>>>>
>>>>> The problem i'm facing is, just after adding the documents to solr
>>>>> using
>>>>> solrj, when I retrieve data from solr I am not getting the updated
>>>>>
>>>> results.
>>>
>>>> This happens time to time. Most of the time I get the correct data but
>>>>>
>>>> in
>>>
>>>> some occasions I get wrong results. so as you suggest, what the best
>>>>> practice to use here ? , should I wait 1 mili second before calling for
>>>>> updated results ?
>>>>>
>>>>> Regards,
>>>>> Lasitha
>>>>>
>>>>> Lasitha Wattaladeniya
>>>>> Software Engineer
>>>>>
>>>>> Mobile : +6593896893
>>>>> Blog : techreadme.blogspot.com
>>>>>
>>>>> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankamaci@gmail.com
>>>>> >
>>>>> wrote:
>>>>>
>>>>> Hi Lasitha,
>>>>>>
>>>>>> First of all, did you check these:
>>>>>>
>>>>>> https://cwiki.apache.org/confluence/display/solr/Near+
>>>>>>
>>>>> Real+Time+Searching
>>>>>
>>>>>> https://lucidworks.com/blog/2013/08/23/understanding-
>>>>>> transaction-logs-softcommit-and-commit-in-sorlcloud/
>>>>>>
>>>>>> after that, if you cannot adjust your configuration you can give more
>>>>>> information and we can find a solution.
>>>>>>
>>>>>> Kind Regards,
>>>>>> Furkan KAMACI
>>>>>>
>>>>>> On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
>>>>>>
>>>>> wattale@gmail.com>
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> Hi furkan,
>>>>>>>
>>>>>>> Thanks for your reply, it is generally a query heavy system. We are
>>>>>>>
>>>>>> using
>>>>>
>>>>>> realtime indexing for editing the available data
>>>>>>>
>>>>>>> Regards,
>>>>>>> Lasitha
>>>>>>>
>>>>>>> Lasitha Wattaladeniya
>>>>>>> Software Engineer
>>>>>>>
>>>>>>> Mobile : +6593896893 <+65%209389%206893>
>>>>>>> Blog : techreadme.blogspot.com
>>>>>>>
>>>>>>> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <
>>>>>>>
>>>>>> furkankamaci@gmail.com>
>>>
>>>> wrote:
>>>>>>>
>>>>>>> Hi Lasitha,
>>>>>>>>
>>>>>>>> What is your indexing / querying requirements. Do you have an index
>>>>>>>> heavy/light - query heavy/light system?
>>>>>>>>
>>>>>>>> Kind Regards,
>>>>>>>> Furkan KAMACI
>>>>>>>>
>>>>>>>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>>>>>>>> wattale@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello devs,
>>>>>>>>>
>>>>>>>>> I'm here with another problem i'm facing. I'm trying to do a
>>>>>>>>>
>>>>>>>> commit
>>>
>>>> (soft
>>>>>>>>
>>>>>>>>> commit) through solrj and just after the commit, retrieve the data
>>>>>>>>>
>>>>>>>> from
>>>>>
>>>>>> solr (requirement is to get updated data list).
>>>>>>>>>
>>>>>>>>> I'm using soft commit instead of the hard commit, is previously I
>>>>>>>>>
>>>>>>>> got
>>>
>>>> an
>>>>>>>>
>>>>>>>>> error "Exceeded limit of maxWarmingSearchers=2, try again later"
>>>>>>>>>
>>>>>>>> because of
>>>>>>>>
>>>>>>>>> too many commit requests. Now I have removed the explicit commit
>>>>>>>>>
>>>>>>>> and
>>>
>>>> has
>>>>>>>>
>>>>>>>>> let the solr to do the commit using autoSoftCommit *(1 mili
>>>>>>>>>
>>>>>>>> second)*
>>>
>>>> and
>>>>>>>>
>>>>>>>>> autoCommit *(30 seconds)* configurations. Now I'm not getting any
>>>>>>>>>
>>>>>>>> errors
>>>>>>>>
>>>>>>>>> when i'm committing frequently.
>>>>>>>>>
>>>>>>>>> The problem i'm facing now is, I'm not getting the updated data
>>>>>>>>>
>>>>>>>> when
>>>
>>>> I
>>>>>
>>>>>> fetch from solr just after the soft commit. So in this case what
>>>>>>>>>
>>>>>>>> are
>>>
>>>> the
>>>>>>>>
>>>>>>>>> best practices to use ? to wait 1 mili second before retrieving
>>>>>>>>>
>>>>>>>> data
>>>
>>>> after
>>>>>>>>
>>>>>>>>> soft commit ? I don't feel like waiting from client side is a good
>>>>>>>>>
>>>>>>>> option.
>>>>>>>>
>>>>>>>>> Please give me some help from your expert knowledge
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Lasitha Wattaladeniya
>>>>>>>>> Software Engineer
>>>>>>>>>
>>>>>>>>> Mobile : +6593896893
>>>>>>>>> Blog : techreadme.blogspot.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>
Re: Soft commit and reading data just after the commit
Posted by Hendrik Haddorp <he...@gmx.net>.
Hi,
the SolrJ API has this method: SolrClient.commit(String collection,
boolean waitFlush, boolean waitSearcher, boolean softCommit).
My assumption so far was that when you set waitSearcher to true that the
method call only returns once a search would find the new data, which
sounds what you want. I used this already and it seemed to work just fine.
regards,
Hendrik
On 19.12.2016 04:09, Lasitha Wattaladeniya wrote:
> Hi all,
>
> Thanks for your replies,
>
> @dorian : the requirement is, we are showing a list of entries on a page.
> For each user there's a read / unread flag. The data for listing is
> fetched from solr. And you can see the entry was previously read or not. So
> when a user views an entry by clicking. We are updating the database flag
> to READ and use real time indexing to update solr entry. So when the user
> close the full view of the entry and go back to entry listing page, the
> data fetched from solr should be updated to READ. That's the use case we
> are trying to fix.
>
> @eric : thanks for the lengthy reply. So let's say I increase the
> autosoftcommit time out to may be 100 ms. In that case do I have to wait
> much that time from client side before calling search ?. What's the
> correct way of achieving this?
>
> Regards,
> Lasitha
>
> On 18 Dec 2016 23:52, "Erick Erickson" <er...@gmail.com> wrote:
>
>> 1 ms autocommit is far too frequent. And it's not
>> helping you anyway.
>>
>> There is some lag between when a commit happens
>> and when the docs are really available. The sequence is:
>> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
>> 2> a new searcher is opened and autowarming starts
>> 3> until the new searcher is opened, queries continue to be served by
>> the old searcher
>> 4> the new searcher is fully opened
>> 5> _new_ requests are served by the new searcher.
>> 6> the last request is finished by the old searcher and it's closed.
>>
>> So what's probably happening is that you send docs and then send a
>> query and Solr is still in step <3>. You can look at your admin UI
>> pluginst/stats page or your log to see how long it takes for a
>> searcher to open and adjust your expectations accordingly.
>>
>> If you want to fetch only the document (not try to get it by a
>> search), Real Time Get is designed to insure that you always get the
>> most recent copy whether it's searchable or not.
>>
>> All that said, Solr wasn't designed for autocommits that are that
>> frequent. That's why the documentation talks about _Near_ Real Time.
>> You may need to adjust your expectations.
>>
>> Best,
>> Erick
>>
>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <do...@gmail.com>
>> wrote:
>>> There's a very high probability that you're using the wrong tool for the
>>> job if you need 1ms softCommit time. Especially when you always need it
>> (ex
>>> there are apps where you need commit-after-insert very rarely).
>>>
>>> So explain what you're using it for ?
>>>
>>> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
>> wattale@gmail.com>
>>> wrote:
>>>
>>>> Hi Furkan,
>>>>
>>>> Thanks for the links. I had read the first one but not the second one. I
>>>> did read it after you sent. So in my current solrconfig.xml settings
>> below
>>>> are the configurations,
>>>>
>>>> <autoSoftCommit>
>>>> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
>>>> </autoSoftCommit>
>>>>
>>>>
>>>> <autoCommit>
>>>> <maxTime>15000</maxTime>
>>>> <openSearcher>false</openSearcher>
>>>> </autoCommit>
>>>>
>>>> The problem i'm facing is, just after adding the documents to solr using
>>>> solrj, when I retrieve data from solr I am not getting the updated
>> results.
>>>> This happens time to time. Most of the time I get the correct data but
>> in
>>>> some occasions I get wrong results. so as you suggest, what the best
>>>> practice to use here ? , should I wait 1 mili second before calling for
>>>> updated results ?
>>>>
>>>> Regards,
>>>> Lasitha
>>>>
>>>> Lasitha Wattaladeniya
>>>> Software Engineer
>>>>
>>>> Mobile : +6593896893
>>>> Blog : techreadme.blogspot.com
>>>>
>>>> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <fu...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Lasitha,
>>>>>
>>>>> First of all, did you check these:
>>>>>
>>>>> https://cwiki.apache.org/confluence/display/solr/Near+
>>>> Real+Time+Searching
>>>>> https://lucidworks.com/blog/2013/08/23/understanding-
>>>>> transaction-logs-softcommit-and-commit-in-sorlcloud/
>>>>>
>>>>> after that, if you cannot adjust your configuration you can give more
>>>>> information and we can find a solution.
>>>>>
>>>>> Kind Regards,
>>>>> Furkan KAMACI
>>>>>
>>>>> On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
>>>> wattale@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi furkan,
>>>>>>
>>>>>> Thanks for your reply, it is generally a query heavy system. We are
>>>> using
>>>>>> realtime indexing for editing the available data
>>>>>>
>>>>>> Regards,
>>>>>> Lasitha
>>>>>>
>>>>>> Lasitha Wattaladeniya
>>>>>> Software Engineer
>>>>>>
>>>>>> Mobile : +6593896893 <+65%209389%206893>
>>>>>> Blog : techreadme.blogspot.com
>>>>>>
>>>>>> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <
>> furkankamaci@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Lasitha,
>>>>>>>
>>>>>>> What is your indexing / querying requirements. Do you have an index
>>>>>>> heavy/light - query heavy/light system?
>>>>>>>
>>>>>>> Kind Regards,
>>>>>>> Furkan KAMACI
>>>>>>>
>>>>>>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>>>>>>> wattale@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello devs,
>>>>>>>>
>>>>>>>> I'm here with another problem i'm facing. I'm trying to do a
>> commit
>>>>>>> (soft
>>>>>>>> commit) through solrj and just after the commit, retrieve the data
>>>> from
>>>>>>>> solr (requirement is to get updated data list).
>>>>>>>>
>>>>>>>> I'm using soft commit instead of the hard commit, is previously I
>> got
>>>>>>> an
>>>>>>>> error "Exceeded limit of maxWarmingSearchers=2, try again later"
>>>>>>> because of
>>>>>>>> too many commit requests. Now I have removed the explicit commit
>> and
>>>>>>> has
>>>>>>>> let the solr to do the commit using autoSoftCommit *(1 mili
>> second)*
>>>>>>> and
>>>>>>>> autoCommit *(30 seconds)* configurations. Now I'm not getting any
>>>>>>> errors
>>>>>>>> when i'm committing frequently.
>>>>>>>>
>>>>>>>> The problem i'm facing now is, I'm not getting the updated data
>> when
>>>> I
>>>>>>>> fetch from solr just after the soft commit. So in this case what
>> are
>>>>>>> the
>>>>>>>> best practices to use ? to wait 1 mili second before retrieving
>> data
>>>>>>> after
>>>>>>>> soft commit ? I don't feel like waiting from client side is a good
>>>>>>> option.
>>>>>>>> Please give me some help from your expert knowledge
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Lasitha Wattaladeniya
>>>>>>>> Software Engineer
>>>>>>>>
>>>>>>>> Mobile : +6593896893
>>>>>>>> Blog : techreadme.blogspot.com
>>>>>>>>
>>>>>>
Re: Soft commit and reading data just after the commit
Posted by Lasitha Wattaladeniya <wa...@gmail.com>.
I didn't look much onto REALTIME GET handler. Thanks for mentioning
that. I'm checking it now
On 19 Dec 2016 10:09, "Lasitha Wattaladeniya" <wa...@gmail.com> wrote:
> Hi all,
>
> Thanks for your replies,
>
> @dorian : the requirement is, we are showing a list of entries on a page.
> For each user there's a read / unread flag. The data for listing is
> fetched from solr. And you can see the entry was previously read or not. So
> when a user views an entry by clicking. We are updating the database flag
> to READ and use real time indexing to update solr entry. So when the user
> close the full view of the entry and go back to entry listing page, the
> data fetched from solr should be updated to READ. That's the use case we
> are trying to fix.
>
> @eric : thanks for the lengthy reply. So let's say I increase the
> autosoftcommit time out to may be 100 ms. In that case do I have to wait
> much that time from client side before calling search ?. What's the
> correct way of achieving this?
>
> Regards,
> Lasitha
>
> On 18 Dec 2016 23:52, "Erick Erickson" <er...@gmail.com> wrote:
>
>> 1 ms autocommit is far too frequent. And it's not
>> helping you anyway.
>>
>> There is some lag between when a commit happens
>> and when the docs are really available. The sequence is:
>> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
>> 2> a new searcher is opened and autowarming starts
>> 3> until the new searcher is opened, queries continue to be served by
>> the old searcher
>> 4> the new searcher is fully opened
>> 5> _new_ requests are served by the new searcher.
>> 6> the last request is finished by the old searcher and it's closed.
>>
>> So what's probably happening is that you send docs and then send a
>> query and Solr is still in step <3>. You can look at your admin UI
>> pluginst/stats page or your log to see how long it takes for a
>> searcher to open and adjust your expectations accordingly.
>>
>> If you want to fetch only the document (not try to get it by a
>> search), Real Time Get is designed to insure that you always get the
>> most recent copy whether it's searchable or not.
>>
>> All that said, Solr wasn't designed for autocommits that are that
>> frequent. That's why the documentation talks about _Near_ Real Time.
>> You may need to adjust your expectations.
>>
>> Best,
>> Erick
>>
>> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <do...@gmail.com>
>> wrote:
>> > There's a very high probability that you're using the wrong tool for the
>> > job if you need 1ms softCommit time. Especially when you always need it
>> (ex
>> > there are apps where you need commit-after-insert very rarely).
>> >
>> > So explain what you're using it for ?
>> >
>> > On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
>> wattale@gmail.com>
>> > wrote:
>> >
>> >> Hi Furkan,
>> >>
>> >> Thanks for the links. I had read the first one but not the second one.
>> I
>> >> did read it after you sent. So in my current solrconfig.xml settings
>> below
>> >> are the configurations,
>> >>
>> >> <autoSoftCommit>
>> >> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
>> >> </autoSoftCommit>
>> >>
>> >>
>> >> <autoCommit>
>> >> <maxTime>15000</maxTime>
>> >> <openSearcher>false</openSearcher>
>> >> </autoCommit>
>> >>
>> >> The problem i'm facing is, just after adding the documents to solr
>> using
>> >> solrj, when I retrieve data from solr I am not getting the updated
>> results.
>> >> This happens time to time. Most of the time I get the correct data but
>> in
>> >> some occasions I get wrong results. so as you suggest, what the best
>> >> practice to use here ? , should I wait 1 mili second before calling for
>> >> updated results ?
>> >>
>> >> Regards,
>> >> Lasitha
>> >>
>> >> Lasitha Wattaladeniya
>> >> Software Engineer
>> >>
>> >> Mobile : +6593896893
>> >> Blog : techreadme.blogspot.com
>> >>
>> >> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankamaci@gmail.com
>> >
>> >> wrote:
>> >>
>> >> > Hi Lasitha,
>> >> >
>> >> > First of all, did you check these:
>> >> >
>> >> > https://cwiki.apache.org/confluence/display/solr/Near+
>> >> Real+Time+Searching
>> >> > https://lucidworks.com/blog/2013/08/23/understanding-
>> >> > transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >> >
>> >> > after that, if you cannot adjust your configuration you can give more
>> >> > information and we can find a solution.
>> >> >
>> >> > Kind Regards,
>> >> > Furkan KAMACI
>> >> >
>> >> > On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
>> >> wattale@gmail.com>
>> >> > wrote:
>> >> >
>> >> >> Hi furkan,
>> >> >>
>> >> >> Thanks for your reply, it is generally a query heavy system. We are
>> >> using
>> >> >> realtime indexing for editing the available data
>> >> >>
>> >> >> Regards,
>> >> >> Lasitha
>> >> >>
>> >> >> Lasitha Wattaladeniya
>> >> >> Software Engineer
>> >> >>
>> >> >> Mobile : +6593896893 <+65%209389%206893>
>> >> >> Blog : techreadme.blogspot.com
>> >> >>
>> >> >> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <
>> furkankamaci@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>> Hi Lasitha,
>> >> >>>
>> >> >>> What is your indexing / querying requirements. Do you have an index
>> >> >>> heavy/light - query heavy/light system?
>> >> >>>
>> >> >>> Kind Regards,
>> >> >>> Furkan KAMACI
>> >> >>>
>> >> >>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>> >> >>> wattale@gmail.com>
>> >> >>> wrote:
>> >> >>>
>> >> >>> > Hello devs,
>> >> >>> >
>> >> >>> > I'm here with another problem i'm facing. I'm trying to do a
>> commit
>> >> >>> (soft
>> >> >>> > commit) through solrj and just after the commit, retrieve the
>> data
>> >> from
>> >> >>> > solr (requirement is to get updated data list).
>> >> >>> >
>> >> >>> > I'm using soft commit instead of the hard commit, is previously
>> I got
>> >> >>> an
>> >> >>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
>> >> >>> because of
>> >> >>> > too many commit requests. Now I have removed the explicit commit
>> and
>> >> >>> has
>> >> >>> > let the solr to do the commit using autoSoftCommit *(1 mili
>> second)*
>> >> >>> and
>> >> >>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any
>> >> >>> errors
>> >> >>> > when i'm committing frequently.
>> >> >>> >
>> >> >>> > The problem i'm facing now is, I'm not getting the updated data
>> when
>> >> I
>> >> >>> > fetch from solr just after the soft commit. So in this case what
>> are
>> >> >>> the
>> >> >>> > best practices to use ? to wait 1 mili second before retrieving
>> data
>> >> >>> after
>> >> >>> > soft commit ? I don't feel like waiting from client side is a
>> good
>> >> >>> option.
>> >> >>> > Please give me some help from your expert knowledge
>> >> >>> >
>> >> >>> > Best regards,
>> >> >>> > Lasitha Wattaladeniya
>> >> >>> > Software Engineer
>> >> >>> >
>> >> >>> > Mobile : +6593896893
>> >> >>> > Blog : techreadme.blogspot.com
>> >> >>> >
>> >> >>>
>> >> >>
>> >> >>
>> >> >
>> >>
>>
>
Re: Soft commit and reading data just after the commit
Posted by Ere Maijala <er...@helsinki.fi>.
Hi,
so, the app already has a database connection because it updates the
READ flag when the user clicks an entry, right? If you only need the
flag for display purposes, it sounds like it would make sense to also
fetch it directly from the database when displaying the listing. Of
course if you also need to search for READ/UNREAD you need to index the
change, but perhaps you could get away with it taking longer.
--Ere
20.12.2016, 4.12, Lasitha Wattaladeniya kirjoitti:
> Hi Shawn,
>
> Thanks for your well detailed explanation. Now I understand, I won't be
> able to achieve the 100ms softcommit timeout with my hardware setup.
> However let's say someone has a requirement as below (quoted from my
> previous mail)
>
> *Requirement *is, we are showing a list of entries on a page. For each
> user there's a read / unread flag. The data for listing is fetched from
> solr. And you can see the entry was previously read or not. So when a user
> views an entry by clicking. We are updating the database flag to READ and
> use real time indexing to update solr index. So when the user close the
> full view of the entry and go back to entry listing page, the data fetched
> from solr should be updated to READ.
>
> Can't we achieve a requirement as described above using solr ? (without
> manipulating the previously fetched results list from solr, because at some
> point we'll have to go back to search results from solr and at that time it
> should be updated).
>
> Regards,
> Lasitha
>
> Lasitha Wattaladeniya
> Software Engineer
>
> Mobile : +6593896893
> Blog : techreadme.blogspot.com
>
> On Mon, Dec 19, 2016 at 6:37 PM, Shawn Heisey <ap...@elyograg.org> wrote:
>
>> On 12/18/2016 7:09 PM, Lasitha Wattaladeniya wrote:
>>> @eric : thanks for the lengthy reply. So let's say I increase the
>>> autosoftcommit time out to may be 100 ms. In that case do I have to
>>> wait much that time from client side before calling search ?. What's
>>> the correct way of achieving this?
>>
>> Some of the following is covered by the links you've already received.
>> Some of it may be new information.
>>
>> Before you can see a change you've just made, you will need to wait for
>> the commit to be fired (in this case, the autoSoftCommit time) plus
>> however long it actually takes to complete the commit and open a new
>> searcher. Opening the searcher is the expensive part.
>>
>> What I typically recommend that people do is have the autoSoftCommit
>> time as long as they can stand, with 60-300 seconds as a "typical"
>> value. That's a setting of 60000 to 300000. What you are trying to
>> achieve is much faster, and much more difficult.
>>
>> 100 milliseconds will typically be far too small a value unless your
>> index is extremely small or your hardware is incredibly fast and has a
>> lot of memory. With a value of 100, you'll want each of those soft
>> commits (which do open a new searcher) to take FAR less than 100
>> milliseconds to complete. This kind of speed can be difficult to
>> achieve, especially if the index is large.
>>
>> To have any hope of fast commit times, you will need to set
>> autowarmCount on all Solr caches to zero. If you are indexing
>> frequently enough, you might even want to completely disable Solr's
>> internal caches, because they may be providing no benefit.
>>
>> You will want to have enough extra memory that your operating system can
>> cache the vast majority (or even maybe all) of your index.
>>
>> https://wiki.apache.org/solr/SolrPerformanceProblems
>>
>> Some other info that's helpful for understanding why plenty of *spare*
>> memory (not allocated by programs) is necessary for good performance:
>>
>> https://en.wikipedia.org/wiki/Page_cache
>> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>>
>> The reason in a nutshell: Disks are EXTREMELY slow. Memory is very fast.
>>
>> Thanks,
>> Shawn
>>
>>
>
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
Re: Soft commit and reading data just after the commit
Posted by Shawn Heisey <ap...@elyograg.org>.
On 12/19/2016 7:12 PM, Lasitha Wattaladeniya wrote:
> *Requirement *is, we are showing a list of entries on a page. For each
> user there's a read / unread flag. The data for listing is fetched
> from solr. And you can see the entry was previously read or not. So
> when a user views an entry by clicking. We are updating the database
> flag to READ and use real time indexing to update solr index. So when
> the user close the full view of the entry and go back to entry listing
> page, the data fetched from solr should be updated to READ. Can't we
> achieve a requirement as described above using solr ? (without
> manipulating the previously fetched results list from solr, because at
> some point we'll have to go back to search results from solr and at
> that time it should be updated).
If you want the user to see that state reflected in their own results,
it's probably far easier to have the application track what the user has
opened in the last minute or so, and if those specific IDs are in the
results, mark them as read regardless of what Solr returns.
Shortly after the application sends the update, the results coming back
from Solr should be current, which is why your application won't need to
hold onto that information for very long.
Thanks,
Shawn
Re: Soft commit and reading data just after the commit
Posted by Lasitha Wattaladeniya <wa...@gmail.com>.
Hi Shawn,
Thanks for your well detailed explanation. Now I understand, I won't be
able to achieve the 100ms softcommit timeout with my hardware setup.
However let's say someone has a requirement as below (quoted from my
previous mail)
*Requirement *is, we are showing a list of entries on a page. For each
user there's a read / unread flag. The data for listing is fetched from
solr. And you can see the entry was previously read or not. So when a user
views an entry by clicking. We are updating the database flag to READ and
use real time indexing to update solr index. So when the user close the
full view of the entry and go back to entry listing page, the data fetched
from solr should be updated to READ.
Can't we achieve a requirement as described above using solr ? (without
manipulating the previously fetched results list from solr, because at some
point we'll have to go back to search results from solr and at that time it
should be updated).
Regards,
Lasitha
Lasitha Wattaladeniya
Software Engineer
Mobile : +6593896893
Blog : techreadme.blogspot.com
On Mon, Dec 19, 2016 at 6:37 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 12/18/2016 7:09 PM, Lasitha Wattaladeniya wrote:
> > @eric : thanks for the lengthy reply. So let's say I increase the
> > autosoftcommit time out to may be 100 ms. In that case do I have to
> > wait much that time from client side before calling search ?. What's
> > the correct way of achieving this?
>
> Some of the following is covered by the links you've already received.
> Some of it may be new information.
>
> Before you can see a change you've just made, you will need to wait for
> the commit to be fired (in this case, the autoSoftCommit time) plus
> however long it actually takes to complete the commit and open a new
> searcher. Opening the searcher is the expensive part.
>
> What I typically recommend that people do is have the autoSoftCommit
> time as long as they can stand, with 60-300 seconds as a "typical"
> value. That's a setting of 60000 to 300000. What you are trying to
> achieve is much faster, and much more difficult.
>
> 100 milliseconds will typically be far too small a value unless your
> index is extremely small or your hardware is incredibly fast and has a
> lot of memory. With a value of 100, you'll want each of those soft
> commits (which do open a new searcher) to take FAR less than 100
> milliseconds to complete. This kind of speed can be difficult to
> achieve, especially if the index is large.
>
> To have any hope of fast commit times, you will need to set
> autowarmCount on all Solr caches to zero. If you are indexing
> frequently enough, you might even want to completely disable Solr's
> internal caches, because they may be providing no benefit.
>
> You will want to have enough extra memory that your operating system can
> cache the vast majority (or even maybe all) of your index.
>
> https://wiki.apache.org/solr/SolrPerformanceProblems
>
> Some other info that's helpful for understanding why plenty of *spare*
> memory (not allocated by programs) is necessary for good performance:
>
> https://en.wikipedia.org/wiki/Page_cache
> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>
> The reason in a nutshell: Disks are EXTREMELY slow. Memory is very fast.
>
> Thanks,
> Shawn
>
>
Re: Soft commit and reading data just after the commit
Posted by Shawn Heisey <ap...@elyograg.org>.
On 12/18/2016 7:09 PM, Lasitha Wattaladeniya wrote:
> @eric : thanks for the lengthy reply. So let's say I increase the
> autosoftcommit time out to may be 100 ms. In that case do I have to
> wait much that time from client side before calling search ?. What's
> the correct way of achieving this?
Some of the following is covered by the links you've already received.
Some of it may be new information.
Before you can see a change you've just made, you will need to wait for
the commit to be fired (in this case, the autoSoftCommit time) plus
however long it actually takes to complete the commit and open a new
searcher. Opening the searcher is the expensive part.
What I typically recommend that people do is have the autoSoftCommit
time as long as they can stand, with 60-300 seconds as a "typical"
value. That's a setting of 60000 to 300000. What you are trying to
achieve is much faster, and much more difficult.
100 milliseconds will typically be far too small a value unless your
index is extremely small or your hardware is incredibly fast and has a
lot of memory. With a value of 100, you'll want each of those soft
commits (which do open a new searcher) to take FAR less than 100
milliseconds to complete. This kind of speed can be difficult to
achieve, especially if the index is large.
To have any hope of fast commit times, you will need to set
autowarmCount on all Solr caches to zero. If you are indexing
frequently enough, you might even want to completely disable Solr's
internal caches, because they may be providing no benefit.
You will want to have enough extra memory that your operating system can
cache the vast majority (or even maybe all) of your index.
https://wiki.apache.org/solr/SolrPerformanceProblems
Some other info that's helpful for understanding why plenty of *spare*
memory (not allocated by programs) is necessary for good performance:
https://en.wikipedia.org/wiki/Page_cache
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
The reason in a nutshell: Disks are EXTREMELY slow. Memory is very fast.
Thanks,
Shawn
Re: Soft commit and reading data just after the commit
Posted by Lasitha Wattaladeniya <wa...@gmail.com>.
Hi all,
Thanks for your replies,
@dorian : the requirement is, we are showing a list of entries on a page.
For each user there's a read / unread flag. The data for listing is
fetched from solr. And you can see the entry was previously read or not. So
when a user views an entry by clicking. We are updating the database flag
to READ and use real time indexing to update solr entry. So when the user
close the full view of the entry and go back to entry listing page, the
data fetched from solr should be updated to READ. That's the use case we
are trying to fix.
@eric : thanks for the lengthy reply. So let's say I increase the
autosoftcommit time out to may be 100 ms. In that case do I have to wait
much that time from client side before calling search ?. What's the
correct way of achieving this?
Regards,
Lasitha
On 18 Dec 2016 23:52, "Erick Erickson" <er...@gmail.com> wrote:
> 1 ms autocommit is far too frequent. And it's not
> helping you anyway.
>
> There is some lag between when a commit happens
> and when the docs are really available. The sequence is:
> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
> 2> a new searcher is opened and autowarming starts
> 3> until the new searcher is opened, queries continue to be served by
> the old searcher
> 4> the new searcher is fully opened
> 5> _new_ requests are served by the new searcher.
> 6> the last request is finished by the old searcher and it's closed.
>
> So what's probably happening is that you send docs and then send a
> query and Solr is still in step <3>. You can look at your admin UI
> pluginst/stats page or your log to see how long it takes for a
> searcher to open and adjust your expectations accordingly.
>
> If you want to fetch only the document (not try to get it by a
> search), Real Time Get is designed to insure that you always get the
> most recent copy whether it's searchable or not.
>
> All that said, Solr wasn't designed for autocommits that are that
> frequent. That's why the documentation talks about _Near_ Real Time.
> You may need to adjust your expectations.
>
> Best,
> Erick
>
> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <do...@gmail.com>
> wrote:
> > There's a very high probability that you're using the wrong tool for the
> > job if you need 1ms softCommit time. Especially when you always need it
> (ex
> > there are apps where you need commit-after-insert very rarely).
> >
> > So explain what you're using it for ?
> >
> > On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
> wattale@gmail.com>
> > wrote:
> >
> >> Hi Furkan,
> >>
> >> Thanks for the links. I had read the first one but not the second one. I
> >> did read it after you sent. So in my current solrconfig.xml settings
> below
> >> are the configurations,
> >>
> >> <autoSoftCommit>
> >> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
> >> </autoSoftCommit>
> >>
> >>
> >> <autoCommit>
> >> <maxTime>15000</maxTime>
> >> <openSearcher>false</openSearcher>
> >> </autoCommit>
> >>
> >> The problem i'm facing is, just after adding the documents to solr using
> >> solrj, when I retrieve data from solr I am not getting the updated
> results.
> >> This happens time to time. Most of the time I get the correct data but
> in
> >> some occasions I get wrong results. so as you suggest, what the best
> >> practice to use here ? , should I wait 1 mili second before calling for
> >> updated results ?
> >>
> >> Regards,
> >> Lasitha
> >>
> >> Lasitha Wattaladeniya
> >> Software Engineer
> >>
> >> Mobile : +6593896893
> >> Blog : techreadme.blogspot.com
> >>
> >> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <fu...@gmail.com>
> >> wrote:
> >>
> >> > Hi Lasitha,
> >> >
> >> > First of all, did you check these:
> >> >
> >> > https://cwiki.apache.org/confluence/display/solr/Near+
> >> Real+Time+Searching
> >> > https://lucidworks.com/blog/2013/08/23/understanding-
> >> > transaction-logs-softcommit-and-commit-in-sorlcloud/
> >> >
> >> > after that, if you cannot adjust your configuration you can give more
> >> > information and we can find a solution.
> >> >
> >> > Kind Regards,
> >> > Furkan KAMACI
> >> >
> >> > On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
> >> wattale@gmail.com>
> >> > wrote:
> >> >
> >> >> Hi furkan,
> >> >>
> >> >> Thanks for your reply, it is generally a query heavy system. We are
> >> using
> >> >> realtime indexing for editing the available data
> >> >>
> >> >> Regards,
> >> >> Lasitha
> >> >>
> >> >> Lasitha Wattaladeniya
> >> >> Software Engineer
> >> >>
> >> >> Mobile : +6593896893 <+65%209389%206893>
> >> >> Blog : techreadme.blogspot.com
> >> >>
> >> >> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <
> furkankamaci@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Hi Lasitha,
> >> >>>
> >> >>> What is your indexing / querying requirements. Do you have an index
> >> >>> heavy/light - query heavy/light system?
> >> >>>
> >> >>> Kind Regards,
> >> >>> Furkan KAMACI
> >> >>>
> >> >>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
> >> >>> wattale@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>> > Hello devs,
> >> >>> >
> >> >>> > I'm here with another problem i'm facing. I'm trying to do a
> commit
> >> >>> (soft
> >> >>> > commit) through solrj and just after the commit, retrieve the data
> >> from
> >> >>> > solr (requirement is to get updated data list).
> >> >>> >
> >> >>> > I'm using soft commit instead of the hard commit, is previously I
> got
> >> >>> an
> >> >>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
> >> >>> because of
> >> >>> > too many commit requests. Now I have removed the explicit commit
> and
> >> >>> has
> >> >>> > let the solr to do the commit using autoSoftCommit *(1 mili
> second)*
> >> >>> and
> >> >>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any
> >> >>> errors
> >> >>> > when i'm committing frequently.
> >> >>> >
> >> >>> > The problem i'm facing now is, I'm not getting the updated data
> when
> >> I
> >> >>> > fetch from solr just after the soft commit. So in this case what
> are
> >> >>> the
> >> >>> > best practices to use ? to wait 1 mili second before retrieving
> data
> >> >>> after
> >> >>> > soft commit ? I don't feel like waiting from client side is a good
> >> >>> option.
> >> >>> > Please give me some help from your expert knowledge
> >> >>> >
> >> >>> > Best regards,
> >> >>> > Lasitha Wattaladeniya
> >> >>> > Software Engineer
> >> >>> >
> >> >>> > Mobile : +6593896893
> >> >>> > Blog : techreadme.blogspot.com
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
>
Re: Soft commit and reading data just after the commit
Posted by Erick Erickson <er...@gmail.com>.
1 ms autocommit is far too frequent. And it's not
helping you anyway.
There is some lag between when a commit happens
and when the docs are really available. The sequence is:
1> commit (soft or hard-with-opensearcher=true doesn't matter).
2> a new searcher is opened and autowarming starts
3> until the new searcher is opened, queries continue to be served by
the old searcher
4> the new searcher is fully opened
5> _new_ requests are served by the new searcher.
6> the last request is finished by the old searcher and it's closed.
So what's probably happening is that you send docs and then send a
query and Solr is still in step <3>. You can look at your admin UI
pluginst/stats page or your log to see how long it takes for a
searcher to open and adjust your expectations accordingly.
If you want to fetch only the document (not try to get it by a
search), Real Time Get is designed to insure that you always get the
most recent copy whether it's searchable or not.
All that said, Solr wasn't designed for autocommits that are that
frequent. That's why the documentation talks about _Near_ Real Time.
You may need to adjust your expectations.
Best,
Erick
On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <do...@gmail.com> wrote:
> There's a very high probability that you're using the wrong tool for the
> job if you need 1ms softCommit time. Especially when you always need it (ex
> there are apps where you need commit-after-insert very rarely).
>
> So explain what you're using it for ?
>
> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <wa...@gmail.com>
> wrote:
>
>> Hi Furkan,
>>
>> Thanks for the links. I had read the first one but not the second one. I
>> did read it after you sent. So in my current solrconfig.xml settings below
>> are the configurations,
>>
>> <autoSoftCommit>
>> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
>> </autoSoftCommit>
>>
>>
>> <autoCommit>
>> <maxTime>15000</maxTime>
>> <openSearcher>false</openSearcher>
>> </autoCommit>
>>
>> The problem i'm facing is, just after adding the documents to solr using
>> solrj, when I retrieve data from solr I am not getting the updated results.
>> This happens time to time. Most of the time I get the correct data but in
>> some occasions I get wrong results. so as you suggest, what the best
>> practice to use here ? , should I wait 1 mili second before calling for
>> updated results ?
>>
>> Regards,
>> Lasitha
>>
>> Lasitha Wattaladeniya
>> Software Engineer
>>
>> Mobile : +6593896893
>> Blog : techreadme.blogspot.com
>>
>> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <fu...@gmail.com>
>> wrote:
>>
>> > Hi Lasitha,
>> >
>> > First of all, did you check these:
>> >
>> > https://cwiki.apache.org/confluence/display/solr/Near+
>> Real+Time+Searching
>> > https://lucidworks.com/blog/2013/08/23/understanding-
>> > transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >
>> > after that, if you cannot adjust your configuration you can give more
>> > information and we can find a solution.
>> >
>> > Kind Regards,
>> > Furkan KAMACI
>> >
>> > On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
>> wattale@gmail.com>
>> > wrote:
>> >
>> >> Hi furkan,
>> >>
>> >> Thanks for your reply, it is generally a query heavy system. We are
>> using
>> >> realtime indexing for editing the available data
>> >>
>> >> Regards,
>> >> Lasitha
>> >>
>> >> Lasitha Wattaladeniya
>> >> Software Engineer
>> >>
>> >> Mobile : +6593896893 <+65%209389%206893>
>> >> Blog : techreadme.blogspot.com
>> >>
>> >> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <fu...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi Lasitha,
>> >>>
>> >>> What is your indexing / querying requirements. Do you have an index
>> >>> heavy/light - query heavy/light system?
>> >>>
>> >>> Kind Regards,
>> >>> Furkan KAMACI
>> >>>
>> >>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>> >>> wattale@gmail.com>
>> >>> wrote:
>> >>>
>> >>> > Hello devs,
>> >>> >
>> >>> > I'm here with another problem i'm facing. I'm trying to do a commit
>> >>> (soft
>> >>> > commit) through solrj and just after the commit, retrieve the data
>> from
>> >>> > solr (requirement is to get updated data list).
>> >>> >
>> >>> > I'm using soft commit instead of the hard commit, is previously I got
>> >>> an
>> >>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
>> >>> because of
>> >>> > too many commit requests. Now I have removed the explicit commit and
>> >>> has
>> >>> > let the solr to do the commit using autoSoftCommit *(1 mili second)*
>> >>> and
>> >>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any
>> >>> errors
>> >>> > when i'm committing frequently.
>> >>> >
>> >>> > The problem i'm facing now is, I'm not getting the updated data when
>> I
>> >>> > fetch from solr just after the soft commit. So in this case what are
>> >>> the
>> >>> > best practices to use ? to wait 1 mili second before retrieving data
>> >>> after
>> >>> > soft commit ? I don't feel like waiting from client side is a good
>> >>> option.
>> >>> > Please give me some help from your expert knowledge
>> >>> >
>> >>> > Best regards,
>> >>> > Lasitha Wattaladeniya
>> >>> > Software Engineer
>> >>> >
>> >>> > Mobile : +6593896893
>> >>> > Blog : techreadme.blogspot.com
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>
Re: Soft commit and reading data just after the commit
Posted by Dorian Hoxha <do...@gmail.com>.
There's a very high probability that you're using the wrong tool for the
job if you need 1ms softCommit time. Especially when you always need it (ex
there are apps where you need commit-after-insert very rarely).
So explain what you're using it for ?
On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <wa...@gmail.com>
wrote:
> Hi Furkan,
>
> Thanks for the links. I had read the first one but not the second one. I
> did read it after you sent. So in my current solrconfig.xml settings below
> are the configurations,
>
> <autoSoftCommit>
> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
> </autoSoftCommit>
>
>
> <autoCommit>
> <maxTime>15000</maxTime>
> <openSearcher>false</openSearcher>
> </autoCommit>
>
> The problem i'm facing is, just after adding the documents to solr using
> solrj, when I retrieve data from solr I am not getting the updated results.
> This happens time to time. Most of the time I get the correct data but in
> some occasions I get wrong results. so as you suggest, what the best
> practice to use here ? , should I wait 1 mili second before calling for
> updated results ?
>
> Regards,
> Lasitha
>
> Lasitha Wattaladeniya
> Software Engineer
>
> Mobile : +6593896893
> Blog : techreadme.blogspot.com
>
> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <fu...@gmail.com>
> wrote:
>
> > Hi Lasitha,
> >
> > First of all, did you check these:
> >
> > https://cwiki.apache.org/confluence/display/solr/Near+
> Real+Time+Searching
> > https://lucidworks.com/blog/2013/08/23/understanding-
> > transaction-logs-softcommit-and-commit-in-sorlcloud/
> >
> > after that, if you cannot adjust your configuration you can give more
> > information and we can find a solution.
> >
> > Kind Regards,
> > Furkan KAMACI
> >
> > On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
> wattale@gmail.com>
> > wrote:
> >
> >> Hi furkan,
> >>
> >> Thanks for your reply, it is generally a query heavy system. We are
> using
> >> realtime indexing for editing the available data
> >>
> >> Regards,
> >> Lasitha
> >>
> >> Lasitha Wattaladeniya
> >> Software Engineer
> >>
> >> Mobile : +6593896893 <+65%209389%206893>
> >> Blog : techreadme.blogspot.com
> >>
> >> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <fu...@gmail.com>
> >> wrote:
> >>
> >>> Hi Lasitha,
> >>>
> >>> What is your indexing / querying requirements. Do you have an index
> >>> heavy/light - query heavy/light system?
> >>>
> >>> Kind Regards,
> >>> Furkan KAMACI
> >>>
> >>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
> >>> wattale@gmail.com>
> >>> wrote:
> >>>
> >>> > Hello devs,
> >>> >
> >>> > I'm here with another problem i'm facing. I'm trying to do a commit
> >>> (soft
> >>> > commit) through solrj and just after the commit, retrieve the data
> from
> >>> > solr (requirement is to get updated data list).
> >>> >
> >>> > I'm using soft commit instead of the hard commit, is previously I got
> >>> an
> >>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
> >>> because of
> >>> > too many commit requests. Now I have removed the explicit commit and
> >>> has
> >>> > let the solr to do the commit using autoSoftCommit *(1 mili second)*
> >>> and
> >>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any
> >>> errors
> >>> > when i'm committing frequently.
> >>> >
> >>> > The problem i'm facing now is, I'm not getting the updated data when
> I
> >>> > fetch from solr just after the soft commit. So in this case what are
> >>> the
> >>> > best practices to use ? to wait 1 mili second before retrieving data
> >>> after
> >>> > soft commit ? I don't feel like waiting from client side is a good
> >>> option.
> >>> > Please give me some help from your expert knowledge
> >>> >
> >>> > Best regards,
> >>> > Lasitha Wattaladeniya
> >>> > Software Engineer
> >>> >
> >>> > Mobile : +6593896893
> >>> > Blog : techreadme.blogspot.com
> >>> >
> >>>
> >>
> >>
> >
>
Re: Soft commit and reading data just after the commit
Posted by Lasitha Wattaladeniya <wa...@gmail.com>.
Hi Furkan,
Thanks for the links. I had read the first one but not the second one. I
did read it after you sent. So in my current solrconfig.xml settings below
are the configurations,
<autoSoftCommit>
<maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
</autoSoftCommit>
<autoCommit>
<maxTime>15000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
The problem i'm facing is, just after adding the documents to solr using
solrj, when I retrieve data from solr I am not getting the updated results.
This happens time to time. Most of the time I get the correct data but in
some occasions I get wrong results. so as you suggest, what the best
practice to use here ? , should I wait 1 mili second before calling for
updated results ?
Regards,
Lasitha
Lasitha Wattaladeniya
Software Engineer
Mobile : +6593896893
Blog : techreadme.blogspot.com
On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <fu...@gmail.com>
wrote:
> Hi Lasitha,
>
> First of all, did you check these:
>
> https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
> https://lucidworks.com/blog/2013/08/23/understanding-
> transaction-logs-softcommit-and-commit-in-sorlcloud/
>
> after that, if you cannot adjust your configuration you can give more
> information and we can find a solution.
>
> Kind Regards,
> Furkan KAMACI
>
> On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <wa...@gmail.com>
> wrote:
>
>> Hi furkan,
>>
>> Thanks for your reply, it is generally a query heavy system. We are using
>> realtime indexing for editing the available data
>>
>> Regards,
>> Lasitha
>>
>> Lasitha Wattaladeniya
>> Software Engineer
>>
>> Mobile : +6593896893 <+65%209389%206893>
>> Blog : techreadme.blogspot.com
>>
>> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <fu...@gmail.com>
>> wrote:
>>
>>> Hi Lasitha,
>>>
>>> What is your indexing / querying requirements. Do you have an index
>>> heavy/light - query heavy/light system?
>>>
>>> Kind Regards,
>>> Furkan KAMACI
>>>
>>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>>> wattale@gmail.com>
>>> wrote:
>>>
>>> > Hello devs,
>>> >
>>> > I'm here with another problem i'm facing. I'm trying to do a commit
>>> (soft
>>> > commit) through solrj and just after the commit, retrieve the data from
>>> > solr (requirement is to get updated data list).
>>> >
>>> > I'm using soft commit instead of the hard commit, is previously I got
>>> an
>>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
>>> because of
>>> > too many commit requests. Now I have removed the explicit commit and
>>> has
>>> > let the solr to do the commit using autoSoftCommit *(1 mili second)*
>>> and
>>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any
>>> errors
>>> > when i'm committing frequently.
>>> >
>>> > The problem i'm facing now is, I'm not getting the updated data when I
>>> > fetch from solr just after the soft commit. So in this case what are
>>> the
>>> > best practices to use ? to wait 1 mili second before retrieving data
>>> after
>>> > soft commit ? I don't feel like waiting from client side is a good
>>> option.
>>> > Please give me some help from your expert knowledge
>>> >
>>> > Best regards,
>>> > Lasitha Wattaladeniya
>>> > Software Engineer
>>> >
>>> > Mobile : +6593896893
>>> > Blog : techreadme.blogspot.com
>>> >
>>>
>>
>>
>
Re: Soft commit and reading data just after the commit
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Lasitha,
First of all, did you check these:
https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching
https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
after that, if you cannot adjust your configuration you can give more
information and we can find a solution.
Kind Regards,
Furkan KAMACI
On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <wa...@gmail.com>
wrote:
> Hi furkan,
>
> Thanks for your reply, it is generally a query heavy system. We are using
> realtime indexing for editing the available data
>
> Regards,
> Lasitha
>
> Lasitha Wattaladeniya
> Software Engineer
>
> Mobile : +6593896893 <+65%209389%206893>
> Blog : techreadme.blogspot.com
>
> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <fu...@gmail.com>
> wrote:
>
>> Hi Lasitha,
>>
>> What is your indexing / querying requirements. Do you have an index
>> heavy/light - query heavy/light system?
>>
>> Kind Regards,
>> Furkan KAMACI
>>
>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>> wattale@gmail.com>
>> wrote:
>>
>> > Hello devs,
>> >
>> > I'm here with another problem i'm facing. I'm trying to do a commit
>> (soft
>> > commit) through solrj and just after the commit, retrieve the data from
>> > solr (requirement is to get updated data list).
>> >
>> > I'm using soft commit instead of the hard commit, is previously I got an
>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
>> because of
>> > too many commit requests. Now I have removed the explicit commit and has
>> > let the solr to do the commit using autoSoftCommit *(1 mili second)* and
>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any errors
>> > when i'm committing frequently.
>> >
>> > The problem i'm facing now is, I'm not getting the updated data when I
>> > fetch from solr just after the soft commit. So in this case what are the
>> > best practices to use ? to wait 1 mili second before retrieving data
>> after
>> > soft commit ? I don't feel like waiting from client side is a good
>> option.
>> > Please give me some help from your expert knowledge
>> >
>> > Best regards,
>> > Lasitha Wattaladeniya
>> > Software Engineer
>> >
>> > Mobile : +6593896893
>> > Blog : techreadme.blogspot.com
>> >
>>
>
>
Re: Soft commit and reading data just after the commit
Posted by Lasitha Wattaladeniya <wa...@gmail.com>.
Hi furkan,
Thanks for your reply, it is generally a query heavy system. We are using
realtime indexing for editing the available data
Regards,
Lasitha
Lasitha Wattaladeniya
Software Engineer
Mobile : +6593896893
Blog : techreadme.blogspot.com
On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <fu...@gmail.com>
wrote:
> Hi Lasitha,
>
> What is your indexing / querying requirements. Do you have an index
> heavy/light - query heavy/light system?
>
> Kind Regards,
> Furkan KAMACI
>
> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <wattale@gmail.com
> >
> wrote:
>
> > Hello devs,
> >
> > I'm here with another problem i'm facing. I'm trying to do a commit (soft
> > commit) through solrj and just after the commit, retrieve the data from
> > solr (requirement is to get updated data list).
> >
> > I'm using soft commit instead of the hard commit, is previously I got an
> > error "Exceeded limit of maxWarmingSearchers=2, try again later" because
> of
> > too many commit requests. Now I have removed the explicit commit and has
> > let the solr to do the commit using autoSoftCommit *(1 mili second)* and
> > autoCommit *(30 seconds)* configurations. Now I'm not getting any errors
> > when i'm committing frequently.
> >
> > The problem i'm facing now is, I'm not getting the updated data when I
> > fetch from solr just after the soft commit. So in this case what are the
> > best practices to use ? to wait 1 mili second before retrieving data
> after
> > soft commit ? I don't feel like waiting from client side is a good
> option.
> > Please give me some help from your expert knowledge
> >
> > Best regards,
> > Lasitha Wattaladeniya
> > Software Engineer
> >
> > Mobile : +6593896893
> > Blog : techreadme.blogspot.com
> >
>
Re: Soft commit and reading data just after the commit
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Lasitha,
What is your indexing / querying requirements. Do you have an index
heavy/light - query heavy/light system?
Kind Regards,
Furkan KAMACI
On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <wa...@gmail.com>
wrote:
> Hello devs,
>
> I'm here with another problem i'm facing. I'm trying to do a commit (soft
> commit) through solrj and just after the commit, retrieve the data from
> solr (requirement is to get updated data list).
>
> I'm using soft commit instead of the hard commit, is previously I got an
> error "Exceeded limit of maxWarmingSearchers=2, try again later" because of
> too many commit requests. Now I have removed the explicit commit and has
> let the solr to do the commit using autoSoftCommit *(1 mili second)* and
> autoCommit *(30 seconds)* configurations. Now I'm not getting any errors
> when i'm committing frequently.
>
> The problem i'm facing now is, I'm not getting the updated data when I
> fetch from solr just after the soft commit. So in this case what are the
> best practices to use ? to wait 1 mili second before retrieving data after
> soft commit ? I don't feel like waiting from client side is a good option.
> Please give me some help from your expert knowledge
>
> Best regards,
> Lasitha Wattaladeniya
> Software Engineer
>
> Mobile : +6593896893
> Blog : techreadme.blogspot.com
>