You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Manupriya <ma...@gmail.com> on 2009/01/21 12:01:44 UTC
How to schedule delta-import and auto commit
Hi,
Our Solr server is a standalone server and some web applications send HTTP
query to search and get back the results.
Now I have following two requirements -
1. we want to schedule 'delta-import' at a specified time. So that we dont
have to explicitly send a HTTP request for delta-import.
http://wiki.apache.org/solr/DataImportHandler mentions 'Schedule full
imports and delta imports' but there is no detail. Even
http://www.ibm.com/developerworks/library/j-solr-update/index.html mentions
'scheduler' but again there is no detail.
2. I had asked peviously regarding caching and memory
management(http://www.nabble.com/How-to-open-a-new-searcher-and-close-the-old-one-by-sending-HTTP-request-td21496803.html).
So how do I schedule auto-commit for my Solr server.
As per my understanding, <autoCommit> tag in Solrconfig.xml will call commit
only if there has been an update. Right? So in case, no document has been
added/updated, how can I call auto commit?
Note: My only purpose to call commit without document change is to close
current Searcher and open a new Searcher. This is for better memory
management with caching.
Please let me know if there is any resources that I can refer for these.
Thanks,
Manu
--
View this message in context: http://www.nabble.com/How-to-schedule-delta-import-and-auto-commit-tp21580961p21580961.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to schedule delta-import and auto commit
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
On Wed, Jan 21, 2009 at 4:31 PM, Manupriya <ma...@gmail.com> wrote:
>
> Hi,
>
> Our Solr server is a standalone server and some web applications send HTTP
> query to search and get back the results.
>
> Now I have following two requirements -
>
> 1. we want to schedule 'delta-import' at a specified time. So that we dont
> have to explicitly send a HTTP request for delta-import.
> http://wiki.apache.org/solr/DataImportHandler mentions 'Schedule full
> imports and delta imports' but there is no detail. Even
> http://www.ibm.com/developerworks/library/j-solr-update/index.html mentions
> 'scheduler' but again there is no detail.
There is no feature in Solr to schedule commands at specific intervals.
You may have to do it externally . If you are using linux you can
setup a cron job to invoke a curl at predetermined intervals
>
> 2. I had asked peviously regarding caching and memory
> management(http://www.nabble.com/How-to-open-a-new-searcher-and-close-the-old-one-by-sending-HTTP-request-td21496803.html).
> So how do I schedule auto-commit for my Solr server.
>
> As per my understanding, <autoCommit> tag in Solrconfig.xml will call commit
> only if there has been an update. Right? So in case, no document has been
> added/updated, how can I call auto commit?
> Note: My only purpose to call commit without document change is to close
> current Searcher and open a new Searcher. This is for better memory
> management with caching.
>
> Please let me know if there is any resources that I can refer for these.
>
> Thanks,
> Manu
>
> --
> View this message in context: http://www.nabble.com/How-to-schedule-delta-import-and-auto-commit-tp21580961p21580961.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
--Noble Paul
Re: How to schedule delta-import and auto commit
Posted by Manupriya <ma...@gmail.com>.
Hi Shalin,
I have not faced any memory problems as of now. But I had perviously asked a
question regarding caching and memory
(http://www.nabble.com/How-to-open-a-new-searcher-and-close-the-old-one-by-sending-HTTP-request-td21496803.html)-
----------------------------------------------------------------------------------------------
So can I safely assume that we will not face any memory issue due to caching
even if we do not send commit that frequently? (If we wont send commit, then
new searcher wont be initialized. So I can assume that the current searcher
will correctly manage cache without any memory issues.)
Thanks,
Manu
---------------------------------------------------------------------------------------------
For which I got the following answer -
No,
You can't assume that. You have to set a good autoCommit value for your
solrconfig.xml, so you don't run out of memory for no commiting to Solr
often, depending on your enviroment, memory share, doc size and update
frequency.
----------------------------------------------------------------------------------------------
But my understanding is that <autoCommit> tag works only if there is some
update in index.
So I wanted to understand that if there are no updates, will caching create
some problems with memory?
Thanks,
Manu
Shalin Shekhar Mangar wrote:
>
> On Wed, Jan 21, 2009 at 4:31 PM, Manupriya
> <ma...@gmail.com>wrote:
>
>>
>> 2. I had asked peviously regarding caching and memory
>> management(
>> http://www.nabble.com/How-to-open-a-new-searcher-and-close-the-old-one-by-sending-HTTP-request-td21496803.html
>> ).
>> So how do I schedule auto-commit for my Solr server.
>>
>> As per my understanding, <autoCommit> tag in Solrconfig.xml will call
>> commit
>> only if there has been an update. Right? So in case, no document has been
>> added/updated, how can I call auto commit?
>> Note: My only purpose to call commit without document change is to close
>> current Searcher and open a new Searcher. This is for better memory
>> management with caching.
>
>
> This confuses me. Why do you think Solr is mis-managing the memory? What
> are
> the problems you are encountering?
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>
--
View this message in context: http://www.nabble.com/How-to-schedule-delta-import-and-auto-commit-tp21580961p21582357.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to schedule delta-import and auto commit
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Wed, Jan 21, 2009 at 4:31 PM, Manupriya <ma...@gmail.com>wrote:
>
> 2. I had asked peviously regarding caching and memory
> management(
> http://www.nabble.com/How-to-open-a-new-searcher-and-close-the-old-one-by-sending-HTTP-request-td21496803.html
> ).
> So how do I schedule auto-commit for my Solr server.
>
> As per my understanding, <autoCommit> tag in Solrconfig.xml will call
> commit
> only if there has been an update. Right? So in case, no document has been
> added/updated, how can I call auto commit?
> Note: My only purpose to call commit without document change is to close
> current Searcher and open a new Searcher. This is for better memory
> management with caching.
This confuses me. Why do you think Solr is mis-managing the memory? What are
the problems you are encountering?
--
Regards,
Shalin Shekhar Mangar.