Re: Flipping data dirs for an (/multiple) SolrCore without affecting search / IndexReaders

[Chris Hostetter <ho...@fucit.org>]

: We have an architecture where we want to flip the solr data.dir (massive
: dataset) while running and serving search requests with minimal downtime.
	...

: 1) What is the fastest / best possible way to get step 1 done ,through a
: pluggable architecture.
: 
: Currently - I have written a request handler as follows, that takes care of
: creating the core. What is the best way to change dataDir (got as input from
: SolrQueryRequest) before creating SolrCore-s.

you shouldn't need any custom plugin code to achieve this ... what you 
describe wounds like exactly what the SWAP command on the CoreAdmin 
handler was designed for -- CREATE your new core (using the new data dir) 
warm it however you want (either via the solrconfig.xml or by explicitly 
hitting it with queries) and once it's warmed up send the SWAP command to 
replace the existing core with the name you want to use.

: 2) When a close() happens on an existing SolrCore - what happens when there is
: a long running IndexReader query on that SolrCore . Is that terminated
: abruptly / would the close wait until the IndexReaders completes the Query.

any existing uses of the Core will continue to finish (i'm not sure of hte 
timeline of your question, ut i'm guessing this was before the recent jira 
issue about the close() method and ref counts where this was better 
explained, correct?)



-Hoss

Re: Flipping data dirs for an (/multiple) SolrCore without affecting search / IndexReaders

[Noble Paul നോബിള്‍ नोब्ळ् <no...@gmail.com>]

Hi Kay,

do you wish to change data dir or just indexdir.
It is possible to switch indexDir w/ minimal overhead.

dataDir can contain a index.properties file which can contain a
property called 'index' it can point to your new index .

but you will have to find a way to populate this index

ReplicationHandler uses this feature.

This may not be a recommended behavior and i'm not sure how you can use this
look at SnapPuller#modifyIndexProps()

--Noble

On Fri, Jan 9, 2009 at 2:12 AM, Kay Kay <ka...@gmail.com> wrote:
> Chris Hostetter wrote:
>>
>> : We have an architecture where we want to flip the solr data.dir (massive
>> : dataset) while running and serving search requests with minimal
>> downtime.
>>        ...
>>
>> : 1) What is the fastest / best possible way to get step 1 done ,through a
>> : pluggable architecture.
>> : : Currently - I have written a request handler as follows, that takes
>> care of
>> : creating the core. What is the best way to change dataDir (got as input
>> from
>> : SolrQueryRequest) before creating SolrCore-s.
>>
>> you shouldn't need any custom plugin code to achieve this ... what you
>> describe wounds like exactly what the SWAP command on the CoreAdmin handler
>> was designed for -- CREATE your new core (using the new data dir) warm it
>> however you want (either via the solrconfig.xml or by explicitly hitting it
>> with queries) and once it's warmed up send the SWAP command to replace the
>> existing core with the name you want to use.
>>
>> : 2) When a close() happens on an existing SolrCore - what happens when
>> there is
>> : a long running IndexReader query on that SolrCore . Is that terminated
>> : abruptly / would the close wait until the IndexReaders completes the
>> Query.
>>
>> any existing uses of the Core will continue to finish (i'm not sure of hte
>> timeline of your question, ut i'm guessing this was before the recent jira
>> issue about the close() method and ref counts where this was better
>> explained, correct?)
>>
>>
>>
>> -Hoss
>>
>>
>>
>
> Thanks Hoss for the explanation regarding changing the data directory. Yes -
> this was before the jira issue discussions for the close() method.
>



-- 
--Noble Paul

Re: Flipping data dirs for an (/multiple) SolrCore without affecting search / IndexReaders

[Kay Kay <ka...@gmail.com>]

Chris Hostetter wrote:
> : We have an architecture where we want to flip the solr data.dir (massive
> : dataset) while running and serving search requests with minimal downtime.
> 	...
>
> : 1) What is the fastest / best possible way to get step 1 done ,through a
> : pluggable architecture.
> : 
> : Currently - I have written a request handler as follows, that takes care of
> : creating the core. What is the best way to change dataDir (got as input from
> : SolrQueryRequest) before creating SolrCore-s.
>
> you shouldn't need any custom plugin code to achieve this ... what you 
> describe wounds like exactly what the SWAP command on the CoreAdmin 
> handler was designed for -- CREATE your new core (using the new data dir) 
> warm it however you want (either via the solrconfig.xml or by explicitly 
> hitting it with queries) and once it's warmed up send the SWAP command to 
> replace the existing core with the name you want to use.
>
> : 2) When a close() happens on an existing SolrCore - what happens when there is
> : a long running IndexReader query on that SolrCore . Is that terminated
> : abruptly / would the close wait until the IndexReaders completes the Query.
>
> any existing uses of the Core will continue to finish (i'm not sure of hte 
> timeline of your question, ut i'm guessing this was before the recent jira 
> issue about the close() method and ref counts where this was better 
> explained, correct?)
>
>
>
> -Hoss
>
>
>   
Thanks Hoss for the explanation regarding changing the data directory. 
Yes - this was before the jira issue discussions for the close() method.