You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Nathan Findley <na...@zenlok.com> on 2012/11/03 09:32:28 UTC
Solr 3.6 -> 4.0
Hi all,
I have one machine running solr 3.6. I would like to move this data to
solr 4.0 and set up a solrcloud.
I feel like I should replicate the existing data. After that, it isn't
clear to me what I need to do.
1) Create a slave (4.0) that replicates from the master (3.6).
2) Somehow turn the slave into a part of a solrcloud?
If there are any online articles about this process or you have any
suggestions, I would appreciate it!
Thanks,
Nate
Re: Solr 3.6 -> 4.0
Posted by Otis Gospodnetic <ot...@gmail.com>.
Depends what you really need. Index aliases are very handy for having a
sliding last N days type search. Solr doesn't have that yet....but it may
be in jira.
Otis
--
Performance Monitoring - http://sematext.com/spm
On Nov 4, 2012 11:34 PM, "Nathan Findley" <na...@zenlok.com> wrote:
> Otis,
>
> I believe I found the thread which contains a link about elasticsearch and
> big data.
>
> http://www.elasticsearch.org/**videos/2012/06/05/big-data-**
> search-and-analytics.html<http://www.elasticsearch.org/videos/2012/06/05/big-data-search-and-analytics.html>
>
> We are dealing with data that is searched using time ranges. Does the
> "time" data flow concept work in SOLR? Does it mean I can be adding shards
> to the existing collection and have it just work? If this concept is more
> readily used in Elasticsearch, I have no problem with using that instead of
> SOLR. We need to be able to maintain searches across shards whatever the
> case may be.
>
> Thanks for your time,
> Nate
>
> On 11/05/2012 03:16 AM, Otis Gospodnetic wrote:
>
>> Correct. There was a good thread on this topic on the ElasticSearch ML.
>> Search for "oversharding" and my name. Same ideas apply to SolrCloud.
>> Neither server offer automatic rebalancing yet, though ES lets you move
>> shards around on demand.
>>
>> Otis
>> --
>> Performance Monitoring - http://sematext.com/spm
>> On Nov 4, 2012 12:20 PM, "Nathan Findley" <na...@zenlok.com> wrote:
>>
>> Otis,
>>>
>>> Thanks for that makes sense. I have one more question: at this point the
>>> only way for future expansion of shard count is by having more than one
>>> shard per machine and then, when things grow, moving each shard to its
>>> own
>>> dedicated machine? That is how I understand it from the wiki.
>>>
>>> So for instance I could have 10 shards where 2 machines have 5 shards
>>> each. Then I could move those shards to their own machines as the index
>>> grows. Is this correct? Does it apply to replicas as well (5 per 2
>>> replica machines)?
>>>
>>> Finally being able to add more shards is something on the feature list?
>>>
>>> Regards,
>>> Nate
>>>
>>>
>>> On 11/03/2012 10:11 PM, Otis Gospodnetic wrote:
>>>
>>> Hi,
>>>>
>>>> Check the archive for a similar Q&A yesterday. Reindexing would be the
>>>> cleanest.
>>>>
>>>> Otis
>>>> --
>>>> Performance Monitoring - http://sematext.com/spm
>>>> On Nov 3, 2012 8:22 AM, "Nathan Findley" <na...@zenlok.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>>> I have one machine running solr 3.6. I would like to move this data to
>>>>> solr 4.0 and set up a solrcloud.
>>>>>
>>>>> I feel like I should replicate the existing data. After that, it isn't
>>>>> clear to me what I need to do.
>>>>>
>>>>> 1) Create a slave (4.0) that replicates from the master (3.6).
>>>>> 2) Somehow turn the slave into a part of a solrcloud?
>>>>>
>>>>> If there are any online articles about this process or you have any
>>>>> suggestions, I would appreciate it!
>>>>>
>>>>> Thanks,
>>>>> Nate
>>>>>
>>>>>
>>>>>
>
> --
> ??????????/?????
> Zenlok????
>
>
Re: Solr 3.6 -> 4.0
Posted by Nathan Findley <na...@zenlok.com>.
Otis,
I believe I found the thread which contains a link about elasticsearch
and big data.
http://www.elasticsearch.org/videos/2012/06/05/big-data-search-and-analytics.html
We are dealing with data that is searched using time ranges. Does the
"time" data flow concept work in SOLR? Does it mean I can be adding
shards to the existing collection and have it just work? If this concept
is more readily used in Elasticsearch, I have no problem with using that
instead of SOLR. We need to be able to maintain searches across shards
whatever the case may be.
Thanks for your time,
Nate
On 11/05/2012 03:16 AM, Otis Gospodnetic wrote:
> Correct. There was a good thread on this topic on the ElasticSearch ML.
> Search for "oversharding" and my name. Same ideas apply to SolrCloud.
> Neither server offer automatic rebalancing yet, though ES lets you move
> shards around on demand.
>
> Otis
> --
> Performance Monitoring - http://sematext.com/spm
> On Nov 4, 2012 12:20 PM, "Nathan Findley" <na...@zenlok.com> wrote:
>
>> Otis,
>>
>> Thanks for that makes sense. I have one more question: at this point the
>> only way for future expansion of shard count is by having more than one
>> shard per machine and then, when things grow, moving each shard to its own
>> dedicated machine? That is how I understand it from the wiki.
>>
>> So for instance I could have 10 shards where 2 machines have 5 shards
>> each. Then I could move those shards to their own machines as the index
>> grows. Is this correct? Does it apply to replicas as well (5 per 2
>> replica machines)?
>>
>> Finally being able to add more shards is something on the feature list?
>>
>> Regards,
>> Nate
>>
>>
>> On 11/03/2012 10:11 PM, Otis Gospodnetic wrote:
>>
>>> Hi,
>>>
>>> Check the archive for a similar Q&A yesterday. Reindexing would be the
>>> cleanest.
>>>
>>> Otis
>>> --
>>> Performance Monitoring - http://sematext.com/spm
>>> On Nov 3, 2012 8:22 AM, "Nathan Findley" <na...@zenlok.com> wrote:
>>>
>>> Hi all,
>>>> I have one machine running solr 3.6. I would like to move this data to
>>>> solr 4.0 and set up a solrcloud.
>>>>
>>>> I feel like I should replicate the existing data. After that, it isn't
>>>> clear to me what I need to do.
>>>>
>>>> 1) Create a slave (4.0) that replicates from the master (3.6).
>>>> 2) Somehow turn the slave into a part of a solrcloud?
>>>>
>>>> If there are any online articles about this process or you have any
>>>> suggestions, I would appreciate it!
>>>>
>>>> Thanks,
>>>> Nate
>>>>
>>>>
--
??????????/?????
Zenlok????
Re: Solr 3.6 -> 4.0
Posted by Otis Gospodnetic <ot...@gmail.com>.
Correct. There was a good thread on this topic on the ElasticSearch ML.
Search for "oversharding" and my name. Same ideas apply to SolrCloud.
Neither server offer automatic rebalancing yet, though ES lets you move
shards around on demand.
Otis
--
Performance Monitoring - http://sematext.com/spm
On Nov 4, 2012 12:20 PM, "Nathan Findley" <na...@zenlok.com> wrote:
> Otis,
>
> Thanks for that makes sense. I have one more question: at this point the
> only way for future expansion of shard count is by having more than one
> shard per machine and then, when things grow, moving each shard to its own
> dedicated machine? That is how I understand it from the wiki.
>
> So for instance I could have 10 shards where 2 machines have 5 shards
> each. Then I could move those shards to their own machines as the index
> grows. Is this correct? Does it apply to replicas as well (5 per 2
> replica machines)?
>
> Finally being able to add more shards is something on the feature list?
>
> Regards,
> Nate
>
>
> On 11/03/2012 10:11 PM, Otis Gospodnetic wrote:
>
>> Hi,
>>
>> Check the archive for a similar Q&A yesterday. Reindexing would be the
>> cleanest.
>>
>> Otis
>> --
>> Performance Monitoring - http://sematext.com/spm
>> On Nov 3, 2012 8:22 AM, "Nathan Findley" <na...@zenlok.com> wrote:
>>
>> Hi all,
>>>
>>> I have one machine running solr 3.6. I would like to move this data to
>>> solr 4.0 and set up a solrcloud.
>>>
>>> I feel like I should replicate the existing data. After that, it isn't
>>> clear to me what I need to do.
>>>
>>> 1) Create a slave (4.0) that replicates from the master (3.6).
>>> 2) Somehow turn the slave into a part of a solrcloud?
>>>
>>> If there are any online articles about this process or you have any
>>> suggestions, I would appreciate it!
>>>
>>> Thanks,
>>> Nate
>>>
>>>
>
Re: Solr 3.6 -> 4.0
Posted by Nathan Findley <na...@zenlok.com>.
Otis,
Thanks for that makes sense. I have one more question: at this point
the only way for future expansion of shard count is by having more than
one shard per machine and then, when things grow, moving each shard to
its own dedicated machine? That is how I understand it from the wiki.
So for instance I could have 10 shards where 2 machines have 5 shards
each. Then I could move those shards to their own machines as the index
grows. Is this correct? Does it apply to replicas as well (5 per 2
replica machines)?
Finally being able to add more shards is something on the feature list?
Regards,
Nate
On 11/03/2012 10:11 PM, Otis Gospodnetic wrote:
> Hi,
>
> Check the archive for a similar Q&A yesterday. Reindexing would be the
> cleanest.
>
> Otis
> --
> Performance Monitoring - http://sematext.com/spm
> On Nov 3, 2012 8:22 AM, "Nathan Findley" <na...@zenlok.com> wrote:
>
>> Hi all,
>>
>> I have one machine running solr 3.6. I would like to move this data to
>> solr 4.0 and set up a solrcloud.
>>
>> I feel like I should replicate the existing data. After that, it isn't
>> clear to me what I need to do.
>>
>> 1) Create a slave (4.0) that replicates from the master (3.6).
>> 2) Somehow turn the slave into a part of a solrcloud?
>>
>> If there are any online articles about this process or you have any
>> suggestions, I would appreciate it!
>>
>> Thanks,
>> Nate
>>
Re: Solr 3.6 -> 4.0
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi,
Check the archive for a similar Q&A yesterday. Reindexing would be the
cleanest.
Otis
--
Performance Monitoring - http://sematext.com/spm
On Nov 3, 2012 8:22 AM, "Nathan Findley" <na...@zenlok.com> wrote:
> Hi all,
>
> I have one machine running solr 3.6. I would like to move this data to
> solr 4.0 and set up a solrcloud.
>
> I feel like I should replicate the existing data. After that, it isn't
> clear to me what I need to do.
>
> 1) Create a slave (4.0) that replicates from the master (3.6).
> 2) Somehow turn the slave into a part of a solrcloud?
>
> If there are any online articles about this process or you have any
> suggestions, I would appreciate it!
>
> Thanks,
> Nate
>