You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by J Mohamed Zahoor <za...@indix.com> on 2013/04/13 17:07:26 UTC
CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
Hi
This question has come up many times in the list with lots of variations (which confuses me a lot).
Iam using Solr 4.1. one collection , 6 shards, 6 machines.
I am using CloudSolrServer inside each mapper to index my documents…. While it is working fine , iam trying to improve the indexing performance.
Question is:
1) is CloudSolrServer multiThreaded?
2) Will using ConcurrentUpdateSolr server increase indexing performance?
./Zahoor
Re: CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
Posted by J Mohamed Zahoor <za...@indix.com>.
It sure increased the performance .
Thanks for the input.
./zahoor
On 14-Apr-2013, at 10:13 PM, J Mohamed Zahoor <za...@indix.com> wrote:
> Thanks..
> Will try multithreading with CloudSolrServer.
>
> ./zahoor
>
> On 13-Apr-2013, at 9:11 PM, Mark Miller <ma...@gmail.com> wrote:
>
>>
>> On Apr 13, 2013, at 11:07 AM, J Mohamed Zahoor <za...@indix.com> wrote:
>>
>>> Hi
>>>
>>> This question has come up many times in the list with lots of variations (which confuses me a lot).
>>>
>>> Iam using Solr 4.1. one collection , 6 shards, 6 machines.
>>> I am using CloudSolrServer inside each mapper to index my documents…. While it is working fine , iam trying to improve the indexing performance.
>>>
>>>
>>> Question is:
>>>
>>> 1) is CloudSolrServer multiThreaded?
>>
>> No. The proper fast way to use it is to start many threads that all add docs to the same CloudSolrServer instance. In other words, currently, you must do the multi threading yourself. CloudSolrServer is "thread safe".
>>
>>>
>>> 2) Will using ConcurrentUpdateSolr server increase indexing performance?
>>
>> Yes, but at the cost of having to specify a server to talk to - if it goes down, so does your indexing. It's also not very great at reporting errors. Finally, using multiple threads and CloudSolrServer, you can approach the performance of ConcurrentUpdateSolr server.
>>
>> - Mark
>>
>>>
>>> ./Zahoor
>>
>
Re: CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
Posted by J Mohamed Zahoor <za...@indix.com>.
Thanks..
Will try multithreading with CloudSolrServer.
./zahoor
On 13-Apr-2013, at 9:11 PM, Mark Miller <ma...@gmail.com> wrote:
>
> On Apr 13, 2013, at 11:07 AM, J Mohamed Zahoor <za...@indix.com> wrote:
>
>> Hi
>>
>> This question has come up many times in the list with lots of variations (which confuses me a lot).
>>
>> Iam using Solr 4.1. one collection , 6 shards, 6 machines.
>> I am using CloudSolrServer inside each mapper to index my documents…. While it is working fine , iam trying to improve the indexing performance.
>>
>>
>> Question is:
>>
>> 1) is CloudSolrServer multiThreaded?
>
> No. The proper fast way to use it is to start many threads that all add docs to the same CloudSolrServer instance. In other words, currently, you must do the multi threading yourself. CloudSolrServer is "thread safe".
>
>>
>> 2) Will using ConcurrentUpdateSolr server increase indexing performance?
>
> Yes, but at the cost of having to specify a server to talk to - if it goes down, so does your indexing. It's also not very great at reporting errors. Finally, using multiple threads and CloudSolrServer, you can approach the performance of ConcurrentUpdateSolr server.
>
> - Mark
>
>>
>> ./Zahoor
>
Re: CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
Posted by Mark Miller <ma...@gmail.com>.
On Apr 13, 2013, at 11:07 AM, J Mohamed Zahoor <za...@indix.com> wrote:
> Hi
>
> This question has come up many times in the list with lots of variations (which confuses me a lot).
>
> Iam using Solr 4.1. one collection , 6 shards, 6 machines.
> I am using CloudSolrServer inside each mapper to index my documents…. While it is working fine , iam trying to improve the indexing performance.
>
>
> Question is:
>
> 1) is CloudSolrServer multiThreaded?
No. The proper fast way to use it is to start many threads that all add docs to the same CloudSolrServer instance. In other words, currently, you must do the multi threading yourself. CloudSolrServer is "thread safe".
>
> 2) Will using ConcurrentUpdateSolr server increase indexing performance?
Yes, but at the cost of having to specify a server to talk to - if it goes down, so does your indexing. It's also not very great at reporting errors. Finally, using multiple threads and CloudSolrServer, you can approach the performance of ConcurrentUpdateSolr server.
- Mark
>
> ./Zahoor
Re: CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
Posted by jmozah <jm...@gmail.com>.
Sorry.. i didn't understand that...
did you mean to configure CloudSolrServer with general hash algorithm?
./zahoor
On 17-Apr-2013, at 1:06 PM, rulinma <ru...@gmail.com> wrote:
> you also can cal(general hash algrothim) solrserver to add docs.
Re: CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
Posted by rulinma <ru...@gmail.com>.
you can use multithread.
for fast , you also can cal(general hash algrothim) solrserver to add docs.
--
View this message in context: http://lucene.472066.n3.nabble.com/CloudSolrServer-vs-ConcurrentUpdateSolrServer-for-indexing-tp4055772p4056606.html
Sent from the Solr - User mailing list archive at Nabble.com.