Re[2]: Clustering Lucene with 40 Servers

[Sven Duzont <sv...@laposte.net>]

Hi,

Me (& my comp) have also implemented the Doug's index replication solution :
- One Node responsible of the indexing.
- X Nodes handling the search requests.
The difference is that it is the role of the indexer Node to notify
(via Jgroups) that an indexation process is over.

For now we dont have the problems with Jgroups coz we dont have such
many Nodes (we handle currently 2 to 7 nodes) and the number of
messages per minute is very very low.

Yours,

--- Sven


Le jeudi 28 décembre 2006 à 08:33:37, vous écriviez :

AF> Hello, 

AF> I saw that Doug Cutting had an interesting solution for his Technorati website:
AF> http://www.mail-archive.com/lucene-user@jakarta.apache.org/msg12709.html

AF> It sounds like it's a single-writer, many readers type of system, but quite robust and efficient.

AF> Cheers, 

AF> Adam 



AF> ----------------------------------------
>> Date: Wed, 27 Dec 2006 10:46:47 -0800
>> From: sharrissf@gmail.com
>> To: java-user@lucene.apache.org
>> Subject: Re: Clustering Lucene with 40 Servers
>> 
>> Some quick questions/points:
>> 
>> What is the update rate?
>> 
>> The number of nodes you described is no problem, the query rate would be
>> no problem too (because they use read locks and act independently).
>> 
>> Do all nodes do updates or just 1? How often do these updates occur?
>> 
>> Probably best thing to do is give it a try and run a few tests.
>> 
>> Cheers,
>> Steve
>> 
>> 
>> On 12/27/06, Biggy <bi...@web.de> wrote:
>> >
>> > I'm currently investigating the best ways of clustering Lucene.
>> > I've heard of both Solr, Terracotta but do not know how well they scale.
>> > Their examples talk of a 4 node cluster. This is way too small for my needs.
>> >
>> > I have 30x JVMs each handling 3 requests/sec and each having their own
>> > Lucene index. The index changes are propagated to the cluster members using
>> > JGroups messages. This solution has more than reached its limit as JGroups
>> > has become unstable and a source of many JVMs crashes. Based on current
>> > traffic trends I anticipate needing to upgrade to 40x + JVMS very soon.
>> >
>> > Can anybody suggest a way to effectivily cluster / replicate document
>> > changes ?
>> >
>> > P.S.:
>> > JMS is not a possible solution as this was the prior JGroups solution. We
>> > had too many memory problems/queue full etc crashing the servers thereafter.
>> >
>> > --
>> > View this message in context: http://www.nabble.com/Clustering-Lucene-with-40-Servers-tf2886546.html#a8064135
>> > Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> 

AF> _________________________________________________________________
AF> Fixing up the home? Live Search can help.
AF> http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=wlmemailtaglinenov06
AF> ---------------------------------------------------------------------
AF> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
AF> For additional commands, e-mail: java-user-help@lucene.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org