Pooling Question

[David Wellman <da...@tynt.com>]

I have a 5 node cassandra cluster and I am wondering if there is any advantage of setting up a connection pool that is balanced across all 5 nodes (IE: 50 connections = 10 per node) over one pool all to one server (50 connection => one node)

Re: Pooling Question

[Jake Luciani <ja...@gmail.com>]

Look in /contrib it's already there.



On May 20, 2010, at 6:23 PM, Mark Robson <ma...@gmail.com> wrote:

> On 20 May 2010 23:16, Ryan Daum <ry...@thimbleware.com> wrote:
> I personally would love to see Cassandra add the concept of a read- 
> only 'proxy' node which acts like the embedded ready only mode (Java  
> 'fat client') but sits as a stand alone server. It would know the  
> the entire ring and watch Gossip and thus be able to direct requests  
> to the most appropriate node. This way thrift clients would just  
> connect to localhost, and it would in turn handle pooling and  
> balancing connections.
>
>
> I agree that would be a very desirable feature; app servers could  
> have a local Cassandra node which handled cluster operations.  
> Certainly makes things easier.
>
> It may also reduce network overhead as things could go directly to/ 
> from the nodes they needed to rather than via a third node.
>
> Mark

Re: Pooling Question

[Mark Robson <ma...@gmail.com>]

On 20 May 2010 23:16, Ryan Daum <ry...@thimbleware.com> wrote:

> I personally would love to see Cassandra add the concept of a read-only
> 'proxy' node which acts like the embedded ready only mode (Java 'fat
> client') but sits as a stand alone server. It would know the the entire ring
> and watch Gossip and thus be able to direct requests to the most appropriate
> node. This way thrift clients would just connect to localhost, and it would
> in turn handle pooling and balancing connections.
>
>
I agree that would be a very desirable feature; app servers could have a
local Cassandra node which handled cluster operations. Certainly makes
things easier.

It may also reduce network overhead as things could go directly to/from the
nodes they needed to rather than via a third node.

Mark

Re: Pooling Question

[Ryan Daum <ry...@thimbleware.com>]

I personally would love to see Cassandra add the concept of a read-only
'proxy' node which acts like the embedded ready only mode (Java 'fat
client') but sits as a stand alone server. It would know the the entire ring
and watch Gossip and thus be able to direct requests to the most appropriate
node. This way thrift clients would just connect to localhost, and it would
in turn handle pooling and balancing connections.

Ryan

On Thu, May 20, 2010 at 6:10 PM, Mark Robson <ma...@gmail.com> wrote:

> On 20 May 2010 20:17, David Wellman <da...@tynt.com> wrote:
>
>> I have a 5 node cassandra cluster and I am wondering if there is any
>> advantage of setting up a connection pool that is balanced across all 5
>> nodes (IE: 50 connections = 10 per node) over one pool all to one server (50
>> connection => one node)
>
>
> Depends on how many app servers you have and what your overall
> infrastructure looks like.
>
> Finding a working server to connect to is a (moderately) hard problem. Your
> app servers can try all the ones they know about until they find a working
> one, but doing that for each request is very expensive.
>
> Your app servers could find one that works and keep using it until it
> breaks.
>
> Another option is to run Cassandra nodes locally on your app servers, this
> makes capacity planning harder, but makes connecting easier as they can just
> always connect to a local instance.
>
> I wonder if any of the other Cassandra client layers have a mechanism for
> picking a working server?
>
> Mark
>

Re: Pooling Question

[Mark Robson <ma...@gmail.com>]

On 20 May 2010 20:17, David Wellman <da...@tynt.com> wrote:

> I have a 5 node cassandra cluster and I am wondering if there is any
> advantage of setting up a connection pool that is balanced across all 5
> nodes (IE: 50 connections = 10 per node) over one pool all to one server (50
> connection => one node)


Depends on how many app servers you have and what your overall
infrastructure looks like.

Finding a working server to connect to is a (moderately) hard problem. Your
app servers can try all the ones they know about until they find a working
one, but doing that for each request is very expensive.

Your app servers could find one that works and keep using it until it
breaks.

Another option is to run Cassandra nodes locally on your app servers, this
makes capacity planning harder, but makes connecting easier as they can just
always connect to a local instance.

I wonder if any of the other Cassandra client layers have a mechanism for
picking a working server?

Mark