You are viewing a plain text version of this content. The canonical link for it is here.
Posted to client-dev@cassandra.apache.org by Paul LeoNerd Evans <le...@leonerd.org.uk> on 2013/09/11 18:20:16 UTC
Multi-node cluster-aware client connection
Having got the first stage of my client connector module nicely working
to a single node, I'm now looking at how to make it cluster-aware,
maintaining multiple connections for reliability and load-spreading.
What are some good strategies to take here?
My current plan involves connecting to a (randomly chosen from a list?)
seed node, to query the list of peers in the cluster, then make a
selection of some number of those to be "primary" nodes, and some more
as "backup" nodes. The primary nodes will be used to spread actual
query load around, the backups sitting idle simply as a fast way to
failover to some known-working connection if a primary falls over. By
registering an interest in topology and status change messages, the
client can keep the list of available nodes up-to-date.
1. What is a good way to handle prepared statements here? Should they
be prepared on all the (primary/all?) nodes, or just one? Some
applications I could imagine having just a handful of heavily-used
prepared statements, so they'd become a hotspot on one node if it
wasn't spread around. But then what to do as new nodes become
elected as primaries? Should they be prepared eagerly on
connection? Lazily at next use?
2. Secondly; what are suggested ways to actually spread load among the
primaries? I could imagine a simple round-robin, or something more
fancy involving picking the node with the fewest outstanding
requests, or the one on which we've been responsible for the least
processing time recently, or something else... Do client libraries
generally provide a selection of these mechanisms, or just pick one?
--
Paul "LeoNerd" Evans
leonerd@leonerd.org.uk
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
Re: Multi-node cluster-aware client connection
Posted by Sylvain Lebresne <sy...@datastax.com>.
On Wed, Sep 11, 2013 at 11:52 PM, Paul LeoNerd <le...@leonerd.org.uk>wrote:
> I have also found this
>
> https://issues.apache.org/jira/browse/CASSANDRA-4449
>
> which requests is be a single global ID across the cluster
Not exactly. The ticket was about making prepared statements global
to one node (not global to the cluster) rather to one client connection.
And yes, that's the case now as this ticket has been fixed in 1.2.0-beta2.
That being said, since you talk about "ID", for a given query string, if
you prepare it, it will always have the same ID technically because the
ID is a hash of the query string (and you can rely on that). That being
said, you still need to prepare the query string on every node of the
cluster: a node won't be able to execute a prepared statement that
hasn't been prepared on that node (and btw, prepared statements are
not persisted across node restart).
--
Sylvain
Re: Multi-node cluster-aware client connection
Posted by Paul LeoNerd Evans <le...@leonerd.org.uk>.
On Wed, 11 Sep 2013 17:20:16 +0100
Paul "LeoNerd" Evans <le...@leonerd.org.uk> wrote:
> 1. What is a good way to handle prepared statements here? Should they
> be prepared on all the (primary/all?) nodes, or just one? Some
> applications I could imagine having just a handful of heavily-used
> prepared statements, so they'd become a hotspot on one node if it
> wasn't spread around. But then what to do as new nodes become
> elected as primaries? Should they be prepared eagerly on
> connection? Lazily at next use?
So a careful reading of the python client [1] suggests that it's just
storing a single statement ID for the prepared statement across the
cluster. I have also found this
https://issues.apache.org/jira/browse/CASSANDRA-4449
which requests is be a single global ID across the cluster, rather than
per node. Is this now the case then - does the statement have a single
ID across the entire cluster? Can I rely on this? In this case, do I
still have to prepare it on the other nodes, or is a single node
sufficient?
[1]: https://github.com/datastax/python-driver/tree/master/cassandra
--
Paul "LeoNerd" Evans
leonerd@leonerd.org.uk
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
Re: Multi-node cluster-aware client connection
Posted by Paul LeoNerd Evans <le...@leonerd.org.uk>.
On Wed, 11 Sep 2013 09:42:20 -0700
Alex Popescu <al...@datastax.com> wrote:
> Out of curiosity, what language is your driver for?
It's a Perl one, written with proper asynchronous support:
https://metacpan.org/module/Net::Async::CassandraCQL
--
Paul "LeoNerd" Evans
leonerd@leonerd.org.uk
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
Re: Multi-node cluster-aware client connection
Posted by Alex Popescu <al...@datastax.com>.
Paul,
Out of curiosity, what language is your driver for?
On Wed, Sep 11, 2013 at 9:20 AM, Paul LeoNerd <le...@leonerd.org.uk>wrote:
> Having got the first stage of my client connector module nicely working
> to a single node, I'm now looking at how to make it cluster-aware,
> maintaining multiple connections for reliability and load-spreading.
> What are some good strategies to take here?
>
> My current plan involves connecting to a (randomly chosen from a list?)
> seed node, to query the list of peers in the cluster, then make a
> selection of some number of those to be "primary" nodes, and some more
> as "backup" nodes. The primary nodes will be used to spread actual
> query load around, the backups sitting idle simply as a fast way to
> failover to some known-working connection if a primary falls over. By
> registering an interest in topology and status change messages, the
> client can keep the list of available nodes up-to-date.
>
> 1. What is a good way to handle prepared statements here? Should they
> be prepared on all the (primary/all?) nodes, or just one? Some
> applications I could imagine having just a handful of heavily-used
> prepared statements, so they'd become a hotspot on one node if it
> wasn't spread around. But then what to do as new nodes become
> elected as primaries? Should they be prepared eagerly on
> connection? Lazily at next use?
>
> 2. Secondly; what are suggested ways to actually spread load among the
> primaries? I could imagine a simple round-robin, or something more
> fancy involving picking the node with the fewest outstanding
> requests, or the one on which we've been responsible for the least
> processing time recently, or something else... Do client libraries
> generally provide a selection of these mechanisms, or just pick one?
>
> --
> Paul "LeoNerd" Evans
>
> leonerd@leonerd.org.uk
> ICQ# 4135350 | Registered Linux# 179460
> http://www.leonerd.org.uk/
>
--
:- a)
Alex Popescu
Sen. Product Manager @ DataStax
@al3xandru