You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Benjamin Black <b...@b3k.us> on 2010/04/03 10:14:29 UTC
multinode cluster wiki page
Just added this to the wiki as it seemed a very frequent request on
irc: http://wiki.apache.org/cassandra/MultinodeCluster
Would very much appreciate feedback and edits to improve it.
b
Re: multinode cluster wiki page
Posted by Ted Zlatanov <tz...@lifelogs.com>.
On Mon, 5 Apr 2010 13:10:38 -0500 Brandon Williams <dr...@gmail.com> wrote:
BW> 2010/4/5 Ted Zlatanov <tz...@lifelogs.com>
>> It would be nice if Cassandra looked at all the available interfaces and
>> selected the one whose reverse DNS lookup returned ".*cassandra.*" (or
>> some keyword the user provided).
>>
>> In other words, when you have
>>
>> eth0 = address X, reverse = "67.frontend.com"
>> eth1 = address Y, reverse = "cassandra-67.backend.com"
>>
>> eth1 should look better. So maybe ListenAddress could support this in
>> the configuration somehow, as a string spec or a
>> ListenAddressPreferReverse option. That would let those of us with
>> multiple interfaces use the exact same config everywhere.
BW> You can already accomplish this. Setup /etc/hosts correctly and leave
BW> ListenAddress blank.
Thanks, that's a much better solution.
Ted
getAddressFromNameService
Re: multinode cluster wiki page
Posted by Brandon Williams <dr...@gmail.com>.
2010/4/5 Ted Zlatanov <tz...@lifelogs.com>
> On Sat, 3 Apr 2010 14:10:37 -0500 Jonathan Ellis <jb...@gmail.com>
> wrote:
>
> JE> IMO the "right" way to do it is to configure your machines so that
> JE> autodetecting listenaddress Just Works, so you can deploy exactly the
> JE> same config to all nodes.
>
> It would be nice if Cassandra looked at all the available interfaces and
> selected the one whose reverse DNS lookup returned ".*cassandra.*" (or
> some keyword the user provided).
>
> In other words, when you have
>
> eth0 = address X, reverse = "67.frontend.com"
> eth1 = address Y, reverse = "cassandra-67.backend.com"
>
> eth1 should look better. So maybe ListenAddress could support this in
> the configuration somehow, as a string spec or a
> ListenAddressPreferReverse option. That would let those of us with
> multiple interfaces use the exact same config everywhere.
You can already accomplish this. Setup /etc/hosts correctly and leave
ListenAddress blank.
-Brandon
Re: multinode cluster wiki page
Posted by Ted Zlatanov <tz...@lifelogs.com>.
On Sat, 3 Apr 2010 14:10:37 -0500 Jonathan Ellis <jb...@gmail.com> wrote:
JE> IMO the "right" way to do it is to configure your machines so that
JE> autodetecting listenaddress Just Works, so you can deploy exactly the
JE> same config to all nodes.
It would be nice if Cassandra looked at all the available interfaces and
selected the one whose reverse DNS lookup returned ".*cassandra.*" (or
some keyword the user provided).
In other words, when you have
eth0 = address X, reverse = "67.frontend.com"
eth1 = address Y, reverse = "cassandra-67.backend.com"
eth1 should look better. So maybe ListenAddress could support this in
the configuration somehow, as a string spec or a
ListenAddressPreferReverse option. That would let those of us with
multiple interfaces use the exact same config everywhere.
Ted
Re: multinode cluster wiki page
Posted by Benjamin Black <b...@b3k.us>.
I do not claim it is the best/right way, just the one least likely to go wrong.
On Sat, Apr 3, 2010 at 12:10 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> IMO the "right" way to do it is to configure your machines so that
> autodetecting listenaddress Just Works, so you can deploy exactly the
> same config to all nodes.
>
> On Sat, Apr 3, 2010 at 3:14 AM, Benjamin Black <b...@b3k.us> wrote:
>> Just added this to the wiki as it seemed a very frequent request on
>> irc: http://wiki.apache.org/cassandra/MultinodeCluster
>>
>> Would very much appreciate feedback and edits to improve it.
>>
>>
>> b
>>
>
Re: multinode cluster wiki page
Posted by Jonathan Ellis <jb...@gmail.com>.
IMO the "right" way to do it is to configure your machines so that
autodetecting listenaddress Just Works, so you can deploy exactly the
same config to all nodes.
On Sat, Apr 3, 2010 at 3:14 AM, Benjamin Black <b...@b3k.us> wrote:
> Just added this to the wiki as it seemed a very frequent request on
> irc: http://wiki.apache.org/cassandra/MultinodeCluster
>
> Would very much appreciate feedback and edits to improve it.
>
>
> b
>
Re: multinode cluster wiki page
Posted by Benjamin Black <b...@b3k.us>.
Thank you! Updated.
On Sat, Apr 3, 2010 at 2:57 AM, Benoit Perroud <be...@noisette.ch> wrote:
> Hi,
>
> Nice work.
> I guess just a small mistake :
>
> the second <ListenAddress>192.168.1.1</ListenAddress> should be
> <ListenAddress>192.168.2.34</ListenAddress>
>
> And I would suggest to add a small part on making the thrift interface
> listening on more than localhost.
>
> Kind regards,
>
> Benoit.
>
> 2010/4/3 Benjamin Black <b...@b3k.us>:
>> Just added this to the wiki as it seemed a very frequent request on
>> irc: http://wiki.apache.org/cassandra/MultinodeCluster
>>
>> Would very much appreciate feedback and edits to improve it.
>>
>>
>> b
>>
>
Re: multinode cluster wiki page
Posted by Benoit Perroud <be...@noisette.ch>.
Hi,
Nice work.
I guess just a small mistake :
the second <ListenAddress>192.168.1.1</ListenAddress> should be
<ListenAddress>192.168.2.34</ListenAddress>
And I would suggest to add a small part on making the thrift interface
listening on more than localhost.
Kind regards,
Benoit.
2010/4/3 Benjamin Black <b...@b3k.us>:
> Just added this to the wiki as it seemed a very frequent request on
> irc: http://wiki.apache.org/cassandra/MultinodeCluster
>
> Would very much appreciate feedback and edits to improve it.
>
>
> b
>
Re: multinode cluster wiki page
Posted by Ted Zlatanov <tz...@lifelogs.com>.
On Sat, 3 Apr 2010 13:52:22 -0700 Benjamin Black <b...@b3k.us> wrote:
BB> What happens if the IP I get back is for a seed that happens to be
BB> down right then? And then that IP is cached locally by my resolver?
You have to set the TTL to be the right number of seconds for your
environment. With tinydns on a dedicated subdomain, even an old machine
could support really short TTLs.
BB> There is certainly a tempting conceptual simplicity to using DNS, I
BB> just don't think the reality is that simple nor is it for the trade in
BB> predictability, for me. IMO, this is better done either through
BB> automation to generate the configs (how I do it; I just update
BB> chef-server) or through a service like ZK (how I might do it in the
BB> future, in combination with automation).
DNS tends to be everywhere and easily configurable, so it's a pretty
good lowest common denominator. I think Zeroconf AKA mDNS/DNS-SD is a
good alternative to simple DNS RR for many environments and I will
eventually propose a contrib plugin for Cassandra that provides it if no
one else gets to it first (we discussed this previously). Modern Linux
systems support Zeroconf AKA mDNS/DNS-SD through Avahi.
Ted
Re: multinode cluster wiki page
Posted by Benjamin Black <b...@b3k.us>.
What happens if the IP I get back is for a seed that happens to be
down right then?
And then that IP is cached locally by my resolver?
There is certainly a tempting conceptual simplicity to using DNS, I
just don't think the reality is that simple nor is it for the trade in
predictability, for me. IMO, this is better done either through
automation to generate the configs (how I do it; I just update
chef-server) or through a service like ZK (how I might do it in the
future, in combination with automation).
b
On Sat, Apr 3, 2010 at 1:42 PM, banks <ba...@gmail.com> wrote:
> I can see the logic of having an internal DNS entry
> 'seednodes.internaldomain.com',
> this might only have 3 defined seed nodes out of 100, but the benefit is
> single point configuration, no need to edit configs across 100 machines,
> easily redefinable on the fly as needed...
>
> On Sat, Apr 3, 2010 at 1:13 PM, Benjamin Black <b...@b3k.us> wrote:
>>
>> Seems like a lot of complexity for a very small win (how often do you
>> bootstrap new nodes? if you only need a handful of seeds, what's all
>> that hard about listing them all on all nodes?). I prefer simple and
>> predictable, and trying to do this with round robin DNS seems to be
>> neither, to me.
>>
>>
>> b
>>
>> On Sat, Apr 3, 2010 at 12:55 PM, gabriele renzi <rf...@gmail.com> wrote:
>> > On Sat, Apr 3, 2010 at 6:40 PM, Avinash Lakshman
>> > <av...@gmail.com> wrote:
>> >> We use anywhere from 3-5 seeds for clusters that have over 150 nodes.
>> >> That
>> >> should suffice for larger sizes too since they are only for initial
>> >> discovery.
>> >
>> >
>> > would it make sense to just use a round robin dns on the available
>> > nodes and use that as seed?
>> > I do not have a clear understanding of the bootstrap mechanics but I
>> > have the impression that seed nodes do not have any special role, am I
>> > wrong?
>> >
>
>
Re: multinode cluster wiki page
Posted by banks <ba...@gmail.com>.
I can see the logic of having an internal DNS entry '
seednodes.internaldomain.com',
this might only have 3 defined seed nodes out of 100, but the benefit is
single point configuration, no need to edit configs across 100 machines,
easily redefinable on the fly as needed...
On Sat, Apr 3, 2010 at 1:13 PM, Benjamin Black <b...@b3k.us> wrote:
> Seems like a lot of complexity for a very small win (how often do you
> bootstrap new nodes? if you only need a handful of seeds, what's all
> that hard about listing them all on all nodes?). I prefer simple and
> predictable, and trying to do this with round robin DNS seems to be
> neither, to me.
>
>
> b
>
> On Sat, Apr 3, 2010 at 12:55 PM, gabriele renzi <rf...@gmail.com> wrote:
> > On Sat, Apr 3, 2010 at 6:40 PM, Avinash Lakshman
> > <av...@gmail.com> wrote:
> >> We use anywhere from 3-5 seeds for clusters that have over 150 nodes.
> That
> >> should suffice for larger sizes too since they are only for initial
> >> discovery.
> >
> >
> > would it make sense to just use a round robin dns on the available
> > nodes and use that as seed?
> > I do not have a clear understanding of the bootstrap mechanics but I
> > have the impression that seed nodes do not have any special role, am I
> > wrong?
> >
>
Re: multinode cluster wiki page
Posted by Benjamin Black <b...@b3k.us>.
Seems like a lot of complexity for a very small win (how often do you
bootstrap new nodes? if you only need a handful of seeds, what's all
that hard about listing them all on all nodes?). I prefer simple and
predictable, and trying to do this with round robin DNS seems to be
neither, to me.
b
On Sat, Apr 3, 2010 at 12:55 PM, gabriele renzi <rf...@gmail.com> wrote:
> On Sat, Apr 3, 2010 at 6:40 PM, Avinash Lakshman
> <av...@gmail.com> wrote:
>> We use anywhere from 3-5 seeds for clusters that have over 150 nodes. That
>> should suffice for larger sizes too since they are only for initial
>> discovery.
>
>
> would it make sense to just use a round robin dns on the available
> nodes and use that as seed?
> I do not have a clear understanding of the bootstrap mechanics but I
> have the impression that seed nodes do not have any special role, am I
> wrong?
>
Re: multinode cluster wiki page
Posted by gabriele renzi <rf...@gmail.com>.
On Sat, Apr 3, 2010 at 6:40 PM, Avinash Lakshman
<av...@gmail.com> wrote:
> We use anywhere from 3-5 seeds for clusters that have over 150 nodes. That
> should suffice for larger sizes too since they are only for initial
> discovery.
would it make sense to just use a round robin dns on the available
nodes and use that as seed?
I do not have a clear understanding of the bootstrap mechanics but I
have the impression that seed nodes do not have any special role, am I
wrong?
Re: multinode cluster wiki page
Posted by Avinash Lakshman <av...@gmail.com>.
We use anywhere from 3-5 seeds for clusters that have over 150 nodes. That
should suffice for larger sizes too since they are only for initial
discovery.
Avinash
On Sat, Apr 3, 2010 at 9:19 AM, Benjamin Black <b...@b3k.us> wrote:
> Seeds are used for ring discovery, so there really isn't a load
> concern for them, afaict. Have enough to meet your availability
> needs, including placement, and rock out.
>
> On Sat, Apr 3, 2010 at 9:01 AM, Joseph Ruscio <jr...@gmail.com> wrote:
> > Ben,
> >
> > Great, was looking for something like this just the other day. One
> question I'm still unclear on, when setting up multiple nodes, say 4-8 (or
> more) what's the suggested ratio of seed vs. non-seed nodes?
> >
> > thanks,
> > Joe
> >
> > On Apr 3, 2010, at 1:14 AM, Benjamin Black wrote:
> >
> >> Just added this to the wiki as it seemed a very frequent request on
> >> irc: http://wiki.apache.org/cassandra/MultinodeCluster
> >>
> >> Would very much appreciate feedback and edits to improve it.
> >>
> >>
> >> b
> >
> >
>
Re: multinode cluster wiki page
Posted by Benjamin Black <b...@b3k.us>.
Seeds are used for ring discovery, so there really isn't a load
concern for them, afaict. Have enough to meet your availability
needs, including placement, and rock out.
On Sat, Apr 3, 2010 at 9:01 AM, Joseph Ruscio <jr...@gmail.com> wrote:
> Ben,
>
> Great, was looking for something like this just the other day. One question I'm still unclear on, when setting up multiple nodes, say 4-8 (or more) what's the suggested ratio of seed vs. non-seed nodes?
>
> thanks,
> Joe
>
> On Apr 3, 2010, at 1:14 AM, Benjamin Black wrote:
>
>> Just added this to the wiki as it seemed a very frequent request on
>> irc: http://wiki.apache.org/cassandra/MultinodeCluster
>>
>> Would very much appreciate feedback and edits to improve it.
>>
>>
>> b
>
>
Re: multinode cluster wiki page
Posted by Joseph Ruscio <jr...@gmail.com>.
Ben,
Great, was looking for something like this just the other day. One question I'm still unclear on, when setting up multiple nodes, say 4-8 (or more) what's the suggested ratio of seed vs. non-seed nodes?
thanks,
Joe
On Apr 3, 2010, at 1:14 AM, Benjamin Black wrote:
> Just added this to the wiki as it seemed a very frequent request on
> irc: http://wiki.apache.org/cassandra/MultinodeCluster
>
> Would very much appreciate feedback and edits to improve it.
>
>
> b