You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by matthew hawthorne <mh...@gmail.com> on 2010/12/22 20:07:20 UTC
interest in creating a "cassandra-gossip" library?
hello,
I'm starting a project at my day job to deploy a gossip protocol
implementation. part of my initial work is to evaluate existing
implementations.
being loosely familiar with Cassandra, I read
http://wiki.apache.org/cassandra/ArchitectureGossip and have looked
over the related code a bit.
is there interest in breaking out the gossip-related portions of
Cassandra into a library that could be reused by other projects? I
work on a team that is ready and willing to contribute heavily. we'd
just need some guidance as to how to structure the Cassandra
subcomponent(s) and properly integrate them with the builds, tests,
etc.
here are a few examples of functionality we're looking to add:
1) hierarchical state - our use case is cross data center gossip,
where we don't want every node in the 2 clusters communicating, but do
want a node from cluster1 to send a summary of the cluster's state to
cluster2, and vice versa. essentially I'm talking about rolling up
the state of multiple nodes into a single "virtual" node
2) mutual authentication - nodes verifying the identity of other nodes
before gossipping
3) encryption - encrypted traffic, especially for the cross data center case
any opinions on this? thanks in advance for any feedback!
-matt
Re: interest in creating a "cassandra-gossip" library?
Posted by Stephen Connolly <st...@gmail.com>.
I've pinged peter, as the only builds of his that I know of are for the
company I'm working for (until march) and he used to work for... Hopefully
he has some public examples
- Stephen
---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 29 Dec 2010 15:00, "Eric Evans" <ee...@rackspace.com> wrote:
> On Tue, 2010-12-28 at 22:25 +0000, Stephen Connolly wrote:
>> In my experience, one big source tree leads to unwanted module
>> dependencies,
>> and can lead to circular module dependencies, which render the
>> modulization
>> moot anyway... (note I am not saying to go to m_v__ as a build tool,
>> just to
>> keep module source code in separate trees so that intra-module
>> dependencies
>> can be controlled... Of course I believe that other build tool is good
>> at
>> helping, but if you've ever seen one of peter reilly's ANT builds
>> you'll
>> know it can be done easily and beautifully in ANT... Admittedly he is
>> on the
>> ANT pmc ;-) )
>
> Can you link to an example of one of these?
>
> --
> Eric Evans
> eevans@rackspace.com
>
Re: interest in creating a "cassandra-gossip" library?
Posted by Eric Evans <ee...@rackspace.com>.
On Tue, 2010-12-28 at 22:25 +0000, Stephen Connolly wrote:
> In my experience, one big source tree leads to unwanted module
> dependencies,
> and can lead to circular module dependencies, which render the
> modulization
> moot anyway... (note I am not saying to go to m_v__ as a build tool,
> just to
> keep module source code in separate trees so that intra-module
> dependencies
> can be controlled... Of course I believe that other build tool is good
> at
> helping, but if you've ever seen one of peter reilly's ANT builds
> you'll
> know it can be done easily and beautifully in ANT... Admittedly he is
> on the
> ANT pmc ;-) )
Can you link to an example of one of these?
--
Eric Evans
eevans@rackspace.com
Re: interest in creating a "cassandra-gossip" library?
Posted by Stephen Connolly <st...@gmail.com>.
Is there a specific reason to avoid modularizing the source folders if we
are modularizing the jars?
In my experience, one big source tree leads to unwanted module dependencies,
and can lead to circular module dependencies, which render the modulization
moot anyway... (note I am not saying to go to m_v__ as a build tool, just to
keep module source code in separate trees so that intra-module dependencies
can be controlled... Of course I believe that other build tool is good at
helping, but if you've ever seen one of peter reilly's ANT builds you'll
know it can be done easily and beautifully in ANT... Admittedly he is on the
ANT pmc ;-) )
- Stephen
---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 28 Dec 2010 20:17, "Ryan King" <ry...@twitter.com> wrote:
> On Wed, Dec 22, 2010 at 11:07 AM, matthew hawthorne
> <mh...@gmail.com> wrote:
>> hello,
>>
>> I'm starting a project at my day job to deploy a gossip protocol
>> implementation. part of my initial work is to evaluate existing
>> implementations.
>>
>> being loosely familiar with Cassandra, I read
>> http://wiki.apache.org/cassandra/ArchitectureGossip and have looked
>> over the related code a bit.
>>
>> is there interest in breaking out the gossip-related portions of
>> Cassandra into a library that could be reused by other projects? I
>> work on a team that is ready and willing to contribute heavily. we'd
>> just need some guidance as to how to structure the Cassandra
>> subcomponent(s) and properly integrate them with the builds, tests,
>> etc.
>
> I think breaking gossip out could be very valuable. Gossip is a
> somewhat under-understood and under-tested part of Cassandra.
> Isolating it could help both of those situations. Ideally we'd not
> change the normal build procedure for cassandra, but add a new target
> for building a gossip library.
>
>> here are a few examples of functionality we're looking to add:
>>
>> 1) hierarchical state - our use case is cross data center gossip,
>> where we don't want every node in the 2 clusters communicating, but do
>> want a node from cluster1 to send a summary of the cluster's state to
>> cluster2, and vice versa. essentially I'm talking about rolling up
>> the state of multiple nodes into a single "virtual" node
>
> I'd be interested in seeing the analysis here. There are some
> challenging problems with having only one host do something in a
> gossip based system.
>
>> 2) mutual authentication - nodes verifying the identity of other nodes
>> before gossipping
>>
>> 3) encryption - encrypted traffic, especially for the cross data center
case
>>
>> any opinions on this? thanks in advance for any feedback!
>
> -ryan
Re: interest in creating a "cassandra-gossip" library?
Posted by Ryan King <ry...@twitter.com>.
On Wed, Dec 22, 2010 at 11:07 AM, matthew hawthorne
<mh...@gmail.com> wrote:
> hello,
>
> I'm starting a project at my day job to deploy a gossip protocol
> implementation. part of my initial work is to evaluate existing
> implementations.
>
> being loosely familiar with Cassandra, I read
> http://wiki.apache.org/cassandra/ArchitectureGossip and have looked
> over the related code a bit.
>
> is there interest in breaking out the gossip-related portions of
> Cassandra into a library that could be reused by other projects? I
> work on a team that is ready and willing to contribute heavily. we'd
> just need some guidance as to how to structure the Cassandra
> subcomponent(s) and properly integrate them with the builds, tests,
> etc.
I think breaking gossip out could be very valuable. Gossip is a
somewhat under-understood and under-tested part of Cassandra.
Isolating it could help both of those situations. Ideally we'd not
change the normal build procedure for cassandra, but add a new target
for building a gossip library.
> here are a few examples of functionality we're looking to add:
>
> 1) hierarchical state - our use case is cross data center gossip,
> where we don't want every node in the 2 clusters communicating, but do
> want a node from cluster1 to send a summary of the cluster's state to
> cluster2, and vice versa. essentially I'm talking about rolling up
> the state of multiple nodes into a single "virtual" node
I'd be interested in seeing the analysis here. There are some
challenging problems with having only one host do something in a
gossip based system.
> 2) mutual authentication - nodes verifying the identity of other nodes
> before gossipping
>
> 3) encryption - encrypted traffic, especially for the cross data center case
>
> any opinions on this? thanks in advance for any feedback!
-ryan
Re: AW: interest in creating a "cassandra-gossip" library?
Posted by Edward Capriolo <ed...@gmail.com>.
Would this help?
http://svn.apache.org/viewvc?view=revision&revision=1157967
On Fri, Sep 2, 2011 at 10:01 AM, Jake Farrell <jf...@apache.org> wrote:
> Roland, I was also interested in this for very similar reasons. I was
> planning on working on this after tackling some of the thrift build clean
> and transport requests i've been working on. I created CASSANDRA-3125 to
> help track the progress of this a little easier
>
> -Jake
>
> On Sep 2, 2011, at 5:05 AM, Roland Gude wrote:
>
> > Hi,
> >
> > Is there any progress on this topic?
> > I would really like to see such a library.
> >
> > -----Ursprüngliche Nachricht-----
> > Von: matthew hawthorne [mailto:mhawthorne@gmail.com]
> > Gesendet: Mittwoch, 22. Dezember 2010 20:07
> > An: dev@cassandra.apache.org
> > Betreff: interest in creating a "cassandra-gossip" library?
> >
> > hello,
> >
> > I'm starting a project at my day job to deploy a gossip protocol
> > implementation. part of my initial work is to evaluate existing
> > implementations.
> >
> > being loosely familiar with Cassandra, I read
> > http://wiki.apache.org/cassandra/ArchitectureGossip and have looked
> > over the related code a bit.
> >
> > is there interest in breaking out the gossip-related portions of
> > Cassandra into a library that could be reused by other projects? I
> > work on a team that is ready and willing to contribute heavily. we'd
> > just need some guidance as to how to structure the Cassandra
> > subcomponent(s) and properly integrate them with the builds, tests,
> > etc.
> >
> > here are a few examples of functionality we're looking to add:
> >
> > 1) hierarchical state - our use case is cross data center gossip,
> > where we don't want every node in the 2 clusters communicating, but do
> > want a node from cluster1 to send a summary of the cluster's state to
> > cluster2, and vice versa. essentially I'm talking about rolling up
> > the state of multiple nodes into a single "virtual" node
> >
> > 2) mutual authentication - nodes verifying the identity of other nodes
> > before gossipping
> >
> > 3) encryption - encrypted traffic, especially for the cross data center
> case
> >
> > any opinions on this? thanks in advance for any feedback!
> >
> > -matt
> >
>
>
Re: AW: interest in creating a "cassandra-gossip" library?
Posted by Jake Farrell <jf...@apache.org>.
Roland, I was also interested in this for very similar reasons. I was planning on working on this after tackling some of the thrift build clean and transport requests i've been working on. I created CASSANDRA-3125 to help track the progress of this a little easier
-Jake
On Sep 2, 2011, at 5:05 AM, Roland Gude wrote:
> Hi,
>
> Is there any progress on this topic?
> I would really like to see such a library.
>
> -----Ursprüngliche Nachricht-----
> Von: matthew hawthorne [mailto:mhawthorne@gmail.com]
> Gesendet: Mittwoch, 22. Dezember 2010 20:07
> An: dev@cassandra.apache.org
> Betreff: interest in creating a "cassandra-gossip" library?
>
> hello,
>
> I'm starting a project at my day job to deploy a gossip protocol
> implementation. part of my initial work is to evaluate existing
> implementations.
>
> being loosely familiar with Cassandra, I read
> http://wiki.apache.org/cassandra/ArchitectureGossip and have looked
> over the related code a bit.
>
> is there interest in breaking out the gossip-related portions of
> Cassandra into a library that could be reused by other projects? I
> work on a team that is ready and willing to contribute heavily. we'd
> just need some guidance as to how to structure the Cassandra
> subcomponent(s) and properly integrate them with the builds, tests,
> etc.
>
> here are a few examples of functionality we're looking to add:
>
> 1) hierarchical state - our use case is cross data center gossip,
> where we don't want every node in the 2 clusters communicating, but do
> want a node from cluster1 to send a summary of the cluster's state to
> cluster2, and vice versa. essentially I'm talking about rolling up
> the state of multiple nodes into a single "virtual" node
>
> 2) mutual authentication - nodes verifying the identity of other nodes
> before gossipping
>
> 3) encryption - encrypted traffic, especially for the cross data center case
>
> any opinions on this? thanks in advance for any feedback!
>
> -matt
>
AW: interest in creating a "cassandra-gossip" library?
Posted by Roland Gude <ro...@yoochoose.com>.
Hi,
Is there any progress on this topic?
I would really like to see such a library.
-----Ursprüngliche Nachricht-----
Von: matthew hawthorne [mailto:mhawthorne@gmail.com]
Gesendet: Mittwoch, 22. Dezember 2010 20:07
An: dev@cassandra.apache.org
Betreff: interest in creating a "cassandra-gossip" library?
hello,
I'm starting a project at my day job to deploy a gossip protocol
implementation. part of my initial work is to evaluate existing
implementations.
being loosely familiar with Cassandra, I read
http://wiki.apache.org/cassandra/ArchitectureGossip and have looked
over the related code a bit.
is there interest in breaking out the gossip-related portions of
Cassandra into a library that could be reused by other projects? I
work on a team that is ready and willing to contribute heavily. we'd
just need some guidance as to how to structure the Cassandra
subcomponent(s) and properly integrate them with the builds, tests,
etc.
here are a few examples of functionality we're looking to add:
1) hierarchical state - our use case is cross data center gossip,
where we don't want every node in the 2 clusters communicating, but do
want a node from cluster1 to send a summary of the cluster's state to
cluster2, and vice versa. essentially I'm talking about rolling up
the state of multiple nodes into a single "virtual" node
2) mutual authentication - nodes verifying the identity of other nodes
before gossipping
3) encryption - encrypted traffic, especially for the cross data center case
any opinions on this? thanks in advance for any feedback!
-matt