You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Edward Capriolo <ed...@gmail.com> on 2014/01/22 16:06:14 UTC
Introducing farsandra: A different way to integration test with c*
The repo:
https://github.com/edwardcapriolo/farsandra
The code:
Farsandra fs = new Farsandra();
fs.withVersion("2.0.4");
fs.withCleanInstanceOnStart(true);
fs.withInstanceName("1");
fs.withCreateConfigurationFiles(true);
fs.withHost("localhost");
fs.withSeeds(Arrays.asList("localhost"));
fs.start();
The story:
For a while I have been developing applications that use Apache Cassandra
as their data store. Personally I am more of an end-to-end test person then
a mock test person. For years I have relied heavily on Hector's embedded
cassandra to bring up Cassandra in a sane way inside a java project.
The concept of Farsandra is to keep Cassandra close (in end to end tests
and not mocked away) but keep your classpath closer (running cassandra
embedded should be seamless and not mess with your client classpath).
Recently there has been much fragmentation with Hector Asytanax, CQL, and
multiple Cassandra releases. Bringing up an embedded test is much harder
then it need be.
Cassandra's core methods get, put, slice over thrift have been
wire-compatible from version 0.7 - current. However Java libraries for
thrift and things like guava differ across the Cassandra versions. This
makes a large number of "issues" when trying to use your favourite client
with your 1 or more versions of Cassandra. (sometimes a thrift mismatch
kills the entire integration and you (CANT)! test anything.
Farsandra is much like https://github.com/pcmanus/ccm in that it launches
Cassandra instances remotely inside a sub-process. Farsandra is done in
java not python, making it easier to use with java development.
I will not go and say Farsandra solves all problems. in fact it has it's
own challenges (building yaml configurations across versions, fetching
binary cassandra from the internet), but it opens up new opportunities to
developer complicated multi-node testing scenarios which are impossible due
to re-entrant embedded cassandra code!
Have fun.
Re: Introducing farsandra: A different way to integration test with c*
Posted by Edward Capriolo <ed...@gmail.com>.
Farsandra 0.0.1 is in maven central. Added a couple features to allow
customizing cassandra.yaml and cassandra env (control memory of forked
instance), auto downloading of version specified.
http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22farsandra%22
On Wednesday, January 22, 2014, Edward Capriolo <ed...@gmail.com>
wrote:
> Right,
>
> This does not have to be thought of as a replacement for ccm or dtest.
>
> The particular problems I tend to have are:
>
> When trying to do Hive and Cassandra storage handler, Cassandra and Hive
had incompatible versions of antlr. Short of rebuilding one or both it can
not be resolved.
>
> I have had a version of Astyanax that is build against thrift 0.7.X and
Cassandra is using thrift 0.9.X. So if I can get the Cassandra Server off
the classpath the conflict goes away.
>
> You could do something like dtest like scenario or ccm thing as well. It
is a 100% java (minus the fork) solution. That has some wins but may not be
worth re-writing something you already have.
>
> Edward
>
>
>
>
> On Wed, Jan 22, 2014 at 10:11 PM, Jonathan Ellis <jb...@gmail.com>
wrote:
>>
>> Nice work, Ed. Personally, I do find it more productive to write
>> system tests in Python (dtest builds on ccm to provide a number of
>> utilities that cut down on the bolierplate [1]), but I can understand
>> that others will feel differently and more testing can only improve
>> Cassandra.
>>
>> Thanks!
>>
>> [1] https://github.com/riptano/cassandra-dtest
>>
>> On Wed, Jan 22, 2014 at 7:06 AM, Edward Capriolo <ed...@gmail.com>
wrote:
>> > The repo:
>> > https://github.com/edwardcapriolo/farsandra
>> >
>> > The code:
>> > Farsandra fs = new Farsandra();
>> > fs.withVersion("2.0.4");
>> > fs.withCleanInstanceOnStart(true);
>> > fs.withInstanceName("1");
>> > fs.withCreateConfigurationFiles(true);
>> > fs.withHost("localhost");
>> > fs.withSeeds(Arrays.asList("localhost"));
>> > fs.start();
>> >
>> > The story:
>> > For a while I have been developing applications that use Apache
Cassandra as
>> > their data store. Personally I am more of an end-to-end test person
then a
>> > mock test person. For years I have relied heavily on Hector's embedded
>> > cassandra to bring up Cassandra in a sane way inside a java project.
>> >
>> > The concept of Farsandra is to keep Cassandra close (in end to end
tests and
>> > not mocked away) but keep your classpath closer (running cassandra
embedded
>> > should be seamless and not mess with your client classpath).
>> >
>> > Recently there has been much fragmentation with Hector Asytanax, CQL,
and
>> > multiple Cassandra releases. Bringing up an embedded test is much
harder
>> > then it need be.
>> >
>> > Cassandra's core methods get, put, slice over thrift have been
>> > wire-compatible from version 0.7 - current. However Java libraries for
>> > thrift and things like guava differ across the Cassandra versions. This
>> > makes a large number of "issues" when trying to use your favourite
client
>> > with your 1 or more versions of Cassandra. (sometimes a thrift mismatch
>> > kills the entire integration and you (CANT)! test anything.
>> >
>> > Farsandra is much like https://github.com/pcmanus/ccm in that it
launches
>> > Cassandra instances remotely inside a sub-process. Farsandra is done
in java
>> > not python, making it easier to use with java development.
>> >
>> > I will not go and say Farsandra solves all problems. in fact it has
it's own
>> > challenges (building yaml configurations across versions, fetching
binary
>> > cassandra from the internet), but it opens up new opportunities to
developer
>> > complicated multi-node testing scenarios which are impossible due to
>> > re-entrant embedded cassandra code!
>> >
>> > Have fun.
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder, http://www.datastax.com
>> @spyced
>
>
--
Sorry this was sent from mobile. Will do less grammar and spell check than
usual.
Re: Introducing farsandra: A different way to integration test with c*
Posted by Edward Capriolo <ed...@gmail.com>.
Right,
This does not have to be thought of as a replacement for ccm or dtest.
The particular problems I tend to have are:
When trying to do Hive and Cassandra storage handler, Cassandra and Hive
had incompatible versions of antlr. Short of rebuilding one or both it can
not be resolved.
I have had a version of Astyanax that is build against thrift 0.7.X and
Cassandra is using thrift 0.9.X. So if I can get the Cassandra Server off
the classpath the conflict goes away.
You could do something like dtest like scenario or ccm thing as well. It is
a 100% java (minus the fork) solution. That has some wins but may not be
worth re-writing something you already have.
Edward
On Wed, Jan 22, 2014 at 10:11 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> Nice work, Ed. Personally, I do find it more productive to write
> system tests in Python (dtest builds on ccm to provide a number of
> utilities that cut down on the bolierplate [1]), but I can understand
> that others will feel differently and more testing can only improve
> Cassandra.
>
> Thanks!
>
> [1] https://github.com/riptano/cassandra-dtest
>
> On Wed, Jan 22, 2014 at 7:06 AM, Edward Capriolo <ed...@gmail.com>
> wrote:
> > The repo:
> > https://github.com/edwardcapriolo/farsandra
> >
> > The code:
> > Farsandra fs = new Farsandra();
> > fs.withVersion("2.0.4");
> > fs.withCleanInstanceOnStart(true);
> > fs.withInstanceName("1");
> > fs.withCreateConfigurationFiles(true);
> > fs.withHost("localhost");
> > fs.withSeeds(Arrays.asList("localhost"));
> > fs.start();
> >
> > The story:
> > For a while I have been developing applications that use Apache
> Cassandra as
> > their data store. Personally I am more of an end-to-end test person then
> a
> > mock test person. For years I have relied heavily on Hector's embedded
> > cassandra to bring up Cassandra in a sane way inside a java project.
> >
> > The concept of Farsandra is to keep Cassandra close (in end to end tests
> and
> > not mocked away) but keep your classpath closer (running cassandra
> embedded
> > should be seamless and not mess with your client classpath).
> >
> > Recently there has been much fragmentation with Hector Asytanax, CQL, and
> > multiple Cassandra releases. Bringing up an embedded test is much harder
> > then it need be.
> >
> > Cassandra's core methods get, put, slice over thrift have been
> > wire-compatible from version 0.7 - current. However Java libraries for
> > thrift and things like guava differ across the Cassandra versions. This
> > makes a large number of "issues" when trying to use your favourite client
> > with your 1 or more versions of Cassandra. (sometimes a thrift mismatch
> > kills the entire integration and you (CANT)! test anything.
> >
> > Farsandra is much like https://github.com/pcmanus/ccm in that it
> launches
> > Cassandra instances remotely inside a sub-process. Farsandra is done in
> java
> > not python, making it easier to use with java development.
> >
> > I will not go and say Farsandra solves all problems. in fact it has it's
> own
> > challenges (building yaml configurations across versions, fetching binary
> > cassandra from the internet), but it opens up new opportunities to
> developer
> > complicated multi-node testing scenarios which are impossible due to
> > re-entrant embedded cassandra code!
> >
> > Have fun.
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>
Re: Introducing farsandra: A different way to integration test with c*
Posted by Jonathan Ellis <jb...@gmail.com>.
Nice work, Ed. Personally, I do find it more productive to write
system tests in Python (dtest builds on ccm to provide a number of
utilities that cut down on the bolierplate [1]), but I can understand
that others will feel differently and more testing can only improve
Cassandra.
Thanks!
[1] https://github.com/riptano/cassandra-dtest
On Wed, Jan 22, 2014 at 7:06 AM, Edward Capriolo <ed...@gmail.com> wrote:
> The repo:
> https://github.com/edwardcapriolo/farsandra
>
> The code:
> Farsandra fs = new Farsandra();
> fs.withVersion("2.0.4");
> fs.withCleanInstanceOnStart(true);
> fs.withInstanceName("1");
> fs.withCreateConfigurationFiles(true);
> fs.withHost("localhost");
> fs.withSeeds(Arrays.asList("localhost"));
> fs.start();
>
> The story:
> For a while I have been developing applications that use Apache Cassandra as
> their data store. Personally I am more of an end-to-end test person then a
> mock test person. For years I have relied heavily on Hector's embedded
> cassandra to bring up Cassandra in a sane way inside a java project.
>
> The concept of Farsandra is to keep Cassandra close (in end to end tests and
> not mocked away) but keep your classpath closer (running cassandra embedded
> should be seamless and not mess with your client classpath).
>
> Recently there has been much fragmentation with Hector Asytanax, CQL, and
> multiple Cassandra releases. Bringing up an embedded test is much harder
> then it need be.
>
> Cassandra's core methods get, put, slice over thrift have been
> wire-compatible from version 0.7 - current. However Java libraries for
> thrift and things like guava differ across the Cassandra versions. This
> makes a large number of "issues" when trying to use your favourite client
> with your 1 or more versions of Cassandra. (sometimes a thrift mismatch
> kills the entire integration and you (CANT)! test anything.
>
> Farsandra is much like https://github.com/pcmanus/ccm in that it launches
> Cassandra instances remotely inside a sub-process. Farsandra is done in java
> not python, making it easier to use with java development.
>
> I will not go and say Farsandra solves all problems. in fact it has it's own
> challenges (building yaml configurations across versions, fetching binary
> cassandra from the internet), but it opens up new opportunities to developer
> complicated multi-node testing scenarios which are impossible due to
> re-entrant embedded cassandra code!
>
> Have fun.
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced
Re: Introducing farsandra: A different way to integration test with c*
Posted by Paulo Motta <pa...@gmail.com>.
Looks promising, thanks for the effort! :-)
I suggest you to add this description and a simple "getting started"
section with the above example to the project README, making it easier for
other to use your project.
Cheers,
2014/1/22 Edward Capriolo <ed...@gmail.com>
> The repo:
> https://github.com/edwardcapriolo/farsandra
>
> The code:
> Farsandra fs = new Farsandra();
> fs.withVersion("2.0.4");
> fs.withCleanInstanceOnStart(true);
> fs.withInstanceName("1");
> fs.withCreateConfigurationFiles(true);
> fs.withHost("localhost");
> fs.withSeeds(Arrays.asList("localhost"));
> fs.start();
>
> The story:
> For a while I have been developing applications that use Apache Cassandra
> as their data store. Personally I am more of an end-to-end test person then
> a mock test person. For years I have relied heavily on Hector's embedded
> cassandra to bring up Cassandra in a sane way inside a java project.
>
> The concept of Farsandra is to keep Cassandra close (in end to end tests
> and not mocked away) but keep your classpath closer (running cassandra
> embedded should be seamless and not mess with your client classpath).
>
> Recently there has been much fragmentation with Hector Asytanax, CQL, and
> multiple Cassandra releases. Bringing up an embedded test is much harder
> then it need be.
>
> Cassandra's core methods get, put, slice over thrift have been
> wire-compatible from version 0.7 - current. However Java libraries for
> thrift and things like guava differ across the Cassandra versions. This
> makes a large number of "issues" when trying to use your favourite client
> with your 1 or more versions of Cassandra. (sometimes a thrift mismatch
> kills the entire integration and you (CANT)! test anything.
>
> Farsandra is much like https://github.com/pcmanus/ccm in that it launches
> Cassandra instances remotely inside a sub-process. Farsandra is done in
> java not python, making it easier to use with java development.
>
> I will not go and say Farsandra solves all problems. in fact it has it's
> own challenges (building yaml configurations across versions, fetching
> binary cassandra from the internet), but it opens up new opportunities to
> developer complicated multi-node testing scenarios which are impossible due
> to re-entrant embedded cassandra code!
>
> Have fun.
>
--
Paulo Ricardo
--
European Master in Distributed Computing
* Royal Institute of Technology - KTH*
*Instituto Superior Técnico - IST*
*http://paulormg.com <http://paulormg.com>*