Data types for cross language access

[Oliver Dungey <ol...@gmail.com>]

I am currently working on a system with Cassandra that is written purely in
Java. I know our end solution will require other languages to access the
data in Cassandra (Python, C++ etc.). What is the best way to store data to
ensure I can do this? Should I serialize everything to strings/json/xml
prior to the byte conversion? We currently use the Hector serializer, I
wondered if we should just switch this to something like Jackson/JAXB? Any
thoughts very welcome.

Re: Data types for cross language access

[Alex Araujo <ca...@alex.otherinbox.com>]

On 5/11/11 5:27 AM, Oliver Dungey wrote:
> I am currently working on a system with Cassandra that is written 
> purely in Java. I know our end solution will require other languages 
> to access the data in Cassandra (Python, C++ etc.). What is the best 
> way to store data to ensure I can do this? Should I serialize 
> everything to strings/json/xml prior to the byte conversion? We 
> currently use the Hector serializer, I wondered if we should just 
> switch this to something like Jackson/JAXB? Any thoughts very welcome. 
I believe most high level (non-Thrift) clients convert types to/from 
bytes consistently without additional serialization (XML, JSON, etc).  
There may be a few tricks to working with TimeUUIDs for slices, but at 
least the Java and Python versions appear to be compatible. It's 
probably worth writing a few tests using your target languages to make sure:

http://wiki.apache.org/cassandra/ClientOptions

Don't see a C++ client, but a quick Google search turned up:

http://github.com/posulliv/libcassandra

Re: Data types for cross language access

[Eric tamme <et...@gmail.com>]

> On Wed, May 11, 2011 at 10:23 AM, Luke Biddell <lu...@gmail.com> wrote:
>> I wouldn't mind knowing how other people are approaching this problem too.
>>
>> On 11 May 2011 11:27, Oliver Dungey <ol...@gmail.com> wrote:
>>> I am currently working on a system with Cassandra that is written purely in
>>> Java. I know our end solution will require other languages to access the
>>> data in Cassandra (Python, C++ etc.). What is the best way to store data to
>>> ensure I can do this? Should I serialize everything to strings/json/xml
>>> prior to the byte conversion? We currently use the Hector serializer, I
>>> wondered if we should just switch this to something like Jackson/JAXB? Any
>>> thoughts very welcome.
>>
>

We have clients that use python and C++.  We just generated thrift
bindings for C++ and use thrift directly - it really is not bad at
all, the cassandra.h file generated defines most all methods that any
higher level API would, it just doesn't have built in pooling, or
reconnect etc.

As far as data formats go - we are capturing data (packets) using
libpcap and storing the raw bytes  (literally the entire raw packet
including ethernet and ip headers) into cassandra which are later read
out by python scripts for ascii display, or conversion to pcap files
to open in wireshark.

-Eric

Re: Data types for cross language access

[Nate McCall <na...@datastax.com>]

You should have no problems with byte conversion consistencies. For
the serialization test cases in Hector, we verify the most of the
results with o.a.c.utils.ByteBufferUtil from Cassandra source.

On Wed, May 11, 2011 at 10:23 AM, Luke Biddell <lu...@gmail.com> wrote:
> I wouldn't mind knowing how other people are approaching this problem too.
>
> On 11 May 2011 11:27, Oliver Dungey <ol...@gmail.com> wrote:
>> I am currently working on a system with Cassandra that is written purely in
>> Java. I know our end solution will require other languages to access the
>> data in Cassandra (Python, C++ etc.). What is the best way to store data to
>> ensure I can do this? Should I serialize everything to strings/json/xml
>> prior to the byte conversion? We currently use the Hector serializer, I
>> wondered if we should just switch this to something like Jackson/JAXB? Any
>> thoughts very welcome.
>

Re: Data types for cross language access

[Luke Biddell <lu...@gmail.com>]

I wouldn't mind knowing how other people are approaching this problem too.

On 11 May 2011 11:27, Oliver Dungey <ol...@gmail.com> wrote:
> I am currently working on a system with Cassandra that is written purely in
> Java. I know our end solution will require other languages to access the
> data in Cassandra (Python, C++ etc.). What is the best way to store data to
> ensure I can do this? Should I serialize everything to strings/json/xml
> prior to the byte conversion? We currently use the Hector serializer, I
> wondered if we should just switch this to something like Jackson/JAXB? Any
> thoughts very welcome.