AVRO client API

[Atul Gosain <at...@gmail.com>]

Hi

Is the client API for cassandra available in AVRO. If so, any links to
examples or some documentation ?
and If so, any comparison between Thrift and Avro API's to determine the
better of them ?

Thanks
Atul

Re: AVRO client API

["B. Todd Burruss" <bb...@real.com>]

i'll jump in ... why AVRO over Thrift.  can you guys point me at a 
comparison?  (i know next to nothing about both of them)

On 06/18/2010 03:41 PM, Paul Brown wrote:
> On Jun 18, 2010, at 2:12 PM, Eric Evans wrote:
>
>    
>> On Fri, 2010-06-18 at 11:00 -0700, Paul Brown wrote:
>>      
>>> At the risk of asking about religion (but with no interest in hearing
>>> about it), why Avro instead of something like plain-old-JSON over
>>> HTTP?
>>>        
>> At the risk of having this thread veer off on a very long tangent...
>> In a nutshell, we need a way of processing requests and responses over
>> the network with typed data. You could of course put something together
>> to do this using JSON and HTTP, but not without reimplementing another
>> framework like Avro or Thrift (both of which can do JSON encoding, and
>> both of which have an HTTP transport).
>>      
> "Rich, natively-provided types" is a fair answer; I was more interested in motivation that making a value judgement.
>
> Cheers.
>
> -- Paul

Re: AVRO client API

[Paul Brown <pa...@gmail.com>]

On Jun 18, 2010, at 2:12 PM, Eric Evans wrote:

> On Fri, 2010-06-18 at 11:00 -0700, Paul Brown wrote:
>> At the risk of asking about religion (but with no interest in hearing
>> about it), why Avro instead of something like plain-old-JSON over
>> HTTP?
> At the risk of having this thread veer off on a very long tangent...
> In a nutshell, we need a way of processing requests and responses over
> the network with typed data. You could of course put something together
> to do this using JSON and HTTP, but not without reimplementing another
> framework like Avro or Thrift (both of which can do JSON encoding, and
> both of which have an HTTP transport).

"Rich, natively-provided types" is a fair answer; I was more interested in motivation that making a value judgement.

Cheers.

-- Paul

Re: AVRO client API

[Tatu Saloranta <ts...@gmail.com>]

On Fri, Jun 18, 2010 at 4:57 PM, Miguel Verde <mi...@gmail.com> wrote:
> On Fri, Jun 18, 2010 at 6:23 PM, Tatu Saloranta <ts...@gmail.com>
> wrote:
>>
>> Not that I wanted to criticize choices, but do they actually allow use
>> of JSON as encoding?
>> Avro does use JSON for specifying schemas, but I wasn't aware of being
>> able to use it for encoding data.
>> Likewise with Thrift.
>
> Yes, each supports a JSON data encoding.  See
> http://avro.apache.org/docs/1.3.3/spec.html#json_encoding for Avro and the
> JSONProtocol in Thrift.  One clear advantage of these two is that they

Ok thanks. I learnt something new today. :-)

> support either stringified JSON or a compact binary encoding, and that they
> each support (or intend to support) a more efficient TCP-based protocol
> instead of only allowing HTTP.

Right. Latter is actually useful, then, as that would suggest
possibility of using alternative binary encodings with other pieces
(schema definition, protocol handling)....
(encoding that supports their respective data sets)

> Re: Avro vs Thrift, Cassandra has historically had difficulty getting Thrift
> bugs fixed and Avro is more malleable at this point.  Additionally, Avro has
> the potential for a more compact encoding and easier integration with
> dynamic languages.

Yes, that has been my impression as well, so I was not surprised to
see plans for this change.
Although I have been interested in learning more about progress, to
know when would new versions be available.

-+ Tatu +-

Re: AVRO client API

[Miguel Verde <mi...@gmail.com>]

On Fri, Jun 18, 2010 at 6:23 PM, Tatu Saloranta <ts...@gmail.com>wrote:

>  Not that I wanted to criticize choices, but do they actually allow use
> of JSON as encoding?
> Avro does use JSON for specifying schemas, but I wasn't aware of being
> able to use it for encoding data.
> Likewise with Thrift.
>

 Yes, each supports a JSON data encoding.  See
http://avro.apache.org/docs/1.3.3/spec.html#json_encoding for Avro and the
JSONProtocol in Thrift.  One clear advantage of these two is that they
support either stringified JSON or a compact binary encoding, and that they
each support (or intend to support) a more efficient TCP-based protocol
instead of only allowing HTTP.

Re: Avro vs Thrift, Cassandra has historically had difficulty getting Thrift
bugs fixed and Avro is more malleable at this point.  Additionally, Avro has
the potential for a more compact encoding and easier integration with
dynamic languages.

Re: AVRO client API

[Tatu Saloranta <ts...@gmail.com>]

On Fri, Jun 18, 2010 at 2:12 PM, Eric Evans <ee...@rackspace.com> wrote:
> On Fri, 2010-06-18 at 11:00 -0700, Paul Brown wrote:
>> At the risk of asking about religion (but with no interest in hearing
>> about it), why Avro instead of something like plain-old-JSON over
>> HTTP?
>
> At the risk of having this thread veer off on a very long tangent...
>
> In a nutshell, we need a way of processing requests and responses over
> the network with typed data. You could of course put something together
> to do this using JSON and HTTP, but not without reimplementing another
> framework like Avro or Thrift (both of which can do JSON encoding, and
> both of which have an HTTP transport).

Not that I wanted to criticize choices, but do they actually allow use
of JSON as encoding?
Avro does use JSON for specifying schemas, but I wasn't aware of being
able to use it for encoding data.
Likewise with Thrift.

I think there's also important question of whether schemas/formatting
choice for payload should follow that of framing.
Avro/Thrift/PB seem reasonable for framing, use by protocol itself;
but for open payload it might make sense to allow different pluggable
formats.
Mostly because Avro/Thrift/PB are schema-bound formats which is not an
optimal choice for many use cases (but are fine for many others)
It is of course possible to just use byte[]/String as payload, handle
encoding and decoding on client end, and maybe that's how it should
be, for cases where strict schema doesn't work for use cases.

-+ Tatu +-

Re: AVRO client API

[Eric Evans <ee...@rackspace.com>]

On Fri, 2010-06-18 at 11:00 -0700, Paul Brown wrote:
> At the risk of asking about religion (but with no interest in hearing
> about it), why Avro instead of something like plain-old-JSON over
> HTTP?

At the risk of having this thread veer off on a very long tangent...

In a nutshell, we need a way of processing requests and responses over
the network with typed data. You could of course put something together
to do this using JSON and HTTP, but not without reimplementing another
framework like Avro or Thrift (both of which can do JSON encoding, and
both of which have an HTTP transport).

-- 
Eric Evans
eevans@rackspace.com

Re: AVRO client API

[Paul Brown <pa...@gmail.com>]

On Jun 18, 2010, at 8:01 AM, Eric Evans wrote:
> On Fri, 2010-06-18 at 12:27 +0530, Atul Gosain wrote:
>> Is the client API for cassandra available in AVRO.
> Significant parts of it, but it is not yet finished.
>> If so, any links to examples or some documentation?
> There is no samples or documentation yet, sorry.
>> and If so, any comparison between Thrift and Avro API's to determine
>> the better of them?
> The Plan is to develop enough critical mass around the Avro API that
> Thrift can be deprecated. We don't want to maintain more than one of
> these long-term.

At the risk of asking about religion (but with no interest in hearing about it), why Avro instead of something like plain-old-JSON over HTTP?

-- Paul

Re: AVRO client API

[Eric Evans <ee...@rackspace.com>]

On Fri, 2010-06-18 at 12:27 +0530, Atul Gosain wrote:
> Is the client API for cassandra available in AVRO.

Significant parts of it, but it is not yet finished.

> If so, any links to examples or some documentation?

There is no samples or documentation yet, sorry.

> and If so, any comparison between Thrift and Avro API's to determine
> the better of them?

The Plan is to develop enough critical mass around the Avro API that
Thrift can be deprecated. We don't want to maintain more than one of
these long-term.

-- 
Eric Evans
eevans@rackspace.com