You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Brimhall, Geoffrey" <ge...@intuit.com> on 2012/02/04 00:42:44 UTC
Anyone started work on RPC implementation for avro C ?
Hi -
Our project is using the Java avro RPC connectivity to flume, we have a bit of C++ and C code that needs to connect to flume also.
Checked out the avro.apache trunk, not seeing any RPC C implementation.
Wanted to double check if anyone out there has already started the C RPC implementation to collaborate with, otherwise will begin doing a prototype.
-geoff Brimhall
Re: Anyone started work on RPC implementation for avro C ?
Posted by Douglas Creager <do...@creagertino.net>.
> One thing to clarify - in looking over comments for 777, there was mention of JIRA-778 ( schema compiler ) code overlap. I'd like to focus on just the RPC mechanism in the patch's rpc.c; My understanding of AVRO is using a compiled schema is an optimization but not a requirement for AVRO's rpc. Or am I missing something ( that I'll probably discover as I dig in more ) ?
Unfortunately the schema compiler in AVRO-778 has languished a bit; with the new value API, and the generic value class (described in [1]), there was less of a need for it. Parts of that patch series have been applied separately, though, since they were useful for implementing the generic value class. But as of right now, there's no schema compilation or code generation in the C bindings.
So at the very least, the AVRO-777 patch should be updated to use the new value API instead of the deprecated datum API. AVRO-777 contains an AVDL compiler using the same Velocity templating engine that's used on the Java side of things. AVRO-778 contains the beginnings of a schema compiler that was based on the C preprocessor. It had fewer (i.e., no) dependencies to install, but it's kind of messy using the preprocessor as a templating language. So you have a couple of options there. I'll try to rebase the patch series from AVRO-778 so that it applies cleanly to trunk, in case you want to use parts of it.
–doug
[1] http://avro.apache.org/docs/current/api/c/index.html#_avro_values
RE: Anyone started work on RPC implementation for avro C ?
Posted by "Brimhall, Geoffrey" <ge...@intuit.com>.
Thanks !
Have downloaded the patches, will start taking a look at updating them to a single patch file to apply to the current trunk/.
One thing to clarify - in looking over comments for 777, there was mention of JIRA-778 ( schema compiler ) code overlap. I'd like to focus on just the RPC mechanism in the patch's rpc.c; My understanding of AVRO is using a compiled schema is an optimization but not a requirement for AVRO's rpc. Or am I missing something ( that I'll probably discover as I dig in more ) ?
-geoff Brimhall
-----Original Message-----
From: Douglas Creager [mailto:douglas@creagertino.net]
Sent: Sat 2/4/2012 7:13 AM
To: dev@avro.apache.org
Subject: Re: Anyone started work on RPC implementation for avro C ?
> Our project is using the Java avro RPC connectivity to flume, we have a bit of C++ and C code that needs to connect to flume also.
>
> Checked out the avro.apache trunk, not seeing any RPC C implementation.
>
> Wanted to double check if anyone out there has already started the C RPC implementation to collaborate with, otherwise will begin doing a prototype.
Hi Geoff,
There's a JIRA issue open for RPC support in the C bindings:
https://issues.apache.org/jira/browse/AVRO-777
This includes a patch, but we've had some API changes since then, so it doesn't apply cleanly. I think that would be a fantastic starting point if you're willing to contribute some development effort fixing any merge conflicts. And I'd be happy to answer any questions you might have about the changes and how they relate to the RPC patch.
cheers
-doug
Re: Anyone started work on RPC implementation for avro C ?
Posted by Douglas Creager <do...@creagertino.net>.
> Our project is using the Java avro RPC connectivity to flume, we have a bit of C++ and C code that needs to connect to flume also.
>
> Checked out the avro.apache trunk, not seeing any RPC C implementation.
>
> Wanted to double check if anyone out there has already started the C RPC implementation to collaborate with, otherwise will begin doing a prototype.
Hi Geoff,
There's a JIRA issue open for RPC support in the C bindings:
https://issues.apache.org/jira/browse/AVRO-777
This includes a patch, but we've had some API changes since then, so it doesn't apply cleanly. I think that would be a fantastic starting point if you're willing to contribute some development effort fixing any merge conflicts. And I'd be happy to answer any questions you might have about the changes and how they relate to the RPC patch.
cheers
–doug