You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by tl <tl...@rat.io> on 2016/02/01 15:03:18 UTC

Re: Autogenerating Java classes from IDL

> On 31.01.2016, at 01:13, Evan McClain <ae...@gmail.com> wrote:
> 
> I have used IDLs with the avro-maven-plugin and that worked for me.
> Evan

Maven is black art for me but your answer encouraged me to stare at the command line output of avro-tools again (since I considered it improbable that the maven-plugin can invoke things that the CLI can't). "idl2schemata" does indeed provide a good part of what I want in that it omits the protocol declarations and only transforms the schema parts.

I guess that Maven would allow me to set up a process where Java classes would be generated from the JSON schemata that idl2schemata extracts from the IDL protocol definition but I’m having trouble figuring out how to configure the POM. 

Just to prove that I’m not totally useless I googled me the following snippet that should generate classes from a schema.

<executions>
  <execution>
    <phase>generate-sources</phase>
    <goals>
      <goal>schema</goal>
    </goals>
    <configuration>
      <sourceDirectory>
        ${project.basedir}/src/main/avro/
      </sourceDirectory>
      <outputDirectory>
        ${project.basedir}/src/main/java/
      </outputDirectory>
    </configuration>
  </execution>
</executions>

I reckon that a similar instruction, preceding this one, could initiate the conversion from an IDl to a schema (like the CLI tool does with "idl2schema"). What should the relevant snippet look like?


Thanks!
Thomas






 








> On Sat, Jan 30, 2016, 8:52 AM tl <tl...@rat.io> wrote:
> Hi,
> 
> I started working with Avro only recently so maybe I missed something but it seems to me that schemas can be defined in JSON as well as in IDL but only from JSON schemas can builder classes be autogenerated. This is a pity since IDLs are much easier to write and read than the JSON representation.
> I wrote a few IDL schemas, converted them to avpr and tweaked those avpr files to become valid avsc schema files from which I autogenerated the classes. This is a rather convoluted process. I wouldn’t mind so much if I wouldn’t know that I or somebody else will have to update the schemas and classes from time to time. This doesn’t look like a robust workflow.
> 
> Deleting the IDLs and avpr files and doing updates only in the avsc schemas reduces the workflow to 2 steps but I loose the nice properties of IDLs [0].
> Using only generic mapping would reduce the workflow by one step too but I’d loose static type checking.
> 
> I’d love to be able to autogenerate the Java classes from the IDLs directly. Is there a way?
> 
> Regards,
> Thomas
> 
> 
> [0] Regarding IDLs it would be cool if I could forward reference objects in the schema or even write nested schemas but that’s a relatively minor gripe.
>

Re: Autogenerating Java classes from IDL

Posted by tl <tl...@rat.io>.
Hi,

that sounds exactly like what I want. I did use the commandline tools but I’ll try to set up the Maven plugin to automatically generate classes with goal IDL next time I need to edit them.

Thanks for the hint!
Thomas


> On 05.02.2016, at 02:16, Matheus Santana <ed...@gmail.com> wrote:
> 
> Thomas,
> 
> did you solve your problem already?
> 
> You don't need nothing but the Avro Maven plugin for generating Java classes from Avro IDL. That is exactly what the idl goal does. Your snippet looks nice, just configure the idl goal instead of the schema goal.
> 
> On Mon, Feb 1, 2016 at 11:03 AM, tl <tl...@rat.io> wrote:
> 
> > On 31.01.2016, at 01:13, Evan McClain <ae...@gmail.com> wrote:
> >
> > I have used IDLs with the avro-maven-plugin and that worked for me.
> > Evan
> 
> Maven is black art for me but your answer encouraged me to stare at the command line output of avro-tools again (since I considered it improbable that the maven-plugin can invoke things that the CLI can't). "idl2schemata" does indeed provide a good part of what I want in that it omits the protocol declarations and only transforms the schema parts.
> 
> I guess that Maven would allow me to set up a process where Java classes would be generated from the JSON schemata that idl2schemata extracts from the IDL protocol definition but I’m having trouble figuring out how to configure the POM.
> 
> Just to prove that I’m not totally useless I googled me the following snippet that should generate classes from a schema.
> 
> <executions>
>   <execution>
>     <phase>generate-sources</phase>
>     <goals>
>       <goal>schema</goal>
>     </goals>
>     <configuration>
>       <sourceDirectory>
>         ${project.basedir}/src/main/avro/
>       </sourceDirectory>
>       <outputDirectory>
>         ${project.basedir}/src/main/java/
>       </outputDirectory>
>     </configuration>
>   </execution>
> </executions>
> 
> I reckon that a similar instruction, preceding this one, could initiate the conversion from an IDl to a schema (like the CLI tool does with "idl2schema"). What should the relevant snippet look like?
> 
> 
> Thanks!
> Thomas
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> > On Sat, Jan 30, 2016, 8:52 AM tl <tl...@rat.io> wrote:
> > Hi,
> >
> > I started working with Avro only recently so maybe I missed something but it seems to me that schemas can be defined in JSON as well as in IDL but only from JSON schemas can builder classes be autogenerated. This is a pity since IDLs are much easier to write and read than the JSON representation.
> > I wrote a few IDL schemas, converted them to avpr and tweaked those avpr files to become valid avsc schema files from which I autogenerated the classes. This is a rather convoluted process. I wouldn’t mind so much if I wouldn’t know that I or somebody else will have to update the schemas and classes from time to time. This doesn’t look like a robust workflow.
> >
> > Deleting the IDLs and avpr files and doing updates only in the avsc schemas reduces the workflow to 2 steps but I loose the nice properties of IDLs [0].
> > Using only generic mapping would reduce the workflow by one step too but I’d loose static type checking.
> >
> > I’d love to be able to autogenerate the Java classes from the IDLs directly. Is there a way?
> >
> > Regards,
> > Thomas
> >
> >
> > [0] Regarding IDLs it would be cool if I could forward reference objects in the schema or even write nested schemas but that’s a relatively minor gripe.
> >
> 





< he not busy being born is busy dying >

Re: Autogenerating Java classes from IDL

Posted by Matheus Santana <ed...@gmail.com>.
Thomas,

did you solve your problem already?

You don't need nothing but the Avro Maven plugin for generating Java
classes from Avro IDL. That is exactly what the idl goal does. Your snippet
looks nice, just configure the idl goal instead of the schema goal.

On Mon, Feb 1, 2016 at 11:03 AM, tl <tl...@rat.io> wrote:

>
> > On 31.01.2016, at 01:13, Evan McClain <ae...@gmail.com> wrote:
> >
> > I have used IDLs with the avro-maven-plugin and that worked for me.
> > Evan
>
> Maven is black art for me but your answer encouraged me to stare at the
> command line output of avro-tools again (since I considered it improbable
> that the maven-plugin can invoke things that the CLI can't). "idl2schemata"
> does indeed provide a good part of what I want in that it omits the
> protocol declarations and only transforms the schema parts.
>
> I guess that Maven would allow me to set up a process where Java classes
> would be generated from the JSON schemata that idl2schemata extracts from
> the IDL protocol definition but I’m having trouble figuring out how to
> configure the POM.
>
> Just to prove that I’m not totally useless I googled me the following
> snippet that should generate classes from a schema.
>
> <executions>
>   <execution>
>     <phase>generate-sources</phase>
>     <goals>
>       <goal>schema</goal>
>     </goals>
>     <configuration>
>       <sourceDirectory>
>         ${project.basedir}/src/main/avro/
>       </sourceDirectory>
>       <outputDirectory>
>         ${project.basedir}/src/main/java/
>       </outputDirectory>
>     </configuration>
>   </execution>
> </executions>
>
> I reckon that a similar instruction, preceding this one, could initiate
> the conversion from an IDl to a schema (like the CLI tool does with
> "idl2schema"). What should the relevant snippet look like?
>
>
> Thanks!
> Thomas
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> > On Sat, Jan 30, 2016, 8:52 AM tl <tl...@rat.io> wrote:
> > Hi,
> >
> > I started working with Avro only recently so maybe I missed something
> but it seems to me that schemas can be defined in JSON as well as in IDL
> but only from JSON schemas can builder classes be autogenerated. This is a
> pity since IDLs are much easier to write and read than the JSON
> representation.
> > I wrote a few IDL schemas, converted them to avpr and tweaked those avpr
> files to become valid avsc schema files from which I autogenerated the
> classes. This is a rather convoluted process. I wouldn’t mind so much if I
> wouldn’t know that I or somebody else will have to update the schemas and
> classes from time to time. This doesn’t look like a robust workflow.
> >
> > Deleting the IDLs and avpr files and doing updates only in the avsc
> schemas reduces the workflow to 2 steps but I loose the nice properties of
> IDLs [0].
> > Using only generic mapping would reduce the workflow by one step too but
> I’d loose static type checking.
> >
> > I’d love to be able to autogenerate the Java classes from the IDLs
> directly. Is there a way?
> >
> > Regards,
> > Thomas
> >
> >
> > [0] Regarding IDLs it would be cool if I could forward reference objects
> in the schema or even write nested schemas but that’s a relatively minor
> gripe.
> >
>