You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Bertrand Dechoux <de...@gmail.com> on 2013/05/17 14:46:35 UTC

Transform a single .avdl or .avpr into one or many .avsc

Hi,

I have lots of avro schemas and most of them are about complex objects (ie
nested definition of stuff). The syntax of avro idl is attractive in order
to build something more readable and thus maintainable. However, it looks
like I can't generate any avsc from a avdl (or the avpr generated from the
avdl). I understand what is a protocol and I don't need one but the idl
syntax is really attractive. Is there really no way to use it for that
purpose?

Regards

Bertrand


PS : I remember seeing a discussion about that subject but I can't find it.

Re: Transform a single .avdl or .avpr into one or many .avsc

Posted by Bertrand Dechoux <de...@gmail.com>.
Thanks for the review. I updated the patch accordingly.

Regards

Bertrand


On Tue, May 21, 2013 at 11:44 PM, Bertrand Dechoux <de...@gmail.com>wrote:

> Waiting for a review : https://issues.apache.org/jira/browse/AVRO-1337
>
> Regards
>
> Bertrand
>
>
>
> On Tue, May 21, 2013 at 9:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> After browsing the (nice) API, it seems indeed trivial.
>>
>> The Idl class allows to read/parse the related file.
>>
>> http://avro.apache.org/docs/current/api/java/org/apache/avro/compiler/idl/Idl.html#Idl%28java.io.File%29
>>
>> The Protocol object can then be requested from it.
>>
>> http://avro.apache.org/docs/current/api/java/org/apache/avro/compiler/idl/Idl.html#ProtocolDeclaration%28%29
>>
>> Of course, the types can then be requested from the Protocol itself.
>>
>> http://avro.apache.org/docs/current/api/java/org/apache/avro/Protocol.html#getTypes%28%29
>>
>> And then it is only a matter of serializing them. And actually the API
>> provides even a 'pretty' option.
>>
>> http://avro.apache.org/docs/current/api/java/org/apache/avro/Schema.html#toString%28boolean%29
>>
>> I will definitely look at it (ie with javac). Contributing it as a tool
>> would be nice. But I won't go into the maven land.
>>
>> Regards
>>
>> Bertrand
>>
>>
>> On Fri, May 17, 2013 at 9:11 PM, Doug Cutting <cu...@apache.org> wrote:
>>
>>> There's not a tool that does this currently.  Note however that
>>> existing tools will generate Java classes for each type in an IDL
>>> file, so if you're only using Java then you might not need a .avsc
>>> file for each type in the IDL.
>>>
>>> It would not be hard to add a tool (or an option to an existing tool)
>>> that wrote a .avsc file for each type in an .avdl file.  One could
>>> also add Maven support for this.  If this is of interest, please file
>>> an issue in Jira.
>>>
>>> https://issues.apache.org/jira/browse/AVRO
>>>
>>> If you're willing and able, please provide an implementation.
>>> Otherwise hopefully someone else will help out.
>>>
>>> Cheers,
>>>
>>> Doug
>>>
>>> On Fri, May 17, 2013 at 7:10 AM, Jeremy Kahn <tr...@trochee.net>
>>> wrote:
>>> > The "types" field in a protocol (.avro) may get you what you need. The
>>> > corresponding schema objects should be able to render to well-formed
>>> avsc
>>> > objects.
>>> >
>>> > On May 17, 2013 5:47 AM, "Bertrand Dechoux" <de...@gmail.com>
>>> wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> I have lots of avro schemas and most of them are about complex
>>> objects (ie
>>> >> nested definition of stuff). The syntax of avro idl is attractive in
>>> order
>>> >> to build something more readable and thus maintainable. However, it
>>> looks
>>> >> like I can't generate any avsc from a avdl (or the avpr generated
>>> from the
>>> >> avdl). I understand what is a protocol and I don't need one but the
>>> idl
>>> >> syntax is really attractive. Is there really no way to use it for that
>>> >> purpose?
>>> >>
>>> >> Regards
>>> >>
>>> >> Bertrand
>>> >>
>>> >>
>>> >> PS : I remember seeing a discussion about that subject but I can't
>>> find
>>> >> it.
>>>
>>
>>
>>
>> --
>> Bertrand Dechoux
>>
>
>
>
> --
> Bertrand Dechoux
>



-- 
Bertrand Dechoux

Re: Transform a single .avdl or .avpr into one or many .avsc

Posted by Bertrand Dechoux <de...@gmail.com>.
Waiting for a review : https://issues.apache.org/jira/browse/AVRO-1337

Regards

Bertrand



On Tue, May 21, 2013 at 9:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:

> After browsing the (nice) API, it seems indeed trivial.
>
> The Idl class allows to read/parse the related file.
>
> http://avro.apache.org/docs/current/api/java/org/apache/avro/compiler/idl/Idl.html#Idl%28java.io.File%29
>
> The Protocol object can then be requested from it.
>
> http://avro.apache.org/docs/current/api/java/org/apache/avro/compiler/idl/Idl.html#ProtocolDeclaration%28%29
>
> Of course, the types can then be requested from the Protocol itself.
>
> http://avro.apache.org/docs/current/api/java/org/apache/avro/Protocol.html#getTypes%28%29
>
> And then it is only a matter of serializing them. And actually the API
> provides even a 'pretty' option.
>
> http://avro.apache.org/docs/current/api/java/org/apache/avro/Schema.html#toString%28boolean%29
>
> I will definitely look at it (ie with javac). Contributing it as a tool
> would be nice. But I won't go into the maven land.
>
> Regards
>
> Bertrand
>
>
> On Fri, May 17, 2013 at 9:11 PM, Doug Cutting <cu...@apache.org> wrote:
>
>> There's not a tool that does this currently.  Note however that
>> existing tools will generate Java classes for each type in an IDL
>> file, so if you're only using Java then you might not need a .avsc
>> file for each type in the IDL.
>>
>> It would not be hard to add a tool (or an option to an existing tool)
>> that wrote a .avsc file for each type in an .avdl file.  One could
>> also add Maven support for this.  If this is of interest, please file
>> an issue in Jira.
>>
>> https://issues.apache.org/jira/browse/AVRO
>>
>> If you're willing and able, please provide an implementation.
>> Otherwise hopefully someone else will help out.
>>
>> Cheers,
>>
>> Doug
>>
>> On Fri, May 17, 2013 at 7:10 AM, Jeremy Kahn <tr...@trochee.net> wrote:
>> > The "types" field in a protocol (.avro) may get you what you need. The
>> > corresponding schema objects should be able to render to well-formed
>> avsc
>> > objects.
>> >
>> > On May 17, 2013 5:47 AM, "Bertrand Dechoux" <de...@gmail.com> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I have lots of avro schemas and most of them are about complex objects
>> (ie
>> >> nested definition of stuff). The syntax of avro idl is attractive in
>> order
>> >> to build something more readable and thus maintainable. However, it
>> looks
>> >> like I can't generate any avsc from a avdl (or the avpr generated from
>> the
>> >> avdl). I understand what is a protocol and I don't need one but the idl
>> >> syntax is really attractive. Is there really no way to use it for that
>> >> purpose?
>> >>
>> >> Regards
>> >>
>> >> Bertrand
>> >>
>> >>
>> >> PS : I remember seeing a discussion about that subject but I can't find
>> >> it.
>>
>
>
>
> --
> Bertrand Dechoux
>



-- 
Bertrand Dechoux

Re: Transform a single .avdl or .avpr into one or many .avsc

Posted by Bertrand Dechoux <de...@gmail.com>.
After browsing the (nice) API, it seems indeed trivial.

The Idl class allows to read/parse the related file.
http://avro.apache.org/docs/current/api/java/org/apache/avro/compiler/idl/Idl.html#Idl%28java.io.File%29

The Protocol object can then be requested from it.
http://avro.apache.org/docs/current/api/java/org/apache/avro/compiler/idl/Idl.html#ProtocolDeclaration%28%29

Of course, the types can then be requested from the Protocol itself.
http://avro.apache.org/docs/current/api/java/org/apache/avro/Protocol.html#getTypes%28%29

And then it is only a matter of serializing them. And actually the API
provides even a 'pretty' option.
http://avro.apache.org/docs/current/api/java/org/apache/avro/Schema.html#toString%28boolean%29

I will definitely look at it (ie with javac). Contributing it as a tool
would be nice. But I won't go into the maven land.

Regards

Bertrand


On Fri, May 17, 2013 at 9:11 PM, Doug Cutting <cu...@apache.org> wrote:

> There's not a tool that does this currently.  Note however that
> existing tools will generate Java classes for each type in an IDL
> file, so if you're only using Java then you might not need a .avsc
> file for each type in the IDL.
>
> It would not be hard to add a tool (or an option to an existing tool)
> that wrote a .avsc file for each type in an .avdl file.  One could
> also add Maven support for this.  If this is of interest, please file
> an issue in Jira.
>
> https://issues.apache.org/jira/browse/AVRO
>
> If you're willing and able, please provide an implementation.
> Otherwise hopefully someone else will help out.
>
> Cheers,
>
> Doug
>
> On Fri, May 17, 2013 at 7:10 AM, Jeremy Kahn <tr...@trochee.net> wrote:
> > The "types" field in a protocol (.avro) may get you what you need. The
> > corresponding schema objects should be able to render to well-formed avsc
> > objects.
> >
> > On May 17, 2013 5:47 AM, "Bertrand Dechoux" <de...@gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> I have lots of avro schemas and most of them are about complex objects
> (ie
> >> nested definition of stuff). The syntax of avro idl is attractive in
> order
> >> to build something more readable and thus maintainable. However, it
> looks
> >> like I can't generate any avsc from a avdl (or the avpr generated from
> the
> >> avdl). I understand what is a protocol and I don't need one but the idl
> >> syntax is really attractive. Is there really no way to use it for that
> >> purpose?
> >>
> >> Regards
> >>
> >> Bertrand
> >>
> >>
> >> PS : I remember seeing a discussion about that subject but I can't find
> >> it.
>



-- 
Bertrand Dechoux

Re: Transform a single .avdl or .avpr into one or many .avsc

Posted by Doug Cutting <cu...@apache.org>.
There's not a tool that does this currently.  Note however that
existing tools will generate Java classes for each type in an IDL
file, so if you're only using Java then you might not need a .avsc
file for each type in the IDL.

It would not be hard to add a tool (or an option to an existing tool)
that wrote a .avsc file for each type in an .avdl file.  One could
also add Maven support for this.  If this is of interest, please file
an issue in Jira.

https://issues.apache.org/jira/browse/AVRO

If you're willing and able, please provide an implementation.
Otherwise hopefully someone else will help out.

Cheers,

Doug

On Fri, May 17, 2013 at 7:10 AM, Jeremy Kahn <tr...@trochee.net> wrote:
> The "types" field in a protocol (.avro) may get you what you need. The
> corresponding schema objects should be able to render to well-formed avsc
> objects.
>
> On May 17, 2013 5:47 AM, "Bertrand Dechoux" <de...@gmail.com> wrote:
>>
>> Hi,
>>
>> I have lots of avro schemas and most of them are about complex objects (ie
>> nested definition of stuff). The syntax of avro idl is attractive in order
>> to build something more readable and thus maintainable. However, it looks
>> like I can't generate any avsc from a avdl (or the avpr generated from the
>> avdl). I understand what is a protocol and I don't need one but the idl
>> syntax is really attractive. Is there really no way to use it for that
>> purpose?
>>
>> Regards
>>
>> Bertrand
>>
>>
>> PS : I remember seeing a discussion about that subject but I can't find
>> it.

Re: Transform a single .avdl or .avpr into one or many .avsc

Posted by Jeremy Kahn <tr...@trochee.net>.
The "types" field in a protocol (.avro) may get you what you need. The
corresponding schema objects should be able to render to well-formed avsc
objects.
On May 17, 2013 5:47 AM, "Bertrand Dechoux" <de...@gmail.com> wrote:

> Hi,
>
> I have lots of avro schemas and most of them are about complex objects (ie
> nested definition of stuff). The syntax of avro idl is attractive in order
> to build something more readable and thus maintainable. However, it looks
> like I can't generate any avsc from a avdl (or the avpr generated from the
> avdl). I understand what is a protocol and I don't need one but the idl
> syntax is really attractive. Is there really no way to use it for that
> purpose?
>
> Regards
>
> Bertrand
>
>
> PS : I remember seeing a discussion about that subject but I can't find it.
>