You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Felix GV <fe...@mate1inc.com> on 2013/05/10 01:39:33 UTC
Best practices for java enums...?
Hello,
I'm currently writing an avro schema which includes an enum field that I
already have as a java enum in my application.
At first, I named the avro field with the same fully qualified name
(package name dot enum name) as my existing java enum. I then ran the avro
compiler and found that it overwrote my existing java enum with an
avro-generated enum.
I find this slightly annoying because my java enum had comments documenting
the purpose of each enum value, and the avro-generated enum doesn't have
this.
I see two or three potential solutions:
1. Accepting to replace my current enum with the avro-generated one in
my code base, which I feel I cannot document properly (since I have access
to just one doc attribute for the whole enum, instead of per symbol). On a
side note, I haven't found any way to have a multi-line doc attribute in an
avro schema, so that makes things slightly more annoying still. I wouldn't
mind settling on using the avro-generated enums without documentation per
symbol if at least I could have one big doc/comment that documents all
symbols at once, but since it seems the doc attribute must be a one-liner,
this is starting to be a little too messy for my taste...
2. Maintaining two separate enums: my manually written (and documented)
enum as well as the avro-generated enum. For now, I think this is what I'm
going to do, because those enums have little chances of changing anyway,
but from a maintenance standpoint, it seems pretty horrendous...
3. I guess there's a third way, which would involve creating a script
that backs up my enums, compiles all my schemas, and then restores my
backed up enums, but this also seems ultra messy :( ... I haven't tested if
it'd work (since the manually written enum is missing the $SCHEMA field),
but I guess it would...
Am I being OCD about this? or is this a concern that others have bumped
into? How do you guys deal with this? Did I miss anything in the way avro
works?
P.S.: I've seen that reflect mappings may be able to work with arbitrary
java enums, but since they seemed discouraged for performance reasons, I
haven't digged much in this direction. I'd like to keep using .avsc files
if possible, but if there's a better way, I can certainly try it.
P.P.S.: We're currently using avro 1.6.1, but if the latest version
provides a nice way of handling my use case, then I guess I could get us to
upgrade...
Thanks a lot :) !
--
Felix
Re: Best practices for java enums...?
Posted by Felix GV <fe...@mate1inc.com>.
Thanks for the feedback guys :) !
I've created AVRO-1329 <https://issues.apache.org/jira/browse/AVRO-1329>.
I'll play around with IDLs, as Doug suggested, and see if that can suit our
documentation and maintenance needs better for the time being.
Thanks again :) !
--
Felix
On Fri, May 10, 2013 at 2:47 PM, Jeremy Kahn <tr...@trochee.net> wrote:
> +1 to this suggestion. Docs on individual symbols in IDL enums would be
> convenient.
>
> Felix, will you file that ticket please? I can promise to upvote.
>
>
> On Fri, May 10, 2013 at 10:55 AM, Doug Cutting <cu...@apache.org> wrote:
>
>> If you use Avro IDL then you can more easily maintain long, multi-line
>> comments for your enum.
>>
>> http://avro.apache.org/docs/current/idl.html#minutiae_comments
>>
>> IDL files (.avdl) can be compiled to Schemas (.avsc) by Maven, which
>> are then compiled to Java.
>>
>> Documentation per enum symbol is not currently supported, but would
>> not be difficult to add. Please file an issue in Jira if you'd like
>> to see this. For compatibility, in Json, this would probably appear
>> as a parallel array of documentation strings, e.g., something like:
>>
>> ("name": "Foo", "type":"enum", "doc":"an enum", "symbols":["X","Y"],
>> "symbols-doc":["X is X", "Y is Y"]}
>>
>> Doug
>>
>> On Thu, May 9, 2013 at 4:39 PM, Felix GV <fe...@mate1inc.com> wrote:
>> > Hello,
>> >
>> > I'm currently writing an avro schema which includes an enum field that I
>> > already have as a java enum in my application.
>> >
>> > At first, I named the avro field with the same fully qualified name
>> (package
>> > name dot enum name) as my existing java enum. I then ran the avro
>> compiler
>> > and found that it overwrote my existing java enum with an avro-generated
>> > enum.
>> >
>> > I find this slightly annoying because my java enum had comments
>> documenting
>> > the purpose of each enum value, and the avro-generated enum doesn't have
>> > this.
>> >
>> > I see two or three potential solutions:
>> >
>> > Accepting to replace my current enum with the avro-generated one in my
>> code
>> > base, which I feel I cannot document properly (since I have access to
>> just
>> > one doc attribute for the whole enum, instead of per symbol). On a side
>> > note, I haven't found any way to have a multi-line doc attribute in an
>> avro
>> > schema, so that makes things slightly more annoying still. I wouldn't
>> mind
>> > settling on using the avro-generated enums without documentation per
>> symbol
>> > if at least I could have one big doc/comment that documents all symbols
>> at
>> > once, but since it seems the doc attribute must be a one-liner, this is
>> > starting to be a little too messy for my taste...
>> > Maintaining two separate enums: my manually written (and documented)
>> enum as
>> > well as the avro-generated enum. For now, I think this is what I'm
>> going to
>> > do, because those enums have little chances of changing anyway, but
>> from a
>> > maintenance standpoint, it seems pretty horrendous...
>> > I guess there's a third way, which would involve creating a script that
>> > backs up my enums, compiles all my schemas, and then restores my backed
>> up
>> > enums, but this also seems ultra messy :( ... I haven't tested if it'd
>> work
>> > (since the manually written enum is missing the $SCHEMA field), but I
>> guess
>> > it would...
>> >
>> > Am I being OCD about this? or is this a concern that others have bumped
>> > into? How do you guys deal with this? Did I miss anything in the way
>> avro
>> > works?
>> >
>> > P.S.: I've seen that reflect mappings may be able to work with arbitrary
>> > java enums, but since they seemed discouraged for performance reasons, I
>> > haven't digged much in this direction. I'd like to keep using .avsc
>> files if
>> > possible, but if there's a better way, I can certainly try it.
>> >
>> > P.P.S.: We're currently using avro 1.6.1, but if the latest version
>> provides
>> > a nice way of handling my use case, then I guess I could get us to
>> > upgrade...
>> >
>> > Thanks a lot :) !
>> >
>> > --
>> > Felix
>>
>
>
Re: Best practices for java enums...?
Posted by Jeremy Kahn <tr...@trochee.net>.
+1 to this suggestion. Docs on individual symbols in IDL enums would be
convenient.
Felix, will you file that ticket please? I can promise to upvote.
On Fri, May 10, 2013 at 10:55 AM, Doug Cutting <cu...@apache.org> wrote:
> If you use Avro IDL then you can more easily maintain long, multi-line
> comments for your enum.
>
> http://avro.apache.org/docs/current/idl.html#minutiae_comments
>
> IDL files (.avdl) can be compiled to Schemas (.avsc) by Maven, which
> are then compiled to Java.
>
> Documentation per enum symbol is not currently supported, but would
> not be difficult to add. Please file an issue in Jira if you'd like
> to see this. For compatibility, in Json, this would probably appear
> as a parallel array of documentation strings, e.g., something like:
>
> ("name": "Foo", "type":"enum", "doc":"an enum", "symbols":["X","Y"],
> "symbols-doc":["X is X", "Y is Y"]}
>
> Doug
>
> On Thu, May 9, 2013 at 4:39 PM, Felix GV <fe...@mate1inc.com> wrote:
> > Hello,
> >
> > I'm currently writing an avro schema which includes an enum field that I
> > already have as a java enum in my application.
> >
> > At first, I named the avro field with the same fully qualified name
> (package
> > name dot enum name) as my existing java enum. I then ran the avro
> compiler
> > and found that it overwrote my existing java enum with an avro-generated
> > enum.
> >
> > I find this slightly annoying because my java enum had comments
> documenting
> > the purpose of each enum value, and the avro-generated enum doesn't have
> > this.
> >
> > I see two or three potential solutions:
> >
> > Accepting to replace my current enum with the avro-generated one in my
> code
> > base, which I feel I cannot document properly (since I have access to
> just
> > one doc attribute for the whole enum, instead of per symbol). On a side
> > note, I haven't found any way to have a multi-line doc attribute in an
> avro
> > schema, so that makes things slightly more annoying still. I wouldn't
> mind
> > settling on using the avro-generated enums without documentation per
> symbol
> > if at least I could have one big doc/comment that documents all symbols
> at
> > once, but since it seems the doc attribute must be a one-liner, this is
> > starting to be a little too messy for my taste...
> > Maintaining two separate enums: my manually written (and documented)
> enum as
> > well as the avro-generated enum. For now, I think this is what I'm going
> to
> > do, because those enums have little chances of changing anyway, but from
> a
> > maintenance standpoint, it seems pretty horrendous...
> > I guess there's a third way, which would involve creating a script that
> > backs up my enums, compiles all my schemas, and then restores my backed
> up
> > enums, but this also seems ultra messy :( ... I haven't tested if it'd
> work
> > (since the manually written enum is missing the $SCHEMA field), but I
> guess
> > it would...
> >
> > Am I being OCD about this? or is this a concern that others have bumped
> > into? How do you guys deal with this? Did I miss anything in the way avro
> > works?
> >
> > P.S.: I've seen that reflect mappings may be able to work with arbitrary
> > java enums, but since they seemed discouraged for performance reasons, I
> > haven't digged much in this direction. I'd like to keep using .avsc
> files if
> > possible, but if there's a better way, I can certainly try it.
> >
> > P.P.S.: We're currently using avro 1.6.1, but if the latest version
> provides
> > a nice way of handling my use case, then I guess I could get us to
> > upgrade...
> >
> > Thanks a lot :) !
> >
> > --
> > Felix
>
Re: Best practices for java enums...?
Posted by Doug Cutting <cu...@apache.org>.
If you use Avro IDL then you can more easily maintain long, multi-line
comments for your enum.
http://avro.apache.org/docs/current/idl.html#minutiae_comments
IDL files (.avdl) can be compiled to Schemas (.avsc) by Maven, which
are then compiled to Java.
Documentation per enum symbol is not currently supported, but would
not be difficult to add. Please file an issue in Jira if you'd like
to see this. For compatibility, in Json, this would probably appear
as a parallel array of documentation strings, e.g., something like:
("name": "Foo", "type":"enum", "doc":"an enum", "symbols":["X","Y"],
"symbols-doc":["X is X", "Y is Y"]}
Doug
On Thu, May 9, 2013 at 4:39 PM, Felix GV <fe...@mate1inc.com> wrote:
> Hello,
>
> I'm currently writing an avro schema which includes an enum field that I
> already have as a java enum in my application.
>
> At first, I named the avro field with the same fully qualified name (package
> name dot enum name) as my existing java enum. I then ran the avro compiler
> and found that it overwrote my existing java enum with an avro-generated
> enum.
>
> I find this slightly annoying because my java enum had comments documenting
> the purpose of each enum value, and the avro-generated enum doesn't have
> this.
>
> I see two or three potential solutions:
>
> Accepting to replace my current enum with the avro-generated one in my code
> base, which I feel I cannot document properly (since I have access to just
> one doc attribute for the whole enum, instead of per symbol). On a side
> note, I haven't found any way to have a multi-line doc attribute in an avro
> schema, so that makes things slightly more annoying still. I wouldn't mind
> settling on using the avro-generated enums without documentation per symbol
> if at least I could have one big doc/comment that documents all symbols at
> once, but since it seems the doc attribute must be a one-liner, this is
> starting to be a little too messy for my taste...
> Maintaining two separate enums: my manually written (and documented) enum as
> well as the avro-generated enum. For now, I think this is what I'm going to
> do, because those enums have little chances of changing anyway, but from a
> maintenance standpoint, it seems pretty horrendous...
> I guess there's a third way, which would involve creating a script that
> backs up my enums, compiles all my schemas, and then restores my backed up
> enums, but this also seems ultra messy :( ... I haven't tested if it'd work
> (since the manually written enum is missing the $SCHEMA field), but I guess
> it would...
>
> Am I being OCD about this? or is this a concern that others have bumped
> into? How do you guys deal with this? Did I miss anything in the way avro
> works?
>
> P.S.: I've seen that reflect mappings may be able to work with arbitrary
> java enums, but since they seemed discouraged for performance reasons, I
> haven't digged much in this direction. I'd like to keep using .avsc files if
> possible, but if there's a better way, I can certainly try it.
>
> P.P.S.: We're currently using avro 1.6.1, but if the latest version provides
> a nice way of handling my use case, then I guess I could get us to
> upgrade...
>
> Thanks a lot :) !
>
> --
> Felix
Re: Best practices for java enums...?
Posted by Felix GV <fe...@mate1inc.com>.
Hi Scott,
How would you envision this working with the avro compiler? Would it be
akin to the capabilites added by
AVRO-1188<https://issues.apache.org/jira/browse/AVRO-1188> where
the existing java enums would be referenced in the importedFiles directory
passed in parameter to the compiler?
Should I file another JIRA for this...?
--
Felix
On Mon, May 13, 2013 at 5:38 PM, Scott Carey <sc...@apache.org> wrote:
> It would be nice to be able to reference an existing class when using the
> specific compiler.
>
> If you have an existing "com.mycompany.Foo" enum (or SpecificRecord, or
> Fixed type), then provide the specific compiler with the type prior to
> parsing the schema, it could accept a reference:
>
> {"type":"record", "name":"com.mycompany.Rec", "fields": [
> {"name":"fooField", "type":"com.mycompany.Foo"}
> ]}
>
> Ordinarily, this would fail to compile, but given a reference to an
> existing compatible type, such as an enum, it could work.
>
> -Scott
>
> On 5/9/13 4:39 PM, "Felix GV" <fe...@mate1inc.com> wrote:
>
> Hello,
>
> I'm currently writing an avro schema which includes an enum field that I
> already have as a java enum in my application.
>
> At first, I named the avro field with the same fully qualified name
> (package name dot enum name) as my existing java enum. I then ran the avro
> compiler and found that it overwrote my existing java enum with an
> avro-generated enum.
>
> I find this slightly annoying because my java enum had comments
> documenting the purpose of each enum value, and the avro-generated enum
> doesn't have this.
>
> I see two or three potential solutions:
>
> 1. Accepting to replace my current enum with the avro-generated one in
> my code base, which I feel I cannot document properly (since I have access
> to just one doc attribute for the whole enum, instead of per symbol). On a
> side note, I haven't found any way to have a multi-line doc attribute in an
> avro schema, so that makes things slightly more annoying still. I wouldn't
> mind settling on using the avro-generated enums without documentation per
> symbol if at least I could have one big doc/comment that documents all
> symbols at once, but since it seems the doc attribute must be a one-liner,
> this is starting to be a little too messy for my taste...
> 2. Maintaining two separate enums: my manually written (and
> documented) enum as well as the avro-generated enum. For now, I think this
> is what I'm going to do, because those enums have little chances of
> changing anyway, but from a maintenance standpoint, it seems pretty
> horrendous...
> 3. I guess there's a third way, which would involve creating a script
> that backs up my enums, compiles all my schemas, and then restores my
> backed up enums, but this also seems ultra messy :( ... I haven't tested if
> it'd work (since the manually written enum is missing the $SCHEMA field),
> but I guess it would...
>
> Am I being OCD about this? or is this a concern that others have bumped
> into? How do you guys deal with this? Did I miss anything in the way avro
> works?
>
> P.S.: I've seen that reflect mappings may be able to work with arbitrary
> java enums, but since they seemed discouraged for performance reasons, I
> haven't digged much in this direction. I'd like to keep using .avsc files
> if possible, but if there's a better way, I can certainly try it.
>
> P.P.S.: We're currently using avro 1.6.1, but if the latest version
> provides a nice way of handling my use case, then I guess I could get us to
> upgrade...
>
> Thanks a lot :) !
>
> --
> Felix
>
>
Re: Best practices for java enums...?
Posted by Scott Carey <sc...@apache.org>.
It would be nice to be able to reference an existing class when using the
specific compiler.
If you have an existing "com.mycompany.Foo" enum (or SpecificRecord, or
Fixed type), then provide the specific compiler with the type prior to
parsing the schema, it could accept a reference:
{"type":"record", "name":"com.mycompany.Rec", "fields": [
{"name":"fooField", "type":"com.mycompany.Foo"}
]}
Ordinarily, this would fail to compile, but given a reference to an existing
compatible type, such as an enum, it could work.
-Scott
On 5/9/13 4:39 PM, "Felix GV" <fe...@mate1inc.com> wrote:
> Hello,
>
> I'm currently writing an avro schema which includes an enum field that I
> already have as a java enum in my application.
>
> At first, I named the avro field with the same fully qualified name (package
> name dot enum name) as my existing java enum. I then ran the avro compiler and
> found that it overwrote my existing java enum with an avro-generated enum.
>
> I find this slightly annoying because my java enum had comments documenting
> the purpose of each enum value, and the avro-generated enum doesn't have this.
>
> I see two or three potential solutions:
> 1. Accepting to replace my current enum with the avro-generated one in my code
> base, which I feel I cannot document properly (since I have access to just one
> doc attribute for the whole enum, instead of per symbol). On a side note, I
> haven't found any way to have a multi-line doc attribute in an avro schema, so
> that makes things slightly more annoying still. I wouldn't mind settling on
> using the avro-generated enums without documentation per symbol if at least I
> could have one big doc/comment that documents all symbols at once, but since
> it seems the doc attribute must be a one-liner, this is starting to be a
> little too messy for my taste...
> 2. Maintaining two separate enums: my manually written (and documented) enum
> as well as the avro-generated enum. For now, I think this is what I'm going to
> do, because those enums have little chances of changing anyway, but from a
> maintenance standpoint, it seems pretty horrendous...
> 3. I guess there's a third way, which would involve creating a script that
> backs up my enums, compiles all my schemas, and then restores my backed up
> enums, but this also seems ultra messy :( ... I haven't tested if it'd work
> (since the manually written enum is missing the $SCHEMA field), but I guess it
> would...
> Am I being OCD about this? or is this a concern that others have bumped into?
> How do you guys deal with this? Did I miss anything in the way avro works?
>
> P.S.: I've seen that reflect mappings may be able to work with arbitrary java
> enums, but since they seemed discouraged for performance reasons, I haven't
> digged much in this direction. I'd like to keep using .avsc files if possible,
> but if there's a better way, I can certainly try it.
>
> P.P.S.: We're currently using avro 1.6.1, but if the latest version provides a
> nice way of handling my use case, then I guess I could get us to upgrade...
>
> Thanks a lot :) !
>
> --
> Felix