You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Sébastien Launay (JIRA)" <ji...@apache.org> on 2012/11/29 02:44:58 UTC
[jira] [Created] (AVRO-1212) Protocol schema generated from
reflection does not support @Union with collections
Sébastien Launay created AVRO-1212:
--------------------------------------
Summary: Protocol schema generated from reflection does not support @Union with collections
Key: AVRO-1212
URL: https://issues.apache.org/jira/browse/AVRO-1212
Project: Avro
Issue Type: Bug
Components: java
Affects Versions: 1.7.2
Reporter: Sébastien Launay
An interface using {{@Union}} for collections (Map, List, Java array) like this one:
{code:java}
public interface P0 {
@Union({String.class,Integer.class}) List<Object> foo(@Union({Integer.class,Long.class}) List<Number> l);
}
{code}
will produce the following schema where the lists have been erased by the component unions:
{code:javascript}
{
"protocol" : "P0",
"namespace" : "",
"types" : [ {
"type" : "record",
"name" : "Number",
"namespace" : "java.lang",
"fields" : [ ]
} ],
"messages" : {
"foo" : {
"request" : [ {
"name" : "l",
"type" : [ "int", "long" ]
} ],
"response" : [ "string", "int" ]
}
}
}
{code}
instead of:
{code:javascript}
{
"protocol" : "P0X",
"namespace" : "",
"types" : [ {
"type" : "record",
"name" : "Object",
"namespace" : "java.lang",
"fields" : [ ]
}, {
"type" : "record",
"name" : "Number",
"namespace" : "java.lang",
"fields" : [ ]
} ],
"messages" : {
"foo" : {
"request" : [ {
"name" : "l",
"type" : {
"type" : "array",
"items" : [ "int", "long" ],
"java-class" : "java.util.List"
}
} ],
"response" : {
"type" : "array",
"items" : [ "string", "int" ],
"java-class" : "java.util.List"
}
}
}
}
{code}
This leads to exceptions when writing a response like List<R1|R2>:
{noformat}
org.apache.avro.UnresolvedUnionException: Not in union [{"type":"record","name":"R1"},{"type":"record","name":"R2"}]: [R1@19f03d7]
at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:542) ~[avro-1.7.2.jar:1.7.2]
at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:137) ~[avro-1.7.2.jar:1.7.2]
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:70) ~[avro-1.7.2.jar:1.7.2]
at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104) ~[avro-1.7.2.jar:1.7.2]
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57) ~[avro-1.7.2.jar:1.7.2]
at org.apache.avro.ipc.generic.GenericResponder.writeResponse(GenericResponder.java:58) ~[avro-ipc-1.7.2.jar:1.7.2]
at org.apache.avro.ipc.Responder.respond(Responder.java:164) [avro-ipc-1.7.2.jar:1.7.2]
at org.apache.avro.ipc.Responder.respond(Responder.java:99) [avro-ipc-1.7.2.jar:1.7.2]
at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48) [avro-ipc-1.7.2.jar:1.7.2]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api-2.5-20081211.jar:na]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [servlet-api-2.5-20081211.jar:na]
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.Server.handle(Server.java:322) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) [jetty-6.1.26.jar:6.1.26]
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) [jetty-util-6.1.26.jar:6.1.26]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1212) Protocol schema generated from
reflection does not support @Union with collections
Posted by "Sébastien Launay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sébastien Launay updated AVRO-1212:
-----------------------------------
Status: Patch Available (was: Open)
Patch fixing the issue by:
- converting annotated List and Java array to ARRAY type with union component type
- converting annotated Map to MAP type with union value type
- detecting double \@Union to prevent issues
- test cases to reproduce the issue and detect future regressions
Note that:
- it is not possible to define a union of Map or List with that approach
- annotating nested List/Map/Array with \@Union will only be used for the leaf components
- @Nullable on the other hand will be applied to the collection (to keep things backward compatible)
Patch is attached but can be found here:
https://github.com/slaunay/avro/commits/fix/AVRO-1212-union-collections
> Protocol schema generated from reflection does not support @Union with collections
> ----------------------------------------------------------------------------------
>
> Key: AVRO-1212
> URL: https://issues.apache.org/jira/browse/AVRO-1212
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.7.2
> Reporter: Sébastien Launay
> Attachments: AVRO-1212-union-with-collections-2012-11-28.patch
>
>
> An interface using {{@Union}} for collections (Map, List, Java array) like this one:
> {code:java}
> public interface P0 {
> @Union({String.class,Integer.class}) List<Object> foo(@Union({Integer.class,Long.class}) List<Number> l);
> }
> {code}
> will produce the following schema where the lists have been erased by the component unions:
> {code:javascript}
> {
> "protocol" : "P0",
> "namespace" : "",
> "types" : [ {
> "type" : "record",
> "name" : "Number",
> "namespace" : "java.lang",
> "fields" : [ ]
> } ],
> "messages" : {
> "foo" : {
> "request" : [ {
> "name" : "l",
> "type" : [ "int", "long" ]
> } ],
> "response" : [ "string", "int" ]
> }
> }
> }
> {code}
> instead of:
> {code:javascript}
> {
> "protocol" : "P0X",
> "namespace" : "",
> "types" : [ {
> "type" : "record",
> "name" : "Object",
> "namespace" : "java.lang",
> "fields" : [ ]
> }, {
> "type" : "record",
> "name" : "Number",
> "namespace" : "java.lang",
> "fields" : [ ]
> } ],
> "messages" : {
> "foo" : {
> "request" : [ {
> "name" : "l",
> "type" : {
> "type" : "array",
> "items" : [ "int", "long" ],
> "java-class" : "java.util.List"
> }
> } ],
> "response" : {
> "type" : "array",
> "items" : [ "string", "int" ],
> "java-class" : "java.util.List"
> }
> }
> }
> }
> {code}
> This leads to exceptions when writing a response like List<R1|R2>:
> {noformat}
> org.apache.avro.UnresolvedUnionException: Not in union [{"type":"record","name":"R1"},{"type":"record","name":"R2"}]: [R1@19f03d7]
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:542) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:137) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:70) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.generic.GenericResponder.writeResponse(GenericResponder.java:58) ~[avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.Responder.respond(Responder.java:164) [avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.Responder.respond(Responder.java:99) [avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48) [avro-ipc-1.7.2.jar:1.7.2]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api-2.5-20081211.jar:na]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [servlet-api-2.5-20081211.jar:na]
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.Server.handle(Server.java:322) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) [jetty-util-6.1.26.jar:6.1.26]
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1212) Protocol schema generated from
reflection does not support @Union with collections
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506838#comment-13506838 ]
Doug Cutting commented on AVRO-1212:
------------------------------------
I worry that this approach is confusing. When I look at the example it looks like an error: you've got code that returns an array that's declared as a union of primitives.
A simpler more direct approach might be to add a new annotation that, in effect, casts a Java type to an arbitrary Avro Schema.
This might look like:
{code}
public interface P0 {
@AvroSchema("{\"type\":\"array\",\"items\":[\"long\",\"int\"]")
List<Object> foo(@Avro([\"long\",\"int\"]") List<Number> l);
}
{code}
Or we might define more specific annotations, like ArrayOfUnion and MapOfUnion so you could do something like:
{code}
public interface P0 {
@ArrayOfUnion({Integer.class, Long.class})
List<Object> foo(@Union({Integer.class, Long.class}) List<Number> l);
}
{code}
This is more limited in its expressive power, but does away with those ugly backslashes and gets some compile-type checking.
Either of those seem much clearer to me.
> Protocol schema generated from reflection does not support @Union with collections
> ----------------------------------------------------------------------------------
>
> Key: AVRO-1212
> URL: https://issues.apache.org/jira/browse/AVRO-1212
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.7.2
> Reporter: Sébastien Launay
> Attachments: AVRO-1212-union-with-collections-2012-11-28.patch
>
>
> An interface using {{@Union}} for collections (Map, List, Java array) like this one:
> {code:java}
> public interface P0 {
> @Union({String.class,Integer.class}) List<Object> foo(@Union({Integer.class,Long.class}) List<Number> l);
> }
> {code}
> will produce the following schema where the lists have been erased by the component unions:
> {code:javascript}
> {
> "protocol" : "P0",
> "namespace" : "",
> "types" : [ {
> "type" : "record",
> "name" : "Number",
> "namespace" : "java.lang",
> "fields" : [ ]
> } ],
> "messages" : {
> "foo" : {
> "request" : [ {
> "name" : "l",
> "type" : [ "int", "long" ]
> } ],
> "response" : [ "string", "int" ]
> }
> }
> }
> {code}
> instead of:
> {code:javascript}
> {
> "protocol" : "P0X",
> "namespace" : "",
> "types" : [ {
> "type" : "record",
> "name" : "Object",
> "namespace" : "java.lang",
> "fields" : [ ]
> }, {
> "type" : "record",
> "name" : "Number",
> "namespace" : "java.lang",
> "fields" : [ ]
> } ],
> "messages" : {
> "foo" : {
> "request" : [ {
> "name" : "l",
> "type" : {
> "type" : "array",
> "items" : [ "int", "long" ],
> "java-class" : "java.util.List"
> }
> } ],
> "response" : {
> "type" : "array",
> "items" : [ "string", "int" ],
> "java-class" : "java.util.List"
> }
> }
> }
> }
> {code}
> This leads to exceptions when writing a response like List<R1|R2>:
> {noformat}
> org.apache.avro.UnresolvedUnionException: Not in union [{"type":"record","name":"R1"},{"type":"record","name":"R2"}]: [R1@19f03d7]
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:542) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:137) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:70) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.generic.GenericResponder.writeResponse(GenericResponder.java:58) ~[avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.Responder.respond(Responder.java:164) [avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.Responder.respond(Responder.java:99) [avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48) [avro-ipc-1.7.2.jar:1.7.2]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api-2.5-20081211.jar:na]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [servlet-api-2.5-20081211.jar:na]
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.Server.handle(Server.java:322) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) [jetty-util-6.1.26.jar:6.1.26]
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1212) Protocol schema generated from
reflection does not support @Union with collections
Posted by "Sébastien Launay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sébastien Launay updated AVRO-1212:
-----------------------------------
Attachment: AVRO-1212-union-with-collections-2012-11-28.patch
> Protocol schema generated from reflection does not support @Union with collections
> ----------------------------------------------------------------------------------
>
> Key: AVRO-1212
> URL: https://issues.apache.org/jira/browse/AVRO-1212
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.7.2
> Reporter: Sébastien Launay
> Attachments: AVRO-1212-union-with-collections-2012-11-28.patch
>
>
> An interface using {{@Union}} for collections (Map, List, Java array) like this one:
> {code:java}
> public interface P0 {
> @Union({String.class,Integer.class}) List<Object> foo(@Union({Integer.class,Long.class}) List<Number> l);
> }
> {code}
> will produce the following schema where the lists have been erased by the component unions:
> {code:javascript}
> {
> "protocol" : "P0",
> "namespace" : "",
> "types" : [ {
> "type" : "record",
> "name" : "Number",
> "namespace" : "java.lang",
> "fields" : [ ]
> } ],
> "messages" : {
> "foo" : {
> "request" : [ {
> "name" : "l",
> "type" : [ "int", "long" ]
> } ],
> "response" : [ "string", "int" ]
> }
> }
> }
> {code}
> instead of:
> {code:javascript}
> {
> "protocol" : "P0X",
> "namespace" : "",
> "types" : [ {
> "type" : "record",
> "name" : "Object",
> "namespace" : "java.lang",
> "fields" : [ ]
> }, {
> "type" : "record",
> "name" : "Number",
> "namespace" : "java.lang",
> "fields" : [ ]
> } ],
> "messages" : {
> "foo" : {
> "request" : [ {
> "name" : "l",
> "type" : {
> "type" : "array",
> "items" : [ "int", "long" ],
> "java-class" : "java.util.List"
> }
> } ],
> "response" : {
> "type" : "array",
> "items" : [ "string", "int" ],
> "java-class" : "java.util.List"
> }
> }
> }
> }
> {code}
> This leads to exceptions when writing a response like List<R1|R2>:
> {noformat}
> org.apache.avro.UnresolvedUnionException: Not in union [{"type":"record","name":"R1"},{"type":"record","name":"R2"}]: [R1@19f03d7]
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:542) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:137) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:70) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.generic.GenericResponder.writeResponse(GenericResponder.java:58) ~[avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.Responder.respond(Responder.java:164) [avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.Responder.respond(Responder.java:99) [avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48) [avro-ipc-1.7.2.jar:1.7.2]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api-2.5-20081211.jar:na]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [servlet-api-2.5-20081211.jar:na]
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.Server.handle(Server.java:322) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) [jetty-util-6.1.26.jar:6.1.26]
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1212) Protocol schema generated from
reflection does not support @Union with collections
Posted by "Sébastien Launay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506982#comment-13506982 ]
Sébastien Launay commented on AVRO-1212:
----------------------------------------
Unfortunately it's not possible to annotate the parameterized type only the whole return type, that's why I went with this approach.
I agree this is a bit confusing and not applied like \@Nullable but we need at least to throw an exception when parsing that configuration (\@Union on a collection/Java array).
I am wondering how to express combination of Map/Array with annotations, I was thinking of going recursive with:
{code:java}
public interface P0 {
@Collection(@Union{Integer.class, Long.class}) // value=... is used implicitly
List<Object> foo();
@Collection(@Union{Integer.class, Long.class}) // value=... is used implicitly
Map<String, Object> bar();
@Collection(of=@Collection(@Union{Integer.class, Long.class})) // use either value= or of=
Map<String, Object[]> baz();
}
{code}
but recursive annotation definition is not allowed in Java today.
The {{Map}} use case is still a bit confusing because it only applies to the value type but having {{MapOf}} and {{ArrayOf}} does not sound DRY to me (and might require checking for avoiding misuse like using Avro type ARRAY for a Map).
Maybe only a single parameterize is allowed and for more complex use case (that looks like a code smell though) one must create a record like:
{code:java}
public interface P0 {
@CollectionOf(@Union{Integer.class, Long.class})
List<Object> foo();
@CollectionOf(@Union{Integer.class, Long.class})
Map<String, Object> bar();
Map<String, IntegerOrLong[]> baz();
}
public class IntegerOrLong {
@Union{Integer.class, Long.class}
private Object value;
...
}
{code}
this will requires @Union annotation to be used on fields (valid use case I think) and I guess will not consume more bytes.
What do you think?
In the meantime, I think I found a workaround for my initial issue by using the {{@Union}} annotation on an abstract class rather than on the method:
{code:java}
public interface P0 {
// List<? extends Message> foo(); does not work but that's another story
List<Message> foo(); // works
}
@Union{MessageA.class, MessageB.class}
public abstract class Message {
...
}
public class MessageA extends Message {
...
}
public class MessageB extends Message {
...
}
{code}
I like the {{\@AvroSchema}} though, it can be quite handy if you need full control.
> Protocol schema generated from reflection does not support @Union with collections
> ----------------------------------------------------------------------------------
>
> Key: AVRO-1212
> URL: https://issues.apache.org/jira/browse/AVRO-1212
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.7.2
> Reporter: Sébastien Launay
> Attachments: AVRO-1212-union-with-collections-2012-11-28.patch
>
>
> An interface using {{@Union}} for collections (Map, List, Java array) like this one:
> {code:java}
> public interface P0 {
> @Union({String.class,Integer.class}) List<Object> foo(@Union({Integer.class,Long.class}) List<Number> l);
> }
> {code}
> will produce the following schema where the lists have been erased by the component unions:
> {code:javascript}
> {
> "protocol" : "P0",
> "namespace" : "",
> "types" : [ {
> "type" : "record",
> "name" : "Number",
> "namespace" : "java.lang",
> "fields" : [ ]
> } ],
> "messages" : {
> "foo" : {
> "request" : [ {
> "name" : "l",
> "type" : [ "int", "long" ]
> } ],
> "response" : [ "string", "int" ]
> }
> }
> }
> {code}
> instead of:
> {code:javascript}
> {
> "protocol" : "P0X",
> "namespace" : "",
> "types" : [ {
> "type" : "record",
> "name" : "Object",
> "namespace" : "java.lang",
> "fields" : [ ]
> }, {
> "type" : "record",
> "name" : "Number",
> "namespace" : "java.lang",
> "fields" : [ ]
> } ],
> "messages" : {
> "foo" : {
> "request" : [ {
> "name" : "l",
> "type" : {
> "type" : "array",
> "items" : [ "int", "long" ],
> "java-class" : "java.util.List"
> }
> } ],
> "response" : {
> "type" : "array",
> "items" : [ "string", "int" ],
> "java-class" : "java.util.List"
> }
> }
> }
> }
> {code}
> This leads to exceptions when writing a response like List<R1|R2>:
> {noformat}
> org.apache.avro.UnresolvedUnionException: Not in union [{"type":"record","name":"R1"},{"type":"record","name":"R2"}]: [R1@19f03d7]
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:542) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:137) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:70) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57) ~[avro-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.generic.GenericResponder.writeResponse(GenericResponder.java:58) ~[avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.Responder.respond(Responder.java:164) [avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.Responder.respond(Responder.java:99) [avro-ipc-1.7.2.jar:1.7.2]
> at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48) [avro-ipc-1.7.2.jar:1.7.2]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api-2.5-20081211.jar:na]
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [servlet-api-2.5-20081211.jar:na]
> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.Server.handle(Server.java:322) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) [jetty-6.1.26.jar:6.1.26]
> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) [jetty-util-6.1.26.jar:6.1.26]
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira