[jira] [Commented] (CRUNCH-572) Avro schema error in Avros.tableOf() can't redefine org.apache.avro.mapred.Pair

["Adric Eckstein (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CRUNCH-572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960570#comment-14960570 ] 

Adric Eckstein commented on CRUNCH-572:
---------------------------------------

Actually, I think I see where tableOf is checking the key/value for pair schema and recreating a schema with a unique name.  The issue here might be the call to Avros.pairs(), which doesn't have that same check on its input.  

This can be mitigated by first calling tableOf(), then converting to a pair:
PTableType<Pair<String, String>, Pair<String, Double>> t4 = Avros.tableOf(t1, t2);
AvroType<Pair<Pair<String, String>, Pair<String, Double>>> t5 = Avros.pairs(t4.getKeyType(), t4.getValueType());

So maybe Avros.pairs() needs to check for instances of PTableType and redefine the schema?

> Avro schema error in Avros.tableOf() can't redefine org.apache.avro.mapred.Pair
> -------------------------------------------------------------------------------
>
>                 Key: CRUNCH-572
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-572
>             Project: Crunch
>          Issue Type: Bug
>    Affects Versions: 0.13.0
>            Reporter: Adric Eckstein
>
> The Avros.tableOf() method produces a record schema named "org.apache.avro.mapred.Pair", which cannot be unique across types, whereas the Avros.pairs() method similarly creates a record schema, but calls Avros.createTupleSchema() which creates a unique name at every call.
> Example that fails on constructing schema:
> AvroType<Pair<String, String>> t1 = Avros.tableOf(Avros.strings(), Avros.strings());
> AvroType<Pair<String, Double>> t2 = Avros.tableOf(Avros.strings(), Avros.doubles());
> AvroType<Pair<Pair<String, String>, Pair<String, Double>>> t3 = Avros.pairs(t1, t2);
> System.out.println("schema: " + t3.getSchema().toString());
> Can the tableOf() method be updated to call the createTupleSchema() method instead of the avro pair constructor?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)