You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Ed Kohlwey (JIRA)" <ji...@apache.org> on 2012/07/17 17:38:33 UTC
[jira] [Created] (AVRO-1128) SpecificRecord doesn't check for
equality correctly
Ed Kohlwey created AVRO-1128:
--------------------------------
Summary: SpecificRecord doesn't check for equality correctly
Key: AVRO-1128
URL: https://issues.apache.org/jira/browse/AVRO-1128
Project: Avro
Issue Type: Bug
Affects Versions: 1.7.0, 1.7.1
Reporter: Ed Kohlwey
Priority: Critical
In Java, map equality is based on the contents of a map rather than ordering, since order is undefined in the plain Java map implementation. The present implementation of SpecificRecord relies on SpecificData's public compare() method, which doesn't expose any method to relax the ordering requirement for maps (a protected-visibility implementation exists that does). Since specificRecord is based on map (and not some sort of ordered map) the current implementation with respect to the Java spec is fundamentally incompatible since Java doesn't require a notion of ordering for the Map interface and Avro can.
There's a few ways this could be solved:
# Require Avro Java to use the LinkedHashMap implementation of map (or similar) so that map ordering can be compared and not break the normal expectations of a java developer. This is problematic because it dramatically restricts the API and forces an implementation of Map that is undesirable under some circumstances. Unfortunately there is no concept of OrderedMap in Java, just SortedMap.
# Change SpecificData to compare maps based on contents (set equals=true when recursively comparing maps)
# Change the visibility of SpecificData.compare(SpecificRecord,SpecificRecord,boolean) to public and change SpecificRecord.equals(Object) to compare only on equality rather than order.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1128) SpecificRecord doesn't check for
equality correctly with respect to maps
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-1128:
-------------------------------
Attachment: AVRO-1128.patch
Here's a patch that I believe addresses the issue you describe.
> SpecificRecord doesn't check for equality correctly with respect to maps
> ------------------------------------------------------------------------
>
> Key: AVRO-1128
> URL: https://issues.apache.org/jira/browse/AVRO-1128
> Project: Avro
> Issue Type: Bug
> Affects Versions: 1.7.0, 1.7.1
> Reporter: Ed Kohlwey
> Priority: Critical
> Attachments: AVRO-1128.patch
>
>
> In Java, map equality is based on the contents of a map rather than ordering, since order is undefined in the plain Java map implementation. The present implementation of SpecificRecord relies on SpecificData's public compare() method, which doesn't expose any method to relax the ordering requirement for maps (a protected-visibility implementation exists that does). Since specificRecord is based on map (and not some sort of ordered map) the current implementation with respect to the Java spec is fundamentally incompatible since Java doesn't require a notion of ordering for the Map interface and Avro can.
> There's a few ways this could be solved:
> # Require Avro Java to use the LinkedHashMap implementation of map (or similar) so that map ordering can be compared and not break the normal expectations of a java developer. This is problematic because it dramatically restricts the API and forces an implementation of Map that is undesirable under some circumstances. Unfortunately there is no concept of OrderedMap in Java, just SortedMap.
> # Change SpecificData to compare maps based on contents (set equals=true when recursively comparing maps)
> # Change the visibility of SpecificData.compare(SpecificRecord,SpecificRecord,boolean) to public and change SpecificRecord.equals(Object) to compare only on equality rather than order.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1128) SpecificRecord doesn't check for
equality correctly with respect to maps
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-1128:
-------------------------------
Fix Version/s: 1.7.2
Ed, does this address your concern? If so, I'll commit it.
> SpecificRecord doesn't check for equality correctly with respect to maps
> ------------------------------------------------------------------------
>
> Key: AVRO-1128
> URL: https://issues.apache.org/jira/browse/AVRO-1128
> Project: Avro
> Issue Type: Bug
> Affects Versions: 1.7.0, 1.7.1
> Reporter: Ed Kohlwey
> Assignee: Doug Cutting
> Priority: Critical
> Fix For: 1.7.2
>
> Attachments: AVRO-1128.patch
>
>
> In Java, map equality is based on the contents of a map rather than ordering, since order is undefined in the plain Java map implementation. The present implementation of SpecificRecord relies on SpecificData's public compare() method, which doesn't expose any method to relax the ordering requirement for maps (a protected-visibility implementation exists that does). Since specificRecord is based on map (and not some sort of ordered map) the current implementation with respect to the Java spec is fundamentally incompatible since Java doesn't require a notion of ordering for the Map interface and Avro can.
> There's a few ways this could be solved:
> # Require Avro Java to use the LinkedHashMap implementation of map (or similar) so that map ordering can be compared and not break the normal expectations of a java developer. This is problematic because it dramatically restricts the API and forces an implementation of Map that is undesirable under some circumstances. Unfortunately there is no concept of OrderedMap in Java, just SortedMap.
> # Change SpecificData to compare maps based on contents (set equals=true when recursively comparing maps)
> # Change the visibility of SpecificData.compare(SpecificRecord,SpecificRecord,boolean) to public and change SpecificRecord.equals(Object) to compare only on equality rather than order.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1128) SpecificRecord doesn't check for
equality correctly with respect to maps
Posted by "Ed Kohlwey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ed Kohlwey updated AVRO-1128:
-----------------------------
Summary: SpecificRecord doesn't check for equality correctly with respect to maps (was: SpecificRecord doesn't check for equality correctly)
> SpecificRecord doesn't check for equality correctly with respect to maps
> ------------------------------------------------------------------------
>
> Key: AVRO-1128
> URL: https://issues.apache.org/jira/browse/AVRO-1128
> Project: Avro
> Issue Type: Bug
> Affects Versions: 1.7.0, 1.7.1
> Reporter: Ed Kohlwey
> Priority: Critical
>
> In Java, map equality is based on the contents of a map rather than ordering, since order is undefined in the plain Java map implementation. The present implementation of SpecificRecord relies on SpecificData's public compare() method, which doesn't expose any method to relax the ordering requirement for maps (a protected-visibility implementation exists that does). Since specificRecord is based on map (and not some sort of ordered map) the current implementation with respect to the Java spec is fundamentally incompatible since Java doesn't require a notion of ordering for the Map interface and Avro can.
> There's a few ways this could be solved:
> # Require Avro Java to use the LinkedHashMap implementation of map (or similar) so that map ordering can be compared and not break the normal expectations of a java developer. This is problematic because it dramatically restricts the API and forces an implementation of Map that is undesirable under some circumstances. Unfortunately there is no concept of OrderedMap in Java, just SortedMap.
> # Change SpecificData to compare maps based on contents (set equals=true when recursively comparing maps)
> # Change the visibility of SpecificData.compare(SpecificRecord,SpecificRecord,boolean) to public and change SpecificRecord.equals(Object) to compare only on equality rather than order.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1128) SpecificRecord doesn't check for
equality correctly with respect to maps
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-1128:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I committed this.
> SpecificRecord doesn't check for equality correctly with respect to maps
> ------------------------------------------------------------------------
>
> Key: AVRO-1128
> URL: https://issues.apache.org/jira/browse/AVRO-1128
> Project: Avro
> Issue Type: Bug
> Affects Versions: 1.7.0, 1.7.1
> Reporter: Ed Kohlwey
> Assignee: Doug Cutting
> Priority: Critical
> Fix For: 1.7.2
>
> Attachments: AVRO-1128.patch
>
>
> In Java, map equality is based on the contents of a map rather than ordering, since order is undefined in the plain Java map implementation. The present implementation of SpecificRecord relies on SpecificData's public compare() method, which doesn't expose any method to relax the ordering requirement for maps (a protected-visibility implementation exists that does). Since specificRecord is based on map (and not some sort of ordered map) the current implementation with respect to the Java spec is fundamentally incompatible since Java doesn't require a notion of ordering for the Map interface and Avro can.
> There's a few ways this could be solved:
> # Require Avro Java to use the LinkedHashMap implementation of map (or similar) so that map ordering can be compared and not break the normal expectations of a java developer. This is problematic because it dramatically restricts the API and forces an implementation of Map that is undesirable under some circumstances. Unfortunately there is no concept of OrderedMap in Java, just SortedMap.
> # Change SpecificData to compare maps based on contents (set equals=true when recursively comparing maps)
> # Change the visibility of SpecificData.compare(SpecificRecord,SpecificRecord,boolean) to public and change SpecificRecord.equals(Object) to compare only on equality rather than order.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-1128) SpecificRecord doesn't check for
equality correctly with respect to maps
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-1128:
-------------------------------
Assignee: Doug Cutting
Status: Patch Available (was: Open)
> SpecificRecord doesn't check for equality correctly with respect to maps
> ------------------------------------------------------------------------
>
> Key: AVRO-1128
> URL: https://issues.apache.org/jira/browse/AVRO-1128
> Project: Avro
> Issue Type: Bug
> Affects Versions: 1.7.0, 1.7.1
> Reporter: Ed Kohlwey
> Assignee: Doug Cutting
> Priority: Critical
> Attachments: AVRO-1128.patch
>
>
> In Java, map equality is based on the contents of a map rather than ordering, since order is undefined in the plain Java map implementation. The present implementation of SpecificRecord relies on SpecificData's public compare() method, which doesn't expose any method to relax the ordering requirement for maps (a protected-visibility implementation exists that does). Since specificRecord is based on map (and not some sort of ordered map) the current implementation with respect to the Java spec is fundamentally incompatible since Java doesn't require a notion of ordering for the Map interface and Avro can.
> There's a few ways this could be solved:
> # Require Avro Java to use the LinkedHashMap implementation of map (or similar) so that map ordering can be compared and not break the normal expectations of a java developer. This is problematic because it dramatically restricts the API and forces an implementation of Map that is undesirable under some circumstances. Unfortunately there is no concept of OrderedMap in Java, just SortedMap.
> # Change SpecificData to compare maps based on contents (set equals=true when recursively comparing maps)
> # Change the visibility of SpecificData.compare(SpecificRecord,SpecificRecord,boolean) to public and change SpecificRecord.equals(Object) to compare only on equality rather than order.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1128) SpecificRecord doesn't check for
equality correctly with respect to maps
Posted by "Ed Kohlwey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418634#comment-13418634 ]
Ed Kohlwey commented on AVRO-1128:
----------------------------------
Yes, thats exactly what I was thinking of. Thanks Doug!
> SpecificRecord doesn't check for equality correctly with respect to maps
> ------------------------------------------------------------------------
>
> Key: AVRO-1128
> URL: https://issues.apache.org/jira/browse/AVRO-1128
> Project: Avro
> Issue Type: Bug
> Affects Versions: 1.7.0, 1.7.1
> Reporter: Ed Kohlwey
> Assignee: Doug Cutting
> Priority: Critical
> Fix For: 1.7.2
>
> Attachments: AVRO-1128.patch
>
>
> In Java, map equality is based on the contents of a map rather than ordering, since order is undefined in the plain Java map implementation. The present implementation of SpecificRecord relies on SpecificData's public compare() method, which doesn't expose any method to relax the ordering requirement for maps (a protected-visibility implementation exists that does). Since specificRecord is based on map (and not some sort of ordered map) the current implementation with respect to the Java spec is fundamentally incompatible since Java doesn't require a notion of ordering for the Map interface and Avro can.
> There's a few ways this could be solved:
> # Require Avro Java to use the LinkedHashMap implementation of map (or similar) so that map ordering can be compared and not break the normal expectations of a java developer. This is problematic because it dramatically restricts the API and forces an implementation of Map that is undesirable under some circumstances. Unfortunately there is no concept of OrderedMap in Java, just SortedMap.
> # Change SpecificData to compare maps based on contents (set equals=true when recursively comparing maps)
> # Change the visibility of SpecificData.compare(SpecificRecord,SpecificRecord,boolean) to public and change SpecificRecord.equals(Object) to compare only on equality rather than order.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira