[jira] [Commented] (KAFKA-3803) JsonConverter deserialized Struct containing bytes field does not return true for equals()

["Randall Hauch (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319515#comment-15319515 ] 

Randall Hauch commented on KAFKA-3803:
--------------------------------------

The {{equals(...)}} method on {{org.apache.kafka.connect.data.Struct}} currently uses {{Arrays.equals(value,o.value)}} to compare the array of field values to that of another. However, this only works when the elements in those arrays (e.g., the field values) are primitives or objects, but fails to work when they are arrays such as {{byte[]}}. 

Interestingly, the {{StructTest}} unit test populates all fields of type {{Schema.BYTES_SCHEMA}} using {{ByteBuffer}} object, which means the current logic works fine. However, the JSON converter rehydrates the {{Struct}} objects using {{byte[]}}, whereas the Avro converter rehydrates using {{ByteBuffer}}. This means that when a {{Struct}} containing a {{Schema.BYTES_SCHEMA}} or {{Schema.OPTIONAL_BYTES_SCHEMA}} field is serialized and then deserialized with the JSON converter, the rehydrated object will not be deemed "equal" to the original.

There are three ways to fix this:

# Change only {{Struct.equals}} to use {{Arrays.deepEquals(...)}} so that it works with object values including byte arrays or {{ByteBuffer}}; or
# Change only the JSON converter to wrap all byte arrays in {{ByteBuffer}}, in which case {{Struct.equals}} will properly compare values only when {{ByteBuffer}} values are used in place of all {{byte[]}}; or  
# Change both the {{Struct.equals}} to use {{Arrays.deepEquals(...)}} and JSON converter to wrap all byte arrays in {{ByteBuffer}}. This will allow both byte arrays and {{ByteBuffer}} values to be used while making rehydrated objects equal to the original, and will make the JSON converter more consistent with what appears to be the expectations.

Personally, I think either option 1 or 3 is best. I have a local fix for 1 including a change to the unit tests. Option 3 is just a bit more work.

> JsonConverter deserialized Struct containing bytes field does not return true for equals()
> ------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-3803
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3803
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 0.10.0.0
>            Reporter: Ewen Cheslack-Postava
>
> The problem is that byte[] comparisons will return false for equality, so even if the two are effectively equal, Struct.equals will not return true.
> It's unclear if this should be fixed or not. Equality wouldn't work for map or array types containing bytes either. However, on possibility is making ByteBuffer the default instead to alleviate this, although then we may end up with asymmetry in equality.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)