You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Ted Yu (JIRA)" <ji...@apache.org> on 2017/11/03 23:36:00 UTC
[jira] [Comment Edited] (KAFKA-6168) Connect Schema comparison is
slow for large schemas
[ https://issues.apache.org/jira/browse/KAFKA-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16238561#comment-16238561 ]
Ted Yu edited comment on KAFKA-6168 at 11/3/17 11:35 PM:
---------------------------------------------------------
bq. considering having each instance precompute and cache a string or byte[] representation of all fields
Should this be done for all fields (quite a few) ?
Or just selected fields such as keySchema ?
was (Author: yuzhihong@gmail.com):
Can I work on this ?
> Connect Schema comparison is slow for large schemas
> ---------------------------------------------------
>
> Key: KAFKA-6168
> URL: https://issues.apache.org/jira/browse/KAFKA-6168
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 1.0.0
> Reporter: Randall Hauch
> Priority: Critical
> Attachments: 6168.v1.txt
>
>
> The {{ConnectSchema}} implementation computes the hash code every time its needed, and {{equals(Object)}} is a deep equality check. This extra work can be expensive for large schemas, especially in code like the {{AvroConverter}} (or rather {{AvroData}} in the converter) that uses instances as keys in a hash map that then requires significant use of {{hashCode}} and {{equals}}.
> The {{ConnectSchema}} is an immutable object and should at a minimum precompute the hash code. Also, the order that the fields are compared in {{equals(...)}} should use the cheapest comparisons first (e.g., the {{name}} field is one of the _last_ fields to be checked). Finally, it might be worth considering having each instance precompute and cache a string or byte[] representation of all fields that can be used for faster equality checking.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)