You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Zoltan Haindrich (JIRA)" <ji...@apache.org> on 2018/12/04 09:02:00 UTC

[jira] [Commented] (HIVE-21000) Upgrade thrift to at least 0.10.0

    [ https://issues.apache.org/jira/browse/HIVE-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708419#comment-16708419 ] 

Zoltan Haindrich commented on HIVE-21000:
-----------------------------------------

we might also improve on this by also caching the hashcode somewhere...but as long as these objects are mutable; it's hard to not introduce bugs by doing that....

Replacing only the hashcode implementation of {{FieldSchema}} with a non-list based one reduced my actual query compile time from 3.3sec to 1.5sec - so I think this is really worth it.

> Upgrade thrift to at least 0.10.0
> ---------------------------------
>
>                 Key: HIVE-21000
>                 URL: https://issues.apache.org/jira/browse/HIVE-21000
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Zoltan Haindrich
>            Priority: Major
>         Attachments: sampler_before.png
>
>
> I was looking into some compile profiles for tables with lots of columns; and it turned out that [thrift 0.9.3 is allocating a List|https://github.com/apache/hive/blob/8e30b5e029570407d8a1db67d322a95db705750e/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FieldSchema.java#L348] during every hashcode calculation; but luckily THRIFT-2877 is improving on that - so I propose to upgrade to at least 0.10.0 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)