[jira] [Comment Edited] (HIVE-19041) Thrift deserialization of Partition objects should intern fields

["Sahil Takiar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-19041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462473#comment-16462473 ] 

Sahil Takiar edited comment on HIVE-19041 at 5/3/18 1:54 PM:
-------------------------------------------------------------

I think this applies to almost every Thrift class where we do string interning: {{SerDeInfo}}, {{StorageDescriptor}}, {{FieldSchema}}, {{Partition}}.


was (Author: stakiar):
I think this applies to almost every Thrift class where we do, {{SerDeInfo}}, {{StorageDescriptor}}, {{FieldSchema}}, {{Partition}}.

> Thrift deserialization of Partition objects should intern fields
> ----------------------------------------------------------------
>
>                 Key: HIVE-19041
>                 URL: https://issues.apache.org/jira/browse/HIVE-19041
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>    Affects Versions: 3.0.0, 2.3.2
>            Reporter: Vihang Karajgaonkar
>            Assignee: Vihang Karajgaonkar
>            Priority: Major
>
> When a client is creating large number of partitions, the thrift objects are deserialized into Partition objects. The read method of these objects does not intern the inputformat, location, outputformat which cause large number of duplicate Strings in the HMS memory. We should intern these objects while deserialization to reduce memory pressure. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)