You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Mohit Sabharwal <mo...@cloudera.com> on 2016/02/17 01:08:07 UTC
Review Request 43553: HIVE-13057: Remove duplicate copies of TableDesc
property values in PartitionDesc
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43553/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-13057
https://issues.apache.org/jira/browse/HIVE-13057
Repository: hive-git
Description
-------
For a partitioned table, each PartitionDesc has a copy of corresponding TableDesc.
While TableDesc is mutable and hence cannot be interned, it's property values can be.
For a simple select on a table with 100K partitions, this cut total number of String instances by ~65%.
Most replicated strings were location, serde, input/output format, column, types, table name, etc.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java b032349b7faf7026928bea90a6ded29eeb2a502c
Diff: https://reviews.apache.org/r/43553/diff/
Testing
-------
Thanks,
Mohit Sabharwal
Re: Review Request 43553: HIVE-13057: Remove duplicate copies of
TableDesc property values in PartitionDesc
Posted by Kapil Rastogi <kr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43553/#review119991
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java (line 267)
<https://reviews.apache.org/r/43553/#comment181353>
Does it makes sense to intern properties in cloning also?
- Kapil Rastogi
On Feb. 17, 2016, 12:08 a.m., Mohit Sabharwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43553/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2016, 12:08 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-13057
> https://issues.apache.org/jira/browse/HIVE-13057
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> For a partitioned table, each PartitionDesc has a copy of corresponding TableDesc.
>
> While TableDesc is mutable and hence cannot be interned, it's property values can be.
>
> For a simple select on a table with 100K partitions, this cut total number of String instances by ~65%.
>
> Most replicated strings were location, serde, input/output format, column, types, table name, etc.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/plan/PartitionDesc.java b032349b7faf7026928bea90a6ded29eeb2a502c
>
> Diff: https://reviews.apache.org/r/43553/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohit Sabharwal
>
>