You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Matt McCline (JIRA)" <ji...@apache.org> on 2018/08/15 03:30:00 UTC
[jira] [Updated] (HIVE-20321) Vectorization: Cut down memory size
of 1 col VectorHashKeyWrapper to <1 CacheLine
[ https://issues.apache.org/jira/browse/HIVE-20321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt McCline updated HIVE-20321:
--------------------------------
Attachment: HIVE-20321.01.patch
> Vectorization: Cut down memory size of 1 col VectorHashKeyWrapper to <1 CacheLine
> ---------------------------------------------------------------------------------
>
> Key: HIVE-20321
> URL: https://issues.apache.org/jira/browse/HIVE-20321
> Project: Hive
> Issue Type: Bug
> Reporter: Gopal V
> Assignee: Matt McCline
> Priority: Major
> Attachments: HIVE-20321.01.patch
>
>
> With a full sized LLAP instance, the memory size of the VectorHashKeyWrapper is bigger than the low Xmx JVMs.
> {code}
> ***** 64-bit VM: **********************************************************
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper object internals:
> OFFSET SIZE TYPE DESCRIPTION VALUE
> 0 16 (object header) N/A
> 16 4 int VectorHashKeyWrapper.hashcode N/A
> 20 4 (alignment/padding gap)
> 24 8 long[] VectorHashKeyWrapper.longValues N/A
> 32 8 double[] VectorHashKeyWrapper.doubleValues N/A
> 40 8 byte[][] VectorHashKeyWrapper.byteValues N/A
> 48 8 int[] VectorHashKeyWrapper.byteStarts N/A
> 56 8 int[] VectorHashKeyWrapper.byteLengths N/A
> 64 8 org.apache.hadoop.hive.serde2.io.HiveDecimalWritable[] VectorHashKeyWrapper.decimalValues N/A
> 72 8 java.sql.Timestamp[] VectorHashKeyWrapper.timestampValues N/A
> 80 8 org.apache.hadoop.hive.common.type.HiveIntervalDayTime[] VectorHashKeyWrapper.intervalDayTimeValues N/A
> 88 8 boolean[] VectorHashKeyWrapper.isNull N/A
> 96 8 org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper.HashContext VectorHashKeyWrapper.hashCtx N/A
> Instance size: 104 bytes
> Space losses: 4 bytes internal + 0 bytes external = 4 bytes total
> {code}
> Pulling this up to a parent class allows for this to be cut down to 32 bytes for the single column case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)