[jira] [Commented] (HIVE-17243) Replace Hive's JavaDataModel with Java Object Layout

["Prasanth Jayachandran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113573#comment-16113573 ] 

Prasanth Jayachandran commented on HIVE-17243:
----------------------------------------------

Can't do this because of licensing issue of jol-core.

> Replace Hive's JavaDataModel with Java Object Layout
> ----------------------------------------------------
>
>                 Key: HIVE-17243
>                 URL: https://issues.apache.org/jira/browse/HIVE-17243
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 3.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>
> org.apache.hadoop.hive.ql.util.JavaDataModel is used for estimating memory of objects in many places. The problem with this approach is that manual accounting for all fields and references has to be done. This will also be problematic for cases where shallow object size vs deep object sizes are required. 
> Hash table memory monitoring does accounting for size of the hash tables. The estimated sizes of hash tables are often very different for non-vector operators whereas the estimates are close to actual object size for vectorized operators. Also addition of fields requires manual changes to memory monitoring. 
> Java object layout is openjdk project that can provide shallow and deep object sizes without using java agents. We can leverage that for much more accurate memory estimates.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)