[jira] [Commented] (HIVE-6579) HiveLockObjectData constructor makes too many queryStr instance causing oom

["Xuefu Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924487#comment-13924487 ] 

Xuefu Zhang commented on HIVE-6579:
-----------------------------------

[~teledi] Thanks for the patch. I saw other callers of the constructor in question. Do we need to take care of those callers as well?

> HiveLockObjectData constructor makes too many queryStr instance causing oom
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-6579
>                 URL: https://issues.apache.org/jira/browse/HIVE-6579
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: xieyuchen
>         Attachments: HIVE-6579.1.patch.txt
>
>
> We have a huge sql which full outer joins 10+ partitoned tables, each table has at least 1k partitions. The sql has 300kb in length(it constructed  automatically of cause).
> So when we running this sql, there are over 10k HiveLockObjectData instances. Because of the constructor of HiveLockObjectData trim the queryStr, there will be 10k individual String instances, each has 300kb in length! Then the Hive client will get an oom exception.
> Trying to trim the queryStr in Driver.compile function instead of doing it in HiveLockObjectData constructor to reduce memory wasting.



--
This message was sent by Atlassian JIRA
(v6.2#6252)