[jira] [Commented] (HIVE-16425) Vectorization: unload old hashtables before reloadHashTable

["Gopal V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-16425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965690#comment-15965690 ] 

Gopal V commented on HIVE-16425:
--------------------------------

[~mmccline]: quick review?

> Vectorization: unload old hashtables before reloadHashTable
> -----------------------------------------------------------
>
>                 Key: HIVE-16425
>                 URL: https://issues.apache.org/jira/browse/HIVE-16425
>             Project: Hive
>          Issue Type: Bug
>          Components: Vectorization
>            Reporter: Gopal V
>            Assignee: Gopal V
>         Attachments: HIVE-16425.1.patch
>
>
> {code}
>   @Override
>   protected void reloadHashTable(byte pos, int partitionId)
>           throws IOException, HiveException, SerDeException, ClassNotFoundException {
>     // The super method will reload a hash table partition of one of the small tables.
>     // Currently, for native vector map join it will only be one small table.
>     super.reloadHashTable(pos, partitionId);
>     MapJoinTableContainer smallTable = spilledMapJoinTables[pos];
>     vectorMapJoinHashTable = VectorMapJoinOptimizedCreateHashTable.createHashTable(conf,
>         smallTable);
>     needHashTableSetup = true;
>     LOG.info("Created " + vectorMapJoinHashTable.getClass().getSimpleName() + " from " + this.getClass().getSimpleName());
>     if (isLogDebugEnabled) {
>       LOG.debug(CLASS_NAME + " reloadHashTable!");
>     }
>   }
> {code}
> The super call causes an OOM because of existing memory usage by vectorMapJoinHashTable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)