[jira] [Commented] (HIVE-19810) StorageHandler fail to ship jars in Tez intermittently

["Daniel Dai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-19810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502619#comment-16502619 ] 

Daniel Dai commented on HIVE-19810:
-----------------------------------

Unit test is tricky, seems tez container reuse properties are not controllable in the client side. I don't want to touch MiniTezCluster setting to affect mass tests. The attached testcase fails by chance in MiniTezCluster without the patch. Similar test on single node cluster failed reliably for me.

> StorageHandler fail to ship jars in Tez intermittently
> ------------------------------------------------------
>
>                 Key: HIVE-19810
>                 URL: https://issues.apache.org/jira/browse/HIVE-19810
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>            Priority: Major
>         Attachments: HIVE-19810.1.patch, testcase.patch
>
>
> Hive relies on StorageHandler to ship jars to backend automatically in several cases: JdbcStorageHandler, HBaseStorageHandler, AccumuloStorageHandler. This does not work reliably, in particular, the first dag in the session will have those jars, the second will not unless container is reused. In the later case, the containers allocated to first dag will be reused in the second dag so the container will have additional resources.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)