[jira] [Comment Edited] (TEZ-4228) TezClassLoader should be used in TezChild and for Configuration objects

["László Bodor (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190174#comment-17190174 ] 

László Bodor edited comment on TEZ-4228 at 9/3/20, 9:27 PM:
------------------------------------------------------------

thanks for the review [~harishjp]
failing hive test has passed with the fix: http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-1311/14/tests/
http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-1311/15
committing this to branch-0.9 / master / branch-0.10.0


was (Author: abstractdog):
thanks for the review [~harishjp]
failing hive test has passed with the fix: http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-1311/14/tests/
committing this to branch-0.9 / master / branch-0.10.0

> TezClassLoader should be used in TezChild and for Configuration objects
> -----------------------------------------------------------------------
>
>                 Key: TEZ-4228
>                 URL: https://issues.apache.org/jira/browse/TEZ-4228
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: László Bodor
>            Assignee: László Bodor
>            Priority: Blocker
>              Labels: 0.10_blocker
>             Fix For: 0.10.0, 0.9.3
>
>         Attachments: TEZ-4228.01.patch, hive_llap.log, syslog_bad.log, syslog_bad_context_classloader.log, syslog_good.log
>
>
> While testing a 0.10.0 staging artifact, I found that there multiple issues of TEZ-3860/TEZ-4223, which should be fixed before 0.10.0 release. The issues came up while hive precommit tests:
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-1311/11/tests
> The [failing hive test|https://github.com/apache/hive/blob/master/ql/src/test/queries/clientpositive/mapjoin_addjar.q] uses ADD JAR functionality:
> {code}
> ADD JAR ${system:maven.local.repository}/org/apache/hive/hive-it-test-serde/${system:hive.version}/hive-it-test-serde-${system:hive.version}.jar;
> {code}
> 1. There are code paths, where Configuration object's classloader is used, so if we add resources to TezClassLoader, we should point Configuration objects to use them, otherwise, we'll face exception as this:  [^syslog_bad.log] . 
> The changes in TezUtils solved the problem for container mode as  [^syslog_good.log] 
> 2. There are codepaths, which use thread context classloader, so calling TezClassLoader.setupTezClassLoader early in TezChild solves issues like this:  [^syslog_bad_context_classloader.log] 
> 3. The Configuration issue is the same in Hive LLAP, which is solved in 1), example failure reproduced locally is:  [^hive_llap.log].
> 4. The context classloader problem is also present in LLAP daemons, which is being solved by HIVE-24108



--
This message was sent by Atlassian Jira
(v8.3.4#803005)