You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Sorabh Hamirwasia (JIRA)" <ji...@apache.org> on 2017/12/29 23:03:01 UTC
[jira] [Updated] (DRILL-6063) Set correct ThreadContext ClassLoader
before using Hadoop Configuration class in DrillClient
[ https://issues.apache.org/jira/browse/DRILL-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sorabh Hamirwasia updated DRILL-6063:
-------------------------------------
Description:
Drill Client uses Hadoop Configuration class in security mechanism factories. Configuration class internally uses ThreadContext class loader to get the default resources for configurations. When Drill JDBC driver is used with 3rd party tool like Squirrel then it creates a class loader with all Drill Driver related dependencies. This class loader is used to instantiate an object of Driver class in a separate thread. But the thread context class loader has dependencies related to Squirrel only. As a result of this when Hadoop Configuration class tries to load the resources with default values it doesn't find them in class path.
If we specify hadoop dependencies in Squirrel's classpath instead of Driver dependencies class path then it will work since Thread Context class loader will have Hadoop libraries loaded too and will find the resources. But this is not good as it is tying up one Squirrel instance with one version of Hadoop which might not be true in all the cases like here when same Squirrel instance can be used to load multiple versions of Drill Driver having different hadoop version dependencies.
To deal with this issue a simple fix is to set the Thread Context class loader to be same as Drill Driver dependencies class loader before accessing Hadoop Configuration class in Drill Client and later restore it back.
Some pointers in code:
* Place where JDBC driver is loaded as part of separate thread in Squirrel: [OpenConnectionCommand.java|https://sourceforge.net/p/squirrel-sql/git/ci/master/tree/sql12/core/src/net/sourceforge/squirrel_sql/client/mainframe/action/OpenConnectionCommand.java#l167]
* Place where JDBC driver class is instantiated in Squirrel using Driver dependencies class loader: [SQLDriverManager.java|https://sourceforge.net/p/squirrel-sql/git/ci/master/tree/sql12/core/src/net/sourceforge/squirrel_sql/fw/sql/SQLDriverManager.java#l74]
* Place where Hadoop Configuration class looks into Thread Context Class Loader: [Configuration.java|https://github.com/apache/hadoop-common/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L615]
was:
Drill Client uses Hadoop Configuration class in security mechanism factories. Configuration class internally uses ThreadContext class loader to get the default resources for configurations. When Drill JDBC driver is used with 3rd party tool like Squirrel then it creates a class loader with all Drill Driver related dependencies. This class loader is used to instantiate an object of Driver class in a separate thread. But the thread context class loader has dependencies related to Squirrel only. As a result of this when Hadoop Configuration class tries to load the resources with default values it doesn't find them in class path.
If we specify hadoop dependencies in Squirrel's classpath instead of Driver dependencies class path then it will work since Thread Context class loader will have Hadoop libraries loaded too and will find the resources. But this is not good as it is tying up one Squirrel instance with one version of Hadoop which might not be true in all the cases like here when same Squirrel instance can be used to load multiple versions of Drill Driver having different hadoop version dependencies.
To deal with this issue a simple fix is to set the Thread Context class loader to be same as Drill Driver dependencies class loader before accessing Hadoop Configuration class in Drill Client and later restore it back.
> Set correct ThreadContext ClassLoader before using Hadoop Configuration class in DrillClient
> --------------------------------------------------------------------------------------------
>
> Key: DRILL-6063
> URL: https://issues.apache.org/jira/browse/DRILL-6063
> Project: Apache Drill
> Issue Type: Improvement
> Components: Client - Java
> Affects Versions: 1.11.0
> Reporter: Sorabh Hamirwasia
> Assignee: Sorabh Hamirwasia
>
> Drill Client uses Hadoop Configuration class in security mechanism factories. Configuration class internally uses ThreadContext class loader to get the default resources for configurations. When Drill JDBC driver is used with 3rd party tool like Squirrel then it creates a class loader with all Drill Driver related dependencies. This class loader is used to instantiate an object of Driver class in a separate thread. But the thread context class loader has dependencies related to Squirrel only. As a result of this when Hadoop Configuration class tries to load the resources with default values it doesn't find them in class path.
> If we specify hadoop dependencies in Squirrel's classpath instead of Driver dependencies class path then it will work since Thread Context class loader will have Hadoop libraries loaded too and will find the resources. But this is not good as it is tying up one Squirrel instance with one version of Hadoop which might not be true in all the cases like here when same Squirrel instance can be used to load multiple versions of Drill Driver having different hadoop version dependencies.
> To deal with this issue a simple fix is to set the Thread Context class loader to be same as Drill Driver dependencies class loader before accessing Hadoop Configuration class in Drill Client and later restore it back.
> Some pointers in code:
> * Place where JDBC driver is loaded as part of separate thread in Squirrel: [OpenConnectionCommand.java|https://sourceforge.net/p/squirrel-sql/git/ci/master/tree/sql12/core/src/net/sourceforge/squirrel_sql/client/mainframe/action/OpenConnectionCommand.java#l167]
> * Place where JDBC driver class is instantiated in Squirrel using Driver dependencies class loader: [SQLDriverManager.java|https://sourceforge.net/p/squirrel-sql/git/ci/master/tree/sql12/core/src/net/sourceforge/squirrel_sql/fw/sql/SQLDriverManager.java#l74]
> * Place where Hadoop Configuration class looks into Thread Context Class Loader: [Configuration.java|https://github.com/apache/hadoop-common/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L615]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)