You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sam An (Jira)" <ji...@apache.org> on 2020/08/12 17:50:00 UTC
[jira] [Commented] (HIVE-24022) Optimise
HiveMetaStoreAuthorizer.createHiveMetaStoreAuthorizer
[ https://issues.apache.org/jira/browse/HIVE-24022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176513#comment-17176513 ]
Sam An commented on HIVE-24022:
-------------------------------
I am trying to use ThreadLocal<HiveConf> to skip creating HiveConf each and every time createHiveMetastoreAuthorizer gets called. Currently there are 2 Unit Test failures. The root cause is in the TestHiveMetastoreAuthorizer unit test set up, the Config overlay was not visible inside the ThreadLocal hiveconf, so it is not using the DummyHiveAuthorizerFactory as Authorization manager, but instead look for SqlStdAuthorizationFactoryForTest, and it fails to find the class, causing failure.
[https://github.com/apache/hive/blob/2b3c689baff857c18164a9610f2854583105734a/ql/src/test/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/TestHiveMetaStoreAuthorizer.java#L83]
So I see 2 problem,
1. DummyHiveAuthorizerFactory is not passed in to ThreadLocal<> for unit test.
2. classloader not able to find SQLStdHiveAuthorizerFactoryForTest class in classpath.
Without my change, it was using the DummyHiveAuthorizerFactory for unit test. So I should make the overlay happen first, then see if it can find the class DummyHiveAuthorizerFactory.
> Optimise HiveMetaStoreAuthorizer.createHiveMetaStoreAuthorizer
> --------------------------------------------------------------
>
> Key: HIVE-24022
> URL: https://issues.apache.org/jira/browse/HIVE-24022
> Project: Hive
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Priority: Minor
> Labels: performance, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For a table with 3000+ partitions, analyze table takes a lot longer time as HiveMetaStoreAuthorizer tries to create HiveConf for every partition request.
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java#L319]
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java#L447]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)