[jira] [Assigned] (FLINK-27999) Fix test failure for Hive when using profile Hive3.1.1

["Chesnay Schepler (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-27999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chesnay Schepler reassigned FLINK-27999:
----------------------------------------

    Assignee: luoyuxia

> Fix test failure for Hive when using profile Hive3.1.1
> ------------------------------------------------------
>
>                 Key: FLINK-27999
>                 URL: https://issues.apache.org/jira/browse/FLINK-27999
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Hive
>    Affects Versions: 1.15.0
>            Reporter: luoyuxia
>            Assignee: luoyuxia
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.16.0
>
>
> When use 'mvn test -PHive3.1.1'  Hive connector, it will throw the following exception "Caused by: MetaException(message:Required table missing : "DBS" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCreateTables")".
> From the error message, we can know the reason is the configuration "datanucleus.schema.autoCreateTables" is not true. But when create HiveCatalog, we do really set the configuration "datanucleus.schema.autoCreateTables" to true in the hive-site.xml.
> After some debuging, I found the reason is that some test modify a static variable, and then boil the other test.
> When running HiveCatalogFactoryTest, it will call  HiveCatalog#createHiveConf(@Nullable String hiveConfDir, @Nullable String hadoopConfDir) to create a HiveConf for HiveCatalog. In this method, it will set  the static variable "hiveSiteURL" to null.
> Then, if we run "HiveCatalogHiveMetadataTest", it will call HiveTestUtils#createHiveConf() to create HiveConf. The following code will create a empty HiveConf;
>  
> {code:java}
> HiveConf hiveConf = new HiveConf(); {code}
> But in the initialize function of HiveConf, it first will apply all default variables, so "datanucleus.schema.autoCreateTables" will be set to false. And the "hiveSiteURL" is null, so skip add the resource. Then, it will check ""hive.metastore.schema.verification" is true or false. If it's true, it will set "datanucleus.schema.autoCreateTables" to false. This have a higher priority and thus overwrite the value we configure in hive-site.xml.
> To fix it, we only need to reset the static variable "hiveSiteURL" with our hive-site.xml.
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)