You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by li...@apache.org on 2019/12/03 01:14:34 UTC
[submarine] branch master updated: SUBMARINE-283. Metadata access
services
This is an automated email from the ASF dual-hosted git repository.
liuxun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new d394781 SUBMARINE-283. Metadata access services
d394781 is described below
commit d394781a2c7d427615fb9886a9a8ebe1bcbb96e1
Author: LinhaoZhu <zh...@163.com>
AuthorDate: Mon Dec 2 16:16:35 2019 +0800
SUBMARINE-283. Metadata access services
### What is this PR for?
1. Put in the hive SQL scripts, create a mysql user(metastore) inside mini-submarine, then create the hide-meta metadata table structure in the metastore user.
2. In submarine-site.xml, add a MYSQL configuration of metastore, Hive's jar connects mysql metadata through this configuration.
### What type of PR is it?
[ Feature ]
### Todos
* [ ] - Task
### What is the Jira issue?
* [SUBMARINE-283](https://issues.apache.org/jira/browse/SUBMARINE-283)
### How should this be tested?
* [Pass-CI](https://travis-ci.org/LinhaoZhu/hadoop-submarine/builds/619433250)
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: LinhaoZhu <zh...@163.com>
Author: Linhao Zhu <zh...@gmail.com>
Closes #118 from LinhaoZhu/SUBMARINE-283 and squashes the following commits:
27df4f1 [LinhaoZhu] [SUBMARINE-283] Update docs.
38b1a48 [LinhaoZhu] [SUBMARINE-283] Delete MetastoreRestApi.
6b31d71 [LinhaoZhu] [SUBMARINE-283] Exclusion more.
b1b3e57 [LinhaoZhu] [SUBMARINE-283] Exclusion jetty-server.
ea4564a [LinhaoZhu] [SUBMARINE-283] Exclusion jetty-webapp.
44b8be6 [LinhaoZhu] [SUBMARINE-283] Exclusion more.
15504fd [LinhaoZhu] [SUBMARINE-283] Exclusion hadoop-auth.
12d1a99 [LinhaoZhu] [SUBMARINE-283] Add MetastoreRestApi.
1dcd86a [LinhaoZhu] [SUBMARINE-283] Metadata access services.
719c3c6 [Linhao Zhu] Merge pull request #7 from apache/master
bf2f678 [Linhao Zhu] Merge pull request #4 from apache/master
722bb9d [Linhao Zhu] Merge pull request #3 from apache/master
d06059a [Linhao Zhu] Merge pull request #2 from apache/master
ca7409f [Linhao Zhu] Merge pull request #1 from apache/master
---
.travis.yml | 4 +
dev-support/mini-submarine/conf/setup-mysql.sh | 10 +
docs/database/README.md | 20 +-
docs/database/metastore.sql | 982 +++++++++++++++++++++
pom.xml | 2 +-
submarine-commons/commons-metastore/pom.xml | 135 +++
.../commons/metastore/SubmarineMetaStore.java | 185 +++-
.../commons/metastore/SubmarineMetaStoreTest.java | 197 +++++
.../src/test/resources/log4j.properties | 20 +-
.../commons/utils/SubmarineConfiguration.java | 34 +
10 files changed, 1570 insertions(+), 19 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 0c0309d..fcd70f1 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -60,6 +60,10 @@ before_install:
- mysql -e "CREATE USER 'submarine_test'@'%' IDENTIFIED BY 'password_test';"
- mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'submarine_test'@'%';"
- mysql -e "use submarine_test; source ./docs/database/submarine.sql; show tables;"
+ - mysql -e "create database metastore_test;"
+ - mysql -e "CREATE USER 'metastore_test'@'%' IDENTIFIED BY 'password_test';"
+ - mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'metastore_test'@'%';"
+ - mysql -e "use metastore_test; source ./docs/database/metastore.sql; show tables;"
- ./dev-support/travis/install_external_dependencies.sh
matrix:
diff --git a/dev-support/mini-submarine/conf/setup-mysql.sh b/dev-support/mini-submarine/conf/setup-mysql.sh
index e9743a4..4ad211f 100755
--- a/dev-support/mini-submarine/conf/setup-mysql.sh
+++ b/dev-support/mini-submarine/conf/setup-mysql.sh
@@ -26,3 +26,13 @@ mysql -e "CREATE DATABASE submarine;"
mysql -e "CREATE USER 'submarine'@'%' IDENTIFIED BY 'password';"
mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'submarine'@'%';"
mysql -e "use submarine; source /home/yarn/database/submarine.sql; source /home/yarn/database/submarine-data.sql;"
+
+mysql -e "CREATE DATABASE metastore_test;"
+mysql -e "CREATE USER 'metastore_test'@'%' IDENTIFIED BY 'password_test';"
+mysql -e "GRANT ALL PRIVILEGES ON * . * TO 'metastore_test'@'%';"
+mysql -e "use metastore_test; source /home/yarn/database/metastore.sql;"
+
+mysql -e "CREATE DATABASE metastore;"
+mysql -e "CREATE USER 'metastore'@'%' IDENTIFIED BY 'password';"
+mysql -e "GRANT ALL PRIVILEGES ON * . * TO 'metastore'@'%';"
+mysql -e "use metastore; source /home/yarn/database/metastore.sql;"
\ No newline at end of file
diff --git a/docs/database/README.md b/docs/database/README.md
index 6f2ad5e..7e9c621 100644
--- a/docs/database/README.md
+++ b/docs/database/README.md
@@ -15,7 +15,7 @@
# Submarine Database
-Submarine needs to use the database to store information about the `organization`, `user`, `projects`, `tasks` and configuration of the system information, So consider using mysql to store this data.
+Submarine needs to use the database to store information about the `organization`, `user`, `projects`, `tasks`, `metastore` and `configuration` of the system information, So consider using mysql to store this data.
+ MySQL will be included in the `mini-submarine` docker image to allow users to quickly experience the `submarine workbench`.
+ In a production environment, the `submarine workbench` can be connected to the official mysql database.
@@ -79,11 +79,12 @@ If you need to store Chinese character data in mysql, you need to execute the fo
## Create Submarine Database
### Create development database
-Copy the files, submarine.sql and submarine-data.sql to the mysql docker.
+Copy the files, submarine.sql, submarine-data.sql and metastore.sql to the mysql docker.
```
docker cp ${SUBMARINE_HOME}/docs/database/submarine.sql ${DOCKER_ID}:/
docker cp ${SUBMARINE_HOME}/docs/database/submarine-data.sql ${DOCKER_ID}:/
+docker cp ${SUBMARINE_HOME}/docs/database/metastore.sql ${DOCKER_ID}:/
```
Development database for development environment.
@@ -97,10 +98,15 @@ mysql> CREATE DATABASE submarine CHARACTER SET utf8 COLLATE utf8_general_ci;
mysql> use submarine;
mysql> source /submarine.sql;
mysql> source /submarine-data.sql;
+mysql> CREATE USER 'metastore'@'%' IDENTIFIED BY 'password';
+mysql> GRANT ALL PRIVILEGES ON * . * TO 'metastore'@'%';
+mysql> CREATE DATABASE metastore CHARACTER SET utf8 COLLATE utf8_general_ci;
+mysql> use metastore;
+mysql> source /metastore.sql;
mysql> quit
```
-> NOTE: submarine development database name is `submarine` and user name is `submarine`, password is `password`, This is the default value in the system's `submarine-site.xml` configuration file and is not recommended for modification.
+> NOTE: submarine development database name is `submarine` and user name is `submarine`, password is `password`, metastore development database name is `metastore` and user name is `metastore`, password is `password`, This is the default value in the system's `submarine-site.xml` configuration file and is not recommended for modification.
### Create test database
@@ -115,15 +121,21 @@ mysql> GRANT ALL PRIVILEGES ON * . * TO 'submarine_test'@'%';
mysql> CREATE DATABASE `submarine_test` CHARACTER SET utf8 COLLATE utf8_general_ci;
mysql> use `submarine_test`;
mysql> source /submarine.sql;
+mysql> CREATE USER 'metastore_test'@'%' IDENTIFIED BY 'password_test';
+mysql> GRANT ALL PRIVILEGES ON * . * TO 'metastore_test'@'%';
+mysql> CREATE DATABASE `metastore_test` CHARACTER SET utf8 COLLATE utf8_general_ci;
+mysql> use `metastore_test`;
+mysql> source /metastore.sql;
mysql> quit
```
-> NOTE: submarine test database name is `submarine_test` and user name is `submarine_test`, password is `password_test`, Cannot be configured, values that cannot be modified.
+> NOTE: submarine test database name is `submarine_test` and user name is `submarine_test`, password is `password_test`, metastore test database name is `metastore_test` and user name is `metastore_test`, password is `password_test`, Cannot be configured, values that cannot be modified.
### mysqldump
```$xslt
mysqldump -uroot -ppassword --databases submarine > submarine.sql;
+mysqldump -umetastore -ppassword metastore > metastore.sql;
```
diff --git a/docs/database/metastore.sql b/docs/database/metastore.sql
new file mode 100644
index 0000000..041c96d
--- /dev/null
+++ b/docs/database/metastore.sql
@@ -0,0 +1,982 @@
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+-- http://www.apache.org/licenses/LICENSE-2.0
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+
+-- fork by https://github.com/apache/hive/blob/master/metastore/scripts/
+-- upgrade/mysql/hive-schema-2.3.0.mysql.sql
+
+-- fork by https://github.com/apache/hive/blob/master/metastore/scripts/
+-- upgrade/mysql/hive-txn-schema-2.3.0.mysql.sql
+
+/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
+/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
+/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
+/*!40101 SET NAMES utf8 */;
+/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
+/*!40103 SET TIME_ZONE='+00:00' */;
+/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
+/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
+/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
+/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
+
+--
+-- Table structure for table `BUCKETING_COLS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `BUCKETING_COLS` (
+ `SD_ID` bigint(20) NOT NULL,
+ `BUCKET_COL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`SD_ID`,`INTEGER_IDX`),
+ KEY `BUCKETING_COLS_N49` (`SD_ID`),
+ CONSTRAINT `BUCKETING_COLS_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `CDS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `CDS` (
+ `CD_ID` bigint(20) NOT NULL,
+ PRIMARY KEY (`CD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `COLUMNS_V2`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `COLUMNS_V2` (
+ `CD_ID` bigint(20) NOT NULL,
+ `COMMENT` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `TYPE_NAME` MEDIUMTEXT DEFAULT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`CD_ID`,`COLUMN_NAME`),
+ KEY `COLUMNS_V2_N49` (`CD_ID`),
+ CONSTRAINT `COLUMNS_V2_FK1` FOREIGN KEY (`CD_ID`) REFERENCES `CDS` (`CD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `DATABASE_PARAMS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `DATABASE_PARAMS` (
+ `DB_ID` bigint(20) NOT NULL,
+ `PARAM_KEY` varchar(180) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PARAM_VALUE` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`DB_ID`,`PARAM_KEY`),
+ KEY `DATABASE_PARAMS_N49` (`DB_ID`),
+ CONSTRAINT `DATABASE_PARAMS_FK1` FOREIGN KEY (`DB_ID`) REFERENCES `DBS` (`DB_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `DBS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `DBS` (
+ `DB_ID` bigint(20) NOT NULL,
+ `DESC` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `DB_LOCATION_URI` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `OWNER_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `OWNER_TYPE` varchar(10) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`DB_ID`),
+ UNIQUE KEY `UNIQUE_DATABASE` (`NAME`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `DB_PRIVS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `DB_PRIVS` (
+ `DB_GRANT_ID` bigint(20) NOT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `DB_ID` bigint(20) DEFAULT NULL,
+ `GRANT_OPTION` smallint(6) NOT NULL,
+ `GRANTOR` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `GRANTOR_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `DB_PRIV` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`DB_GRANT_ID`),
+ UNIQUE KEY `DBPRIVILEGEINDEX` (`DB_ID`,`PRINCIPAL_NAME`,`PRINCIPAL_TYPE`,`DB_PRIV`,`GRANTOR`,`GRANTOR_TYPE`),
+ KEY `DB_PRIVS_N49` (`DB_ID`),
+ CONSTRAINT `DB_PRIVS_FK1` FOREIGN KEY (`DB_ID`) REFERENCES `DBS` (`DB_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `GLOBAL_PRIVS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `GLOBAL_PRIVS` (
+ `USER_GRANT_ID` bigint(20) NOT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `GRANT_OPTION` smallint(6) NOT NULL,
+ `GRANTOR` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `GRANTOR_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `USER_PRIV` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`USER_GRANT_ID`),
+ UNIQUE KEY `GLOBALPRIVILEGEINDEX` (`PRINCIPAL_NAME`,`PRINCIPAL_TYPE`,`USER_PRIV`,`GRANTOR`,`GRANTOR_TYPE`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `IDXS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `IDXS` (
+ `INDEX_ID` bigint(20) NOT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `DEFERRED_REBUILD` bit(1) NOT NULL,
+ `INDEX_HANDLER_CLASS` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `INDEX_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `INDEX_TBL_ID` bigint(20) DEFAULT NULL,
+ `LAST_ACCESS_TIME` int(11) NOT NULL,
+ `ORIG_TBL_ID` bigint(20) DEFAULT NULL,
+ `SD_ID` bigint(20) DEFAULT NULL,
+ PRIMARY KEY (`INDEX_ID`),
+ UNIQUE KEY `UNIQUEINDEX` (`INDEX_NAME`,`ORIG_TBL_ID`),
+ KEY `IDXS_N51` (`SD_ID`),
+ KEY `IDXS_N50` (`INDEX_TBL_ID`),
+ KEY `IDXS_N49` (`ORIG_TBL_ID`),
+ CONSTRAINT `IDXS_FK1` FOREIGN KEY (`ORIG_TBL_ID`) REFERENCES `TBLS` (`TBL_ID`),
+ CONSTRAINT `IDXS_FK2` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`),
+ CONSTRAINT `IDXS_FK3` FOREIGN KEY (`INDEX_TBL_ID`) REFERENCES `TBLS` (`TBL_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `INDEX_PARAMS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `INDEX_PARAMS` (
+ `INDEX_ID` bigint(20) NOT NULL,
+ `PARAM_KEY` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PARAM_VALUE` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`INDEX_ID`,`PARAM_KEY`),
+ KEY `INDEX_PARAMS_N49` (`INDEX_ID`),
+ CONSTRAINT `INDEX_PARAMS_FK1` FOREIGN KEY (`INDEX_ID`) REFERENCES `IDXS` (`INDEX_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `NUCLEUS_TABLES`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `NUCLEUS_TABLES` (
+ `CLASS_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `TABLE_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `TYPE` varchar(4) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `OWNER` varchar(2) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `VERSION` varchar(20) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `INTERFACE_NAME` varchar(255) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`CLASS_NAME`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `PARTITIONS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `PARTITIONS` (
+ `PART_ID` bigint(20) NOT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `LAST_ACCESS_TIME` int(11) NOT NULL,
+ `PART_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `SD_ID` bigint(20) DEFAULT NULL,
+ `TBL_ID` bigint(20) DEFAULT NULL,
+ PRIMARY KEY (`PART_ID`),
+ UNIQUE KEY `UNIQUEPARTITION` (`PART_NAME`,`TBL_ID`),
+ KEY `PARTITIONS_N49` (`TBL_ID`),
+ KEY `PARTITIONS_N50` (`SD_ID`),
+ CONSTRAINT `PARTITIONS_FK1` FOREIGN KEY (`TBL_ID`) REFERENCES `TBLS` (`TBL_ID`),
+ CONSTRAINT `PARTITIONS_FK2` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `PARTITION_EVENTS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `PARTITION_EVENTS` (
+ `PART_NAME_ID` bigint(20) NOT NULL,
+ `DB_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `EVENT_TIME` bigint(20) NOT NULL,
+ `EVENT_TYPE` int(11) NOT NULL,
+ `PARTITION_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `TBL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`PART_NAME_ID`),
+ KEY `PARTITIONEVENTINDEX` (`PARTITION_NAME`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `PARTITION_KEYS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `PARTITION_KEYS` (
+ `TBL_ID` bigint(20) NOT NULL,
+ `PKEY_COMMENT` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PKEY_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PKEY_TYPE` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`TBL_ID`,`PKEY_NAME`),
+ KEY `PARTITION_KEYS_N49` (`TBL_ID`),
+ CONSTRAINT `PARTITION_KEYS_FK1` FOREIGN KEY (`TBL_ID`) REFERENCES `TBLS` (`TBL_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `PARTITION_KEY_VALS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `PARTITION_KEY_VALS` (
+ `PART_ID` bigint(20) NOT NULL,
+ `PART_KEY_VAL` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`PART_ID`,`INTEGER_IDX`),
+ KEY `PARTITION_KEY_VALS_N49` (`PART_ID`),
+ CONSTRAINT `PARTITION_KEY_VALS_FK1` FOREIGN KEY (`PART_ID`) REFERENCES `PARTITIONS` (`PART_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `PARTITION_PARAMS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `PARTITION_PARAMS` (
+ `PART_ID` bigint(20) NOT NULL,
+ `PARAM_KEY` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PARAM_VALUE` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`PART_ID`,`PARAM_KEY`),
+ KEY `PARTITION_PARAMS_N49` (`PART_ID`),
+ CONSTRAINT `PARTITION_PARAMS_FK1` FOREIGN KEY (`PART_ID`) REFERENCES `PARTITIONS` (`PART_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `PART_COL_PRIVS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `PART_COL_PRIVS` (
+ `PART_COLUMN_GRANT_ID` bigint(20) NOT NULL,
+ `COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `GRANT_OPTION` smallint(6) NOT NULL,
+ `GRANTOR` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `GRANTOR_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PART_ID` bigint(20) DEFAULT NULL,
+ `PRINCIPAL_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PART_COL_PRIV` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`PART_COLUMN_GRANT_ID`),
+ KEY `PART_COL_PRIVS_N49` (`PART_ID`),
+ KEY `PARTITIONCOLUMNPRIVILEGEINDEX` (`PART_ID`,`COLUMN_NAME`,`PRINCIPAL_NAME`,`PRINCIPAL_TYPE`,`PART_COL_PRIV`,`GRANTOR`,`GRANTOR_TYPE`),
+ CONSTRAINT `PART_COL_PRIVS_FK1` FOREIGN KEY (`PART_ID`) REFERENCES `PARTITIONS` (`PART_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `PART_PRIVS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `PART_PRIVS` (
+ `PART_GRANT_ID` bigint(20) NOT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `GRANT_OPTION` smallint(6) NOT NULL,
+ `GRANTOR` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `GRANTOR_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PART_ID` bigint(20) DEFAULT NULL,
+ `PRINCIPAL_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PART_PRIV` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`PART_GRANT_ID`),
+ KEY `PARTPRIVILEGEINDEX` (`PART_ID`,`PRINCIPAL_NAME`,`PRINCIPAL_TYPE`,`PART_PRIV`,`GRANTOR`,`GRANTOR_TYPE`),
+ KEY `PART_PRIVS_N49` (`PART_ID`),
+ CONSTRAINT `PART_PRIVS_FK1` FOREIGN KEY (`PART_ID`) REFERENCES `PARTITIONS` (`PART_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `ROLES`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `ROLES` (
+ `ROLE_ID` bigint(20) NOT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `OWNER_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `ROLE_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`ROLE_ID`),
+ UNIQUE KEY `ROLEENTITYINDEX` (`ROLE_NAME`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `ROLE_MAP`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `ROLE_MAP` (
+ `ROLE_GRANT_ID` bigint(20) NOT NULL,
+ `ADD_TIME` int(11) NOT NULL,
+ `GRANT_OPTION` smallint(6) NOT NULL,
+ `GRANTOR` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `GRANTOR_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `ROLE_ID` bigint(20) DEFAULT NULL,
+ PRIMARY KEY (`ROLE_GRANT_ID`),
+ UNIQUE KEY `USERROLEMAPINDEX` (`PRINCIPAL_NAME`,`ROLE_ID`,`GRANTOR`,`GRANTOR_TYPE`),
+ KEY `ROLE_MAP_N49` (`ROLE_ID`),
+ CONSTRAINT `ROLE_MAP_FK1` FOREIGN KEY (`ROLE_ID`) REFERENCES `ROLES` (`ROLE_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SDS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SDS` (
+ `SD_ID` bigint(20) NOT NULL,
+ `CD_ID` bigint(20) DEFAULT NULL,
+ `INPUT_FORMAT` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `IS_COMPRESSED` bit(1) NOT NULL,
+ `IS_STOREDASSUBDIRECTORIES` bit(1) NOT NULL,
+ `LOCATION` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `NUM_BUCKETS` int(11) NOT NULL,
+ `OUTPUT_FORMAT` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `SERDE_ID` bigint(20) DEFAULT NULL,
+ PRIMARY KEY (`SD_ID`),
+ KEY `SDS_N49` (`SERDE_ID`),
+ KEY `SDS_N50` (`CD_ID`),
+ CONSTRAINT `SDS_FK1` FOREIGN KEY (`SERDE_ID`) REFERENCES `SERDES` (`SERDE_ID`),
+ CONSTRAINT `SDS_FK2` FOREIGN KEY (`CD_ID`) REFERENCES `CDS` (`CD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SD_PARAMS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SD_PARAMS` (
+ `SD_ID` bigint(20) NOT NULL,
+ `PARAM_KEY` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PARAM_VALUE` MEDIUMTEXT CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`SD_ID`,`PARAM_KEY`),
+ KEY `SD_PARAMS_N49` (`SD_ID`),
+ CONSTRAINT `SD_PARAMS_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SEQUENCE_TABLE`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SEQUENCE_TABLE` (
+ `SEQUENCE_NAME` varchar(255) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `NEXT_VAL` bigint(20) NOT NULL,
+ PRIMARY KEY (`SEQUENCE_NAME`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SERDES`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SERDES` (
+ `SERDE_ID` bigint(20) NOT NULL,
+ `NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `SLIB` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`SERDE_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SERDE_PARAMS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SERDE_PARAMS` (
+ `SERDE_ID` bigint(20) NOT NULL,
+ `PARAM_KEY` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PARAM_VALUE` MEDIUMTEXT CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`SERDE_ID`,`PARAM_KEY`),
+ KEY `SERDE_PARAMS_N49` (`SERDE_ID`),
+ CONSTRAINT `SERDE_PARAMS_FK1` FOREIGN KEY (`SERDE_ID`) REFERENCES `SERDES` (`SERDE_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SKEWED_COL_NAMES`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SKEWED_COL_NAMES` (
+ `SD_ID` bigint(20) NOT NULL,
+ `SKEWED_COL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`SD_ID`,`INTEGER_IDX`),
+ KEY `SKEWED_COL_NAMES_N49` (`SD_ID`),
+ CONSTRAINT `SKEWED_COL_NAMES_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SKEWED_COL_VALUE_LOC_MAP`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SKEWED_COL_VALUE_LOC_MAP` (
+ `SD_ID` bigint(20) NOT NULL,
+ `STRING_LIST_ID_KID` bigint(20) NOT NULL,
+ `LOCATION` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`SD_ID`,`STRING_LIST_ID_KID`),
+ KEY `SKEWED_COL_VALUE_LOC_MAP_N49` (`STRING_LIST_ID_KID`),
+ KEY `SKEWED_COL_VALUE_LOC_MAP_N50` (`SD_ID`),
+ CONSTRAINT `SKEWED_COL_VALUE_LOC_MAP_FK2` FOREIGN KEY (`STRING_LIST_ID_KID`) REFERENCES `SKEWED_STRING_LIST` (`STRING_LIST_ID`),
+ CONSTRAINT `SKEWED_COL_VALUE_LOC_MAP_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SKEWED_STRING_LIST`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SKEWED_STRING_LIST` (
+ `STRING_LIST_ID` bigint(20) NOT NULL,
+ PRIMARY KEY (`STRING_LIST_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SKEWED_STRING_LIST_VALUES`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SKEWED_STRING_LIST_VALUES` (
+ `STRING_LIST_ID` bigint(20) NOT NULL,
+ `STRING_LIST_VALUE` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`STRING_LIST_ID`,`INTEGER_IDX`),
+ KEY `SKEWED_STRING_LIST_VALUES_N49` (`STRING_LIST_ID`),
+ CONSTRAINT `SKEWED_STRING_LIST_VALUES_FK1` FOREIGN KEY (`STRING_LIST_ID`) REFERENCES `SKEWED_STRING_LIST` (`STRING_LIST_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SKEWED_VALUES`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SKEWED_VALUES` (
+ `SD_ID_OID` bigint(20) NOT NULL,
+ `STRING_LIST_ID_EID` bigint(20) NOT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`SD_ID_OID`,`INTEGER_IDX`),
+ KEY `SKEWED_VALUES_N50` (`SD_ID_OID`),
+ KEY `SKEWED_VALUES_N49` (`STRING_LIST_ID_EID`),
+ CONSTRAINT `SKEWED_VALUES_FK2` FOREIGN KEY (`STRING_LIST_ID_EID`) REFERENCES `SKEWED_STRING_LIST` (`STRING_LIST_ID`),
+ CONSTRAINT `SKEWED_VALUES_FK1` FOREIGN KEY (`SD_ID_OID`) REFERENCES `SDS` (`SD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `SORT_COLS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `SORT_COLS` (
+ `SD_ID` bigint(20) NOT NULL,
+ `COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `ORDER` int(11) NOT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`SD_ID`,`INTEGER_IDX`),
+ KEY `SORT_COLS_N49` (`SD_ID`),
+ CONSTRAINT `SORT_COLS_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `TABLE_PARAMS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `TABLE_PARAMS` (
+ `TBL_ID` bigint(20) NOT NULL,
+ `PARAM_KEY` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PARAM_VALUE` MEDIUMTEXT CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`TBL_ID`,`PARAM_KEY`),
+ KEY `TABLE_PARAMS_N49` (`TBL_ID`),
+ CONSTRAINT `TABLE_PARAMS_FK1` FOREIGN KEY (`TBL_ID`) REFERENCES `TBLS` (`TBL_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `TBLS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `TBLS` (
+ `TBL_ID` bigint(20) NOT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `DB_ID` bigint(20) DEFAULT NULL,
+ `LAST_ACCESS_TIME` int(11) NOT NULL,
+ `OWNER` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `RETENTION` int(11) NOT NULL,
+ `SD_ID` bigint(20) DEFAULT NULL,
+ `TBL_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `TBL_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `VIEW_EXPANDED_TEXT` mediumtext,
+ `VIEW_ORIGINAL_TEXT` mediumtext,
+ `IS_REWRITE_ENABLED` bit(1) NOT NULL DEFAULT 0,
+ PRIMARY KEY (`TBL_ID`),
+ UNIQUE KEY `UNIQUETABLE` (`TBL_NAME`,`DB_ID`),
+ KEY `TBLS_N50` (`SD_ID`),
+ KEY `TBLS_N49` (`DB_ID`),
+ CONSTRAINT `TBLS_FK1` FOREIGN KEY (`SD_ID`) REFERENCES `SDS` (`SD_ID`),
+ CONSTRAINT `TBLS_FK2` FOREIGN KEY (`DB_ID`) REFERENCES `DBS` (`DB_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `TBL_COL_PRIVS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `TBL_COL_PRIVS` (
+ `TBL_COLUMN_GRANT_ID` bigint(20) NOT NULL,
+ `COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `GRANT_OPTION` smallint(6) NOT NULL,
+ `GRANTOR` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `GRANTOR_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `TBL_COL_PRIV` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `TBL_ID` bigint(20) DEFAULT NULL,
+ PRIMARY KEY (`TBL_COLUMN_GRANT_ID`),
+ KEY `TABLECOLUMNPRIVILEGEINDEX` (`TBL_ID`,`COLUMN_NAME`,`PRINCIPAL_NAME`,`PRINCIPAL_TYPE`,`TBL_COL_PRIV`,`GRANTOR`,`GRANTOR_TYPE`),
+ KEY `TBL_COL_PRIVS_N49` (`TBL_ID`),
+ CONSTRAINT `TBL_COL_PRIVS_FK1` FOREIGN KEY (`TBL_ID`) REFERENCES `TBLS` (`TBL_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `TBL_PRIVS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `TBL_PRIVS` (
+ `TBL_GRANT_ID` bigint(20) NOT NULL,
+ `CREATE_TIME` int(11) NOT NULL,
+ `GRANT_OPTION` smallint(6) NOT NULL,
+ `GRANTOR` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `GRANTOR_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `PRINCIPAL_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `TBL_PRIV` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `TBL_ID` bigint(20) DEFAULT NULL,
+ PRIMARY KEY (`TBL_GRANT_ID`),
+ KEY `TBL_PRIVS_N49` (`TBL_ID`),
+ KEY `TABLEPRIVILEGEINDEX` (`TBL_ID`,`PRINCIPAL_NAME`,`PRINCIPAL_TYPE`,`TBL_PRIV`,`GRANTOR`,`GRANTOR_TYPE`),
+ CONSTRAINT `TBL_PRIVS_FK1` FOREIGN KEY (`TBL_ID`) REFERENCES `TBLS` (`TBL_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `TAB_COL_STATS`
+--
+CREATE TABLE IF NOT EXISTS `TAB_COL_STATS` (
+ `CS_ID` bigint(20) NOT NULL,
+ `DB_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `TABLE_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `COLUMN_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `TBL_ID` bigint(20) NOT NULL,
+ `LONG_LOW_VALUE` bigint(20),
+ `LONG_HIGH_VALUE` bigint(20),
+ `DOUBLE_HIGH_VALUE` double(53,4),
+ `DOUBLE_LOW_VALUE` double(53,4),
+ `BIG_DECIMAL_LOW_VALUE` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin,
+ `BIG_DECIMAL_HIGH_VALUE` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin,
+ `NUM_NULLS` bigint(20) NOT NULL,
+ `NUM_DISTINCTS` bigint(20),
+ `AVG_COL_LEN` double(53,4),
+ `MAX_COL_LEN` bigint(20),
+ `NUM_TRUES` bigint(20),
+ `NUM_FALSES` bigint(20),
+ `LAST_ANALYZED` bigint(20) NOT NULL,
+ PRIMARY KEY (`CS_ID`),
+ CONSTRAINT `TAB_COL_STATS_FK` FOREIGN KEY (`TBL_ID`) REFERENCES `TBLS` (`TBL_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table `PART_COL_STATS`
+--
+CREATE TABLE IF NOT EXISTS `PART_COL_STATS` (
+ `CS_ID` bigint(20) NOT NULL,
+ `DB_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `TABLE_NAME` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PARTITION_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `COLUMN_TYPE` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `PART_ID` bigint(20) NOT NULL,
+ `LONG_LOW_VALUE` bigint(20),
+ `LONG_HIGH_VALUE` bigint(20),
+ `DOUBLE_HIGH_VALUE` double(53,4),
+ `DOUBLE_LOW_VALUE` double(53,4),
+ `BIG_DECIMAL_LOW_VALUE` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin,
+ `BIG_DECIMAL_HIGH_VALUE` varchar(4000) CHARACTER SET latin1 COLLATE latin1_bin,
+ `NUM_NULLS` bigint(20) NOT NULL,
+ `NUM_DISTINCTS` bigint(20),
+ `AVG_COL_LEN` double(53,4),
+ `MAX_COL_LEN` bigint(20),
+ `NUM_TRUES` bigint(20),
+ `NUM_FALSES` bigint(20),
+ `LAST_ANALYZED` bigint(20) NOT NULL,
+ PRIMARY KEY (`CS_ID`),
+ CONSTRAINT `PART_COL_STATS_FK` FOREIGN KEY (`PART_ID`) REFERENCES `PARTITIONS` (`PART_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE INDEX PCS_STATS_IDX ON PART_COL_STATS (DB_NAME,TABLE_NAME,COLUMN_NAME,PARTITION_NAME) USING BTREE;
+
+--
+-- Table structure for table `TYPES`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `TYPES` (
+ `TYPES_ID` bigint(20) NOT NULL,
+ `TYPE_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `TYPE1` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `TYPE2` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ PRIMARY KEY (`TYPES_ID`),
+ UNIQUE KEY `UNIQUE_TYPE` (`TYPE_NAME`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+/*!40101 SET character_set_client = @saved_cs_client */;
+
+--
+-- Table structure for table `TYPE_FIELDS`
+--
+
+/*!40101 SET @saved_cs_client = @@character_set_client */;
+/*!40101 SET character_set_client = utf8 */;
+CREATE TABLE IF NOT EXISTS `TYPE_FIELDS` (
+ `TYPE_NAME` bigint(20) NOT NULL,
+ `COMMENT` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
+ `FIELD_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `FIELD_TYPE` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
+ `INTEGER_IDX` int(11) NOT NULL,
+ PRIMARY KEY (`TYPE_NAME`,`FIELD_NAME`),
+ KEY `TYPE_FIELDS_N49` (`TYPE_NAME`),
+ CONSTRAINT `TYPE_FIELDS_FK1` FOREIGN KEY (`TYPE_NAME`) REFERENCES `TYPES` (`TYPES_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+-- Table `MASTER_KEYS` for classes [org.apache.hadoop.hive.metastore.model.MMasterKey]
+CREATE TABLE IF NOT EXISTS `MASTER_KEYS`
+(
+ `KEY_ID` INTEGER NOT NULL AUTO_INCREMENT,
+ `MASTER_KEY` VARCHAR(767) BINARY NULL,
+ PRIMARY KEY (`KEY_ID`)
+) ENGINE=INNODB DEFAULT CHARSET=latin1;
+
+-- Table `DELEGATION_TOKENS` for classes [org.apache.hadoop.hive.metastore.model.MDelegationToken]
+CREATE TABLE IF NOT EXISTS `DELEGATION_TOKENS`
+(
+ `TOKEN_IDENT` VARCHAR(767) BINARY NOT NULL,
+ `TOKEN` VARCHAR(767) BINARY NULL,
+ PRIMARY KEY (`TOKEN_IDENT`)
+) ENGINE=INNODB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for VERSION
+--
+CREATE TABLE IF NOT EXISTS `VERSION` (
+ `VER_ID` BIGINT NOT NULL,
+ `SCHEMA_VERSION` VARCHAR(127) NOT NULL,
+ `VERSION_COMMENT` VARCHAR(255),
+ PRIMARY KEY (`VER_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table FUNCS
+--
+CREATE TABLE IF NOT EXISTS `FUNCS` (
+ `FUNC_ID` BIGINT(20) NOT NULL,
+ `CLASS_NAME` VARCHAR(4000) CHARACTER SET latin1 COLLATE latin1_bin,
+ `CREATE_TIME` INT(11) NOT NULL,
+ `DB_ID` BIGINT(20),
+ `FUNC_NAME` VARCHAR(128) CHARACTER SET latin1 COLLATE latin1_bin,
+ `FUNC_TYPE` INT(11) NOT NULL,
+ `OWNER_NAME` VARCHAR(128) CHARACTER SET latin1 COLLATE latin1_bin,
+ `OWNER_TYPE` VARCHAR(10) CHARACTER SET latin1 COLLATE latin1_bin,
+ PRIMARY KEY (`FUNC_ID`),
+ UNIQUE KEY `UNIQUEFUNCTION` (`FUNC_NAME`, `DB_ID`),
+ KEY `FUNCS_N49` (`DB_ID`),
+ CONSTRAINT `FUNCS_FK1` FOREIGN KEY (`DB_ID`) REFERENCES `DBS` (`DB_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+--
+-- Table structure for table FUNC_RU
+--
+CREATE TABLE IF NOT EXISTS `FUNC_RU` (
+ `FUNC_ID` BIGINT(20) NOT NULL,
+ `RESOURCE_TYPE` INT(11) NOT NULL,
+ `RESOURCE_URI` VARCHAR(4000) CHARACTER SET latin1 COLLATE latin1_bin,
+ `INTEGER_IDX` INT(11) NOT NULL,
+ PRIMARY KEY (`FUNC_ID`, `INTEGER_IDX`),
+ CONSTRAINT `FUNC_RU_FK1` FOREIGN KEY (`FUNC_ID`) REFERENCES `FUNCS` (`FUNC_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE IF NOT EXISTS `NOTIFICATION_LOG`
+(
+ `NL_ID` BIGINT(20) NOT NULL,
+ `EVENT_ID` BIGINT(20) NOT NULL,
+ `EVENT_TIME` INT(11) NOT NULL,
+ `EVENT_TYPE` varchar(32) NOT NULL,
+ `DB_NAME` varchar(128),
+ `TBL_NAME` varchar(256),
+ `MESSAGE` longtext,
+ `MESSAGE_FORMAT` varchar(16),
+ PRIMARY KEY (`NL_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE IF NOT EXISTS `NOTIFICATION_SEQUENCE`
+(
+ `NNI_ID` BIGINT(20) NOT NULL,
+ `NEXT_EVENT_ID` BIGINT(20) NOT NULL,
+ PRIMARY KEY (`NNI_ID`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE IF NOT EXISTS `KEY_CONSTRAINTS`
+(
+ `CHILD_CD_ID` BIGINT,
+ `CHILD_INTEGER_IDX` INT(11),
+ `CHILD_TBL_ID` BIGINT,
+ `PARENT_CD_ID` BIGINT NOT NULL,
+ `PARENT_INTEGER_IDX` INT(11) NOT NULL,
+ `PARENT_TBL_ID` BIGINT NOT NULL,
+ `POSITION` BIGINT NOT NULL,
+ `CONSTRAINT_NAME` VARCHAR(400) NOT NULL,
+ `CONSTRAINT_TYPE` SMALLINT(6) NOT NULL,
+ `UPDATE_RULE` SMALLINT(6),
+ `DELETE_RULE` SMALLINT(6),
+ `ENABLE_VALIDATE_RELY` SMALLINT(6) NOT NULL,
+ PRIMARY KEY (`CONSTRAINT_NAME`, `POSITION`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE INDEX `CONSTRAINTS_PARENT_TABLE_ID_INDEX` ON KEY_CONSTRAINTS (`PARENT_TBL_ID`) USING BTREE;
+
+--
+-- Tables for transaction management
+--
+
+CREATE TABLE TXNS (
+ TXN_ID bigint PRIMARY KEY,
+ TXN_STATE char(1) NOT NULL,
+ TXN_STARTED bigint NOT NULL,
+ TXN_LAST_HEARTBEAT bigint NOT NULL,
+ TXN_USER varchar(128) NOT NULL,
+ TXN_HOST varchar(128) NOT NULL,
+ TXN_AGENT_INFO varchar(128),
+ TXN_META_INFO varchar(128),
+ TXN_HEARTBEAT_COUNT int
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE TXN_COMPONENTS (
+ TC_TXNID bigint NOT NULL,
+ TC_DATABASE varchar(128) NOT NULL,
+ TC_TABLE varchar(128) NOT NULL,
+ TC_PARTITION varchar(767),
+ TC_OPERATION_TYPE char(1) NOT NULL,
+ FOREIGN KEY (TC_TXNID) REFERENCES TXNS (TXN_ID)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE INDEX TC_TXNID_INDEX ON TXN_COMPONENTS (TC_TXNID);
+
+CREATE TABLE COMPLETED_TXN_COMPONENTS (
+ CTC_TXNID bigint NOT NULL,
+ CTC_DATABASE varchar(128) NOT NULL,
+ CTC_TABLE varchar(256),
+ CTC_PARTITION varchar(767)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE NEXT_TXN_ID (
+ NTXN_NEXT bigint NOT NULL
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+INSERT INTO NEXT_TXN_ID VALUES(1);
+
+CREATE TABLE HIVE_LOCKS (
+ HL_LOCK_EXT_ID bigint NOT NULL,
+ HL_LOCK_INT_ID bigint NOT NULL,
+ HL_TXNID bigint,
+ HL_DB varchar(128) NOT NULL,
+ HL_TABLE varchar(128),
+ HL_PARTITION varchar(767),
+ HL_LOCK_STATE char(1) not null,
+ HL_LOCK_TYPE char(1) not null,
+ HL_LAST_HEARTBEAT bigint NOT NULL,
+ HL_ACQUIRED_AT bigint,
+ HL_USER varchar(128) NOT NULL,
+ HL_HOST varchar(128) NOT NULL,
+ HL_HEARTBEAT_COUNT int,
+ HL_AGENT_INFO varchar(128),
+ HL_BLOCKEDBY_EXT_ID bigint,
+ HL_BLOCKEDBY_INT_ID bigint,
+ PRIMARY KEY(HL_LOCK_EXT_ID, HL_LOCK_INT_ID),
+ KEY HIVE_LOCK_TXNID_INDEX (HL_TXNID)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE INDEX HL_TXNID_IDX ON HIVE_LOCKS (HL_TXNID);
+
+CREATE TABLE NEXT_LOCK_ID (
+ NL_NEXT bigint NOT NULL
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+INSERT INTO NEXT_LOCK_ID VALUES(1);
+
+CREATE TABLE COMPACTION_QUEUE (
+ CQ_ID bigint PRIMARY KEY,
+ CQ_DATABASE varchar(128) NOT NULL,
+ CQ_TABLE varchar(128) NOT NULL,
+ CQ_PARTITION varchar(767),
+ CQ_STATE char(1) NOT NULL,
+ CQ_TYPE char(1) NOT NULL,
+ CQ_TBLPROPERTIES varchar(2048),
+ CQ_WORKER_ID varchar(128),
+ CQ_START bigint,
+ CQ_RUN_AS varchar(128),
+ CQ_HIGHEST_TXN_ID bigint,
+ CQ_META_INFO varbinary(2048),
+ CQ_HADOOP_JOB_ID varchar(32)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE COMPLETED_COMPACTIONS (
+ CC_ID bigint PRIMARY KEY,
+ CC_DATABASE varchar(128) NOT NULL,
+ CC_TABLE varchar(128) NOT NULL,
+ CC_PARTITION varchar(767),
+ CC_STATE char(1) NOT NULL,
+ CC_TYPE char(1) NOT NULL,
+ CC_TBLPROPERTIES varchar(2048),
+ CC_WORKER_ID varchar(128),
+ CC_START bigint,
+ CC_END bigint,
+ CC_RUN_AS varchar(128),
+ CC_HIGHEST_TXN_ID bigint,
+ CC_META_INFO varbinary(2048),
+ CC_HADOOP_JOB_ID varchar(32)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE NEXT_COMPACTION_QUEUE_ID (
+ NCQ_NEXT bigint NOT NULL
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+INSERT INTO NEXT_COMPACTION_QUEUE_ID VALUES(1);
+
+CREATE TABLE AUX_TABLE (
+ MT_KEY1 varchar(128) NOT NULL,
+ MT_KEY2 bigint NOT NULL,
+ MT_COMMENT varchar(255),
+ PRIMARY KEY(MT_KEY1, MT_KEY2)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE WRITE_SET (
+ WS_DATABASE varchar(128) NOT NULL,
+ WS_TABLE varchar(128) NOT NULL,
+ WS_PARTITION varchar(767),
+ WS_TXNID bigint NOT NULL,
+ WS_COMMIT_ID bigint NOT NULL,
+ WS_OPERATION_TYPE char(1) NOT NULL
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+-- -----------------------------------------------------------------
+-- Record schema version. Should be the last step in the init script
+-- -----------------------------------------------------------------
+INSERT INTO VERSION (VER_ID, SCHEMA_VERSION, VERSION_COMMENT) VALUES (1, '2.3.0', 'Hive release version 2.3.0');
+
+/*!40101 SET character_set_client = @saved_cs_client */;
+/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
+
+/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
+/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
+/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
+/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
+/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
+/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
+/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
+
+-- Dump completed on 2012-08-23 0:56:31
diff --git a/pom.xml b/pom.xml
index 5465cd7..dc9ad25 100644
--- a/pom.xml
+++ b/pom.xml
@@ -122,7 +122,7 @@
<atomix.version>3.0.0-rc4</atomix.version>
<spark.scala.version>2.11.8</spark.scala.version>
<spark.scala.binary.version>2.11</spark.scala.binary.version>
- <hive.version>2.1.1</hive.version>
+ <hive.version>2.3.6</hive.version>
<!-- Submarine on Kubernetes -->
<k8s.client-java.version>6.0.1</k8s.client-java.version>
<jersey.test-framework>2.27</jersey.test-framework>
diff --git a/submarine-commons/commons-metastore/pom.xml b/submarine-commons/commons-metastore/pom.xml
index 81935ed..4118545 100644
--- a/submarine-commons/commons-metastore/pom.xml
+++ b/submarine-commons/commons-metastore/pom.xml
@@ -36,6 +36,12 @@
<dependencies>
<dependency>
+ <groupId>org.apache.submarine</groupId>
+ <artifactId>commons-utils</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>${hive.version}</version>
@@ -96,8 +102,37 @@
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.httpcomponents</groupId>
+ <artifactId>httpcore</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.httpcomponents</groupId>
+ <artifactId>httpclient</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.zookeeper</groupId>
+ <artifactId>zookeeper</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-1.2-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ </exclusion>
</exclusions>
</dependency>
+
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-metastore</artifactId>
@@ -199,6 +234,60 @@
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>io.netty</groupId>
+ <artifactId>netty</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ <version>${slf4j.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <version>${junit.version}</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>mysql</groupId>
+ <artifactId>mysql-connector-java</artifactId>
+ <version>${mysql-connector-java.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-auth</artifactId>
+ <version>${hadoop.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>org.apache.httpcomponents</groupId>
+ <artifactId>httpclient</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.apache.curator</groupId>
+ <artifactId>curator-framework</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ </exclusion>
</exclusions>
</dependency>
</dependencies>
@@ -208,6 +297,52 @@
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <version>${plugin.shade.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <outputFile>target/submarine-${project.artifactId}-${project.version}-shade.jar</outputFile>
+ <filters>
+ <filter>
+ <artifact>*:*</artifact>
+ <excludes>
+ <exclude>META-INF/*.SF</exclude>
+ <exclude>META-INF/*.DSA</exclude>
+ <exclude>META-INF/*.RSA</exclude>
+ </excludes>
+ </filter>
+ </filters>
+ <relocations>
+ <relocation>
+ <pattern>org</pattern>
+ <shadedPattern>${shaded.dependency.prefix}.org</shadedPattern>
+ <excludes>
+ <exclude>org/apache/submarine/*</exclude>
+ <exclude>org/apache/submarine/**/*</exclude>
+ <exclude>org/slf4j/*</exclude>
+ <exclude>org/slf4j/**/*</exclude>
+ <exclude>org/apache/log4j/*</exclude>
+ <exclude>org/apache/log4j/**/*</exclude>
+ <exclude>**/pom.xml</exclude>
+ </excludes>
+ </relocation>
+ <relocation>
+ <pattern>com.google</pattern>
+ <shadedPattern>${shaded.dependency.prefix}.com.google</shadedPattern>
+ </relocation>
+ </relocations>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<configuration>
<skip>false</skip>
diff --git a/submarine-commons/commons-metastore/src/main/java/org/apache/submarine/commons/metastore/SubmarineMetaStore.java b/submarine-commons/commons-metastore/src/main/java/org/apache/submarine/commons/metastore/SubmarineMetaStore.java
index 296bb88..56d0c4b 100644
--- a/submarine-commons/commons-metastore/src/main/java/org/apache/submarine/commons/metastore/SubmarineMetaStore.java
+++ b/submarine-commons/commons-metastore/src/main/java/org/apache/submarine/commons/metastore/SubmarineMetaStore.java
@@ -16,8 +16,191 @@
*/
package org.apache.submarine.commons.metastore;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.RawStore;
+import org.apache.hadoop.hive.metastore.RawStoreProxy;
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.Index;
+import org.apache.hadoop.hive.metastore.api.InvalidInputException;
+import org.apache.hadoop.hive.metastore.api.InvalidObjectException;
+import org.apache.hadoop.hive.metastore.api.MetaException;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.Partition;
+import org.apache.hadoop.hive.metastore.api.SQLForeignKey;
+import org.apache.hadoop.hive.metastore.api.SQLPrimaryKey;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.metastore.api.TableMeta;
+import org.apache.submarine.commons.utils.SubmarineConfiguration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+
public class SubmarineMetaStore {
- public static void main(String[] args) {
+ private static final Logger LOG = LoggerFactory.getLogger(SubmarineMetaStore.class);
+
+ private RawStore rs = null;
+
+ public SubmarineMetaStore(SubmarineConfiguration submarineConf) {
+ LOG.info("JdbcDriverClassName = {}", submarineConf.getJdbcDriverClassName());
+ LOG.info("MetastoreJdbcUrl = {}", submarineConf.getMetastoreJdbcUrl());
+ LOG.info("MetastoreJdbcUserName = {}", submarineConf.getMetastoreJdbcUserName());
+ LOG.info("MetastoreJdbcPassword = {}", submarineConf.getMetastoreJdbcPassword());
+
+ HiveConf conf = new HiveConf();
+ conf.setVar(HiveConf.ConfVars.METASTORE_CONNECTION_DRIVER, submarineConf.getJdbcDriverClassName());
+ conf.setVar(HiveConf.ConfVars.METASTORECONNECTURLKEY, submarineConf.getMetastoreJdbcUrl());
+ conf.setVar(HiveConf.ConfVars.METASTORE_CONNECTION_USER_NAME, submarineConf.getMetastoreJdbcUserName());
+ conf.setVar(HiveConf.ConfVars.METASTOREPWD, submarineConf.getMetastoreJdbcPassword());
+ // Id can be set to any int value
+ try {
+ this.rs = RawStoreProxy.getProxy(conf, conf,
+ conf.getVar(HiveConf.ConfVars.METASTORE_RAW_STORE_IMPL), 1);
+ } catch (MetaException e) {
+ LOG.error(e.getMessage(), e);
+ }
+ }
+
+ /**
+ * Gets total number of databases.
+ * @return
+ * @throws MetaException
+ */
+ public int getDatabaseCount() throws MetaException {
+ int databaseCount = rs.getDatabaseCount();
+ return databaseCount;
+ }
+
+ public List<String> getAllDatabases() throws MetaException {
+ List<String> databases = rs.getAllDatabases();
+ return databases;
+ }
+
+ public Database getDatabase(String databaseName) throws NoSuchObjectException {
+ Database database = rs.getDatabase(databaseName);
+ return database;
+ }
+
+ /**
+ * Gets total number of tables.
+ * @return
+ * @throws MetaException
+ */
+ public int getTableCount() throws MetaException {
+ int tableCount = rs.getTableCount();
+ return tableCount;
+ }
+
+ public List<String> getAllTables(String databaseName) throws MetaException {
+ List<String> tables = rs.getAllTables(databaseName);
+ return tables;
+ }
+
+ public Table getTable(String databaseName, String tableName) throws MetaException {
+ Table table = rs.getTable(databaseName, tableName);
+ return table;
+ }
+
+ public void createDatabase(Database db) throws InvalidObjectException, MetaException {
+ rs.createDatabase(db);
+ }
+
+ /**
+ * Alter the database object in metastore. Currently only the parameters
+ * of the database or the owner can be changed.
+ * @param dbName the database name
+ * @param db the Hive Database object
+ * @throws MetaException
+ * @throws NoSuchObjectException
+ */
+ public boolean alterDatabase(String dbName, Database db) throws NoSuchObjectException, MetaException {
+ boolean result = rs.alterDatabase(dbName, db);
+ return result;
+ }
+
+ public boolean dropDatabase(String dbName) throws NoSuchObjectException, MetaException {
+ boolean result = rs.dropDatabase(dbName);
+ return result;
+ }
+
+ public void createTable(Table table) throws InvalidObjectException, MetaException {
+ rs.createTable(table);
+ }
+
+ public boolean dropTable(String dbName, String tableName)
+ throws MetaException, InvalidObjectException, NoSuchObjectException, InvalidInputException {
+ boolean result = rs.dropTable(dbName, tableName);
+ return result;
+ }
+
+ public int getPartitionCount() throws MetaException {
+ int partitionCount = rs.getPartitionCount();
+ return partitionCount;
+ }
+
+ public Partition getPartition(String dbName, String tableName, List<String> partVals)
+ throws NoSuchObjectException, MetaException {
+ Partition partition = rs.getPartition(dbName, tableName, partVals);
+ return partition;
+ }
+
+ public List<TableMeta> getTableMeta(String dbNames, String tableNames, List<String> tableTypes)
+ throws MetaException {
+ List<TableMeta> tableMetas = rs.getTableMeta(dbNames, tableNames, tableTypes);
+ return tableMetas;
+ }
+
+ public void createTableWithConstraints(Table tbl,
+ List<SQLPrimaryKey> primaryKeys,
+ List<SQLForeignKey> foreignKeys)
+ throws InvalidObjectException, MetaException {
+ rs.createTableWithConstraints(tbl, primaryKeys, foreignKeys);
+ }
+
+ public boolean addPartitions(String dbName, String tblName, List<Partition> parts)
+ throws InvalidObjectException, MetaException {
+ boolean result = rs.addPartitions(dbName, tblName, parts);
+ return result;
+ }
+
+ public boolean dropPartition(String dbName, String tableName, List<String> partVals)
+ throws MetaException, NoSuchObjectException, InvalidObjectException,
+ InvalidInputException {
+ boolean result = rs.dropPartition(dbName, tableName, partVals);
+ return result;
+ }
+
+ public void alterTable(String dbname, String tableName, Table newTable)
+ throws InvalidObjectException, MetaException {
+ rs.alterTable(dbname, tableName, newTable);
+ }
+
+ public void alterIndex(String dbname, String baseTblName, String indexName, Index newIndex)
+ throws InvalidObjectException, MetaException {
+ rs.alterIndex(dbname, baseTblName, indexName, newIndex);
+ }
+
+ public void alterPartition(String dbname, String tableName, List<String> partVals, Partition newPart)
+ throws InvalidObjectException, MetaException {
+ rs.alterPartition(dbname, tableName, partVals, newPart);
+ }
+
+ public void addPrimaryKeys(List<SQLPrimaryKey> pks) throws InvalidObjectException, MetaException {
+ rs.addPrimaryKeys(pks);
+ }
+
+ public boolean addIndex(Index index) throws InvalidObjectException, MetaException {
+ boolean result = rs.addIndex(index);
+ return result;
+ }
+
+ public boolean dropIndex(String dbName, String origTableName, String indexName) throws MetaException {
+ boolean result = rs.dropIndex(dbName, origTableName, indexName);
+ return result;
+ }
+ public Index getIndex(String dbName, String origTableName, String indexName) throws MetaException {
+ Index index = rs.getIndex(dbName, origTableName, indexName);
+ return index;
}
}
diff --git a/submarine-commons/commons-metastore/src/test/java/org/apache/submarine/commons/metastore/SubmarineMetaStoreTest.java b/submarine-commons/commons-metastore/src/test/java/org/apache/submarine/commons/metastore/SubmarineMetaStoreTest.java
new file mode 100644
index 0000000..3012b33
--- /dev/null
+++ b/submarine-commons/commons-metastore/src/test/java/org/apache/submarine/commons/metastore/SubmarineMetaStoreTest.java
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.submarine.commons.metastore;
+
+import org.apache.hadoop.hive.metastore.api.Database;
+import org.apache.hadoop.hive.metastore.api.FieldSchema;
+import org.apache.hadoop.hive.metastore.api.InvalidObjectException;
+import org.apache.hadoop.hive.metastore.api.MetaException;
+import org.apache.hadoop.hive.metastore.api.NoSuchObjectException;
+import org.apache.hadoop.hive.metastore.api.PrincipalType;
+import org.apache.hadoop.hive.metastore.api.SerDeInfo;
+import org.apache.hadoop.hive.metastore.api.StorageDescriptor;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.submarine.commons.utils.SubmarineConfiguration;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.Assert.assertEquals;
+
+public class SubmarineMetaStoreTest {
+ private static final Logger LOG = LoggerFactory.getLogger(SubmarineMetaStoreTest.class);
+ private static final SubmarineConfiguration submarineConf = SubmarineConfiguration.newInstance();
+ private SubmarineMetaStore submarineMetaStore = new SubmarineMetaStore(submarineConf);
+
+ static {
+ submarineConf.setMetastoreJdbcUrl("jdbc:mysql://127.0.0.1:3306/metastore_test?" +
+ "useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&" +
+ "failOverReadOnly=false&zeroDateTimeBehavior=convertToNull&useSSL=false");
+ submarineConf.setMetastoreJdbcUserName("metastore_test");
+ submarineConf.setMetastoreJdbcPassword("password_test");
+ }
+
+ @Test
+ public void listTables() {
+ LOG.info("listTables >>> ");
+
+ String url = "jdbc:mysql://127.0.0.1:3306/metastore_test?" +
+ "useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&" +
+ "failOverReadOnly=false&zeroDateTimeBehavior=convertToNull&useSSL=false";
+ String username = "metastore_test";
+ String password = "password_test";
+ boolean flag = false;
+ Connection con = null;
+ Statement stmt = null;
+ try {
+ con = DriverManager.getConnection(url, username, password);
+ stmt = con.createStatement();
+ String sql = "show tables";
+ LOG.info(">>>>> sql:" + sql);
+ ResultSet rs = stmt.executeQuery(sql);
+ LOG.info("rs:" + rs);
+
+ while (rs.next()) {
+ String pass = rs.getString(1);
+ LOG.info("table:" + pass);
+ }
+ } catch (SQLException se) {
+ LOG.error(se.getMessage(), se);
+ }
+
+ LOG.info("listTables <<< ");
+ }
+
+ @Before
+ public void createDatabase() throws InvalidObjectException, MetaException {
+ listTables();
+
+ Database database = new Database();
+ database.setName("testdb");
+ database.setDescription("testdb");
+ database.setLocationUri("hdfs://mycluster/user/hive/warehouse/testdb.db");
+ Map map = new HashMap();
+ map.put("key", "value");
+ database.setParameters(map);
+ database.setOwnerName("root");
+ database.setOwnerType(PrincipalType.USER);
+ submarineMetaStore.createDatabase(database);
+ assertEquals(1, submarineMetaStore.getDatabaseCount());
+
+ Table table = new Table();
+ table.setTableName("testtable");
+ table.setDbName("testdb");
+ table.setOwner("root");
+ table.setCreateTime((int) new Date().getTime() / 1000);
+ table.setLastAccessTime((int) new Date().getTime() / 1000);
+ table.setRetention(0);
+ StorageDescriptor sd = new StorageDescriptor();
+ List<FieldSchema> fieldSchemas = new ArrayList<>();
+ FieldSchema fieldSchema = new FieldSchema();
+ fieldSchema.setName("a");
+ fieldSchema.setType("int");
+ fieldSchema.setComment("a");
+ fieldSchemas.add(fieldSchema);
+ sd.setCols(fieldSchemas);
+ sd.setLocation("hdfs://mycluster/user/hive/warehouse/testdb.db/testtable");
+ sd.setInputFormat("org.apache.hadoop.mapred.TextInputFormat");
+ sd.setOutputFormat("org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat");
+ sd.setCompressed(false);
+ sd.setNumBuckets(-1);
+ SerDeInfo serdeInfo = new SerDeInfo();
+ serdeInfo.setName("test");
+ serdeInfo.setSerializationLib("org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe");
+ Map<String, String> parametersMap = new HashMap();
+ parametersMap.put("serialization.format", "|");
+ parametersMap.put("field.delim", "|");
+ serdeInfo.setParameters(parametersMap);
+ sd.setSerdeInfo(serdeInfo);
+ table.setSd(sd);
+ List<FieldSchema> partitionKeys = new ArrayList<>();
+ table.setPartitionKeys(partitionKeys);
+ Map<String, String> parameters = new HashMap<>();
+ table.setParameters(parameters);
+ String viewOriginalText = "";
+ table.setViewOriginalText(viewOriginalText);
+ String viewExpandedText = "";
+ table.setViewExpandedText(viewExpandedText);
+ String tableType = "MANAGED_TABLE";
+ table.setTableType(tableType);
+ submarineMetaStore.createTable(table);
+
+ Table tableTest = submarineMetaStore.getTable("testdb", "testtable");
+ assertEquals("testtable", tableTest.getTableName());
+ int tableCount = submarineMetaStore.getTableCount();
+ assertEquals(1, tableCount);
+ }
+
+ @After
+ public void removeAllRecord() throws Exception {
+ submarineMetaStore.dropTable("testdb", "testtable");
+ int tableCount = submarineMetaStore.getTableCount();
+ assertEquals(0, tableCount);
+
+ submarineMetaStore.dropDatabase("testdb");
+ assertEquals(0, submarineMetaStore.getDatabaseCount());
+ }
+
+ @Test
+ public void getDatabaseCount() throws InvalidObjectException, MetaException {
+ assertEquals(1, submarineMetaStore.getDatabaseCount());
+ }
+
+ @Test
+ public void getAllDatabases() throws MetaException {
+ List<String> databases = submarineMetaStore.getAllDatabases();
+ assertEquals(true, databases.contains("testdb"));
+ }
+
+ @Test
+ public void getDatabase() throws NoSuchObjectException {
+ Database database = submarineMetaStore.getDatabase("testdb");
+ assertEquals("testdb", database.getName());
+ }
+
+ @Test
+ public void getAllTables() throws MetaException {
+ List<String> tables = submarineMetaStore.getAllTables("testdb");
+ assertEquals(true, tables.contains("testtable"));
+ }
+
+ @Test
+ public void getTableCount() throws MetaException {
+ int tableCount = submarineMetaStore.getTableCount();
+ assertEquals(1, tableCount);
+ }
+
+}
diff --git a/dev-support/mini-submarine/conf/setup-mysql.sh b/submarine-commons/commons-metastore/src/test/resources/log4j.properties
old mode 100755
new mode 100644
similarity index 50%
copy from dev-support/mini-submarine/conf/setup-mysql.sh
copy to submarine-commons/commons-metastore/src/test/resources/log4j.properties
index e9743a4..3c3cb4a
--- a/dev-support/mini-submarine/conf/setup-mysql.sh
+++ b/submarine-commons/commons-metastore/src/test/resources/log4j.properties
@@ -1,4 +1,3 @@
-#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
@@ -6,7 +5,7 @@
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
-# http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
@@ -14,15 +13,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Install mariadb
-apt-get -y install mariadb-server
-service mysql start
-mysql -e "CREATE DATABASE submarine_test;"
-mysql -e "CREATE USER 'submarine_test'@'%' IDENTIFIED BY 'password_test';"
-mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'submarine_test'@'%';"
-mysql -e "use submarine_test; source /home/yarn/database/submarine.sql; source /home/yarn/database/submarine-data.sql;"
+# Root logger option
+log4j.rootLogger=INFO, stdout
-mysql -e "CREATE DATABASE submarine;"
-mysql -e "CREATE USER 'submarine'@'%' IDENTIFIED BY 'password';"
-mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'submarine'@'%';"
-mysql -e "use submarine; source /home/yarn/database/submarine.sql; source /home/yarn/database/submarine-data.sql;"
+# Direct log messages to stdout
+log4j.appender.stdout=org.apache.log4j.ConsoleAppender
+log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
+log4j.appender.stdout.layout.ConversionPattern=%5p [%d] ({%t} %F[%M]:%L) - %m%n
diff --git a/submarine-commons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfiguration.java b/submarine-commons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfiguration.java
index dc19752..d76d7d1 100644
--- a/submarine-commons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfiguration.java
+++ b/submarine-commons/commons-utils/src/main/java/org/apache/submarine/commons/utils/SubmarineConfiguration.java
@@ -231,6 +231,16 @@ public class SubmarineConfiguration extends XMLConfiguration {
return getString(ConfVars.JDBC_URL);
}
+ public String getMetastoreJdbcUrl() {
+ return getString(ConfVars.METASTORE_JDBC_URL);
+ }
+
+ @VisibleForTesting
+ public void setMetastoreJdbcUrl(String testMetastoreJdbcUrl) {
+ properties.put(ConfVars.METASTORE_JDBC_URL.getVarName(), testMetastoreJdbcUrl);
+ }
+
+
@VisibleForTesting
public void setJdbcUrl(String testJdbcUrl) {
properties.put(ConfVars.JDBC_URL.getVarName(), testJdbcUrl);
@@ -240,20 +250,38 @@ public class SubmarineConfiguration extends XMLConfiguration {
return getString(ConfVars.JDBC_USERNAME);
}
+ public String getMetastoreJdbcUserName() {
+ return getString(ConfVars.METASTORE_JDBC_USERNAME);
+ }
+
@VisibleForTesting
public void setJdbcUserName(String userName) {
properties.put(ConfVars.JDBC_USERNAME.getVarName(), userName);
}
+ @VisibleForTesting
+ public void setMetastoreJdbcUserName(String metastoreUserName) {
+ properties.put(ConfVars.METASTORE_JDBC_USERNAME.getVarName(), metastoreUserName);
+ }
+
public String getJdbcPassword() {
return getString(ConfVars.JDBC_PASSWORD);
}
+ public String getMetastoreJdbcPassword() {
+ return getString(ConfVars.METASTORE_JDBC_PASSWORD);
+ }
+
@VisibleForTesting
public void setJdbcPassword(String password) {
properties.put(ConfVars.JDBC_PASSWORD.getVarName(), password);
}
+ @VisibleForTesting
+ public void setMetastoreJdbcPassword(String metastorePassword) {
+ properties.put(ConfVars.METASTORE_JDBC_PASSWORD.getVarName(), metastorePassword);
+ }
+
public String getClusterAddress() {
return getString(ConfVars.SUBMARINE_CLUSTER_ADDR);
}
@@ -436,8 +464,14 @@ public class SubmarineConfiguration extends XMLConfiguration {
JDBC_URL("jdbc.url", "jdbc:mysql://127.0.0.1:3306/submarine" +
"?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&" +
"failOverReadOnly=false&zeroDateTimeBehavior=convertToNull&useSSL=false"),
+
JDBC_USERNAME("jdbc.username", "submarine"),
JDBC_PASSWORD("jdbc.password", "password"),
+ METASTORE_JDBC_URL("metastore.jdbc.url", "jdbc:mysql://127.0.0.1:3306/metastore" +
+ "?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&" +
+ "failOverReadOnly=false&zeroDateTimeBehavior=convertToNull&useSSL=false"),
+ METASTORE_JDBC_USERNAME("metastore.jdbc.username", "metastore"),
+ METASTORE_JDBC_PASSWORD("metastore.jdbc.password", "password"),
WORKBENCH_WEBSOCKET_MAX_TEXT_MESSAGE_SIZE(
"workbench.websocket.max.text.message.size", "1024000"),
WORKBENCH_WEB_WAR("workbench.web.war", "submarine-workbench/workbench-web/dist"),
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org