[GitHub] [hudi] vinothchandar commented on a change in pull request #1968: [HUDI-1192] Make create hive database automatically configurable

[GitBox <gi...@apache.org>]

vinothchandar commented on a change in pull request #1968:
URL: https://github.com/apache/hudi/pull/1968#discussion_r480260405



##########
File path: hudi-spark/src/main/scala/org/apache/hudi/DataSourceOptions.scala
##########
@@ -282,6 +282,7 @@ object DataSourceWriteOptions {
   val HIVE_ASSUME_DATE_PARTITION_OPT_KEY = "hoodie.datasource.hive_sync.assume_date_partitioning"
   val HIVE_USE_PRE_APACHE_INPUT_FORMAT_OPT_KEY = "hoodie.datasource.hive_sync.use_pre_apache_input_format"
   val HIVE_USE_JDBC_OPT_KEY = "hoodie.datasource.hive_sync.use_jdbc"
+  val HIVE_CREATE_DATABASE_ENABLED_OPT_KEY = "hoodie.datasource.hive_sync.create.database.enable"

Review comment:
       rename this well to `.auto.create.database` ? 

##########
File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java
##########
@@ -117,11 +117,13 @@ private void syncHoodieTable(String tableName, boolean useRealtimeInputFormat) {
     boolean tableExists = hoodieHiveClient.doesTableExist(tableName);
 
     // check if the database exists else create it
-    try {
-      hoodieHiveClient.updateHiveSQL("create database if not exists " + cfg.databaseName);
-    } catch (Exception e) {
-      // this is harmless since table creation will fail anyways, creation of DB is needed for in-memory testing
-      LOG.warn("Unable to create database", e);
+    if (cfg.enableCreateDatabase) {
+      try {
+        hoodieHiveClient.updateHiveSQL("create database if not exists " + cfg.databaseName);

Review comment:
       Should we just throw an error and exit if `enableCreateDatabase=false` and the database does not exist? 

##########
File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java
##########
@@ -71,6 +71,9 @@
   @Parameter(names = {"--use-jdbc"}, description = "Hive jdbc connect url")
   public Boolean useJdbc = true;
 
+  @Parameter(names = {"--enable-create-database"}, description = "Enable create hive database")
+  public Boolean enableCreateDatabase = false;

Review comment:
       rename: autoCreateDatabase ? 

##########
File path: hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java
##########
@@ -71,6 +71,9 @@
   @Parameter(names = {"--use-jdbc"}, description = "Hive jdbc connect url")
   public Boolean useJdbc = true;
 
+  @Parameter(names = {"--enable-create-database"}, description = "Enable create hive database")
+  public Boolean enableCreateDatabase = false;

Review comment:
       Also should the value be `true` by default to preserve existing behavior




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org