[GitHub] [hudi] xushiyan commented on a diff in pull request #7722: [HUDI-5594] Add metaserver bundle validation

["xushiyan (via GitHub)" <gi...@apache.org>]

xushiyan commented on code in PR #7722:
URL: https://github.com/apache/hudi/pull/7722#discussion_r1083244062


##########
hudi-platform-service/hudi-metaserver/hudi-metaserver-server/src/main/java/org/apache/hudi/metaserver/service/HoodieMetaserverProxyHandler.java:
##########
@@ -34,9 +34,9 @@
 public class HoodieMetaserverProxyHandler implements InvocationHandler {
   private static final Logger LOG = LogManager.getLogger(HoodieMetaserverProxyHandler.class);
 
-  private final HoodieMetaserverService metaserverService;
+  private final HoodieMetaserverGateway metaserverService;

Review Comment:
   /nit var name also



##########
packaging/bundle-validation/validate.sh:
##########
@@ -185,6 +186,42 @@ test_kafka_connect_bundle() {
     kill $ZOOKEEPER_PID $KAFKA_SERVER_PID $SCHEMA_REG_PID
 }
 
+##
+# Function to test the hudi metaserver bundles.
+#
+# env vars (defined in container):
+#   SPARK_HOME: path to the spark directory
+##
+test_hudi_metaserver_bundles () {
+    echo "::warning::validate.sh setting up hudi metaserver bundles validation"
+
+    echo "::warning::validate.sh Start hudi metaserver"
+    java -jar $JARS_DIR/metaserver.jar & local METASEVER=$!
+
+    echo "::warning::validate.sh Start hive server"
+    $DERBY_HOME/bin/startNetworkServer -h 0.0.0.0 &
+    local DERBY_PID=$!
+    $HIVE_HOME/bin/hiveserver2 --hiveconf hive.aux.jars.path=$JARS_DIR/hadoop-mr.jar &
+    local HIVE_PID=$!

Review Comment:
   we don't need hive server here as we don't need to run hiveql to validate



##########
hudi-platform-service/hudi-metaserver/hudi-metaserver-server/src/main/java/org/apache/hudi/metaserver/HoodieMetaserver.java:
##########
@@ -49,7 +49,7 @@ public class HoodieMetaserver {
   private static TServer server;
   private static Thread serverThread;
   private static volatile MetaserverStorage metaserverStorage;
-  private static HoodieMetaserverService metaserverService;
+  private static HoodieMetaserverGateway metaserverService;

Review Comment:
   /nit var name should also update



##########
packaging/hudi-metaserver-server-bundle/pom.xml:
##########
@@ -6,20 +6,66 @@
         <artifactId>hudi</artifactId>
         <groupId>org.apache.hudi</groupId>
         <version>0.13.0-SNAPSHOT</version>
+        <relativePath>../../pom.xml</relativePath>
     </parent>
     <modelVersion>4.0.0</modelVersion>
 
     <artifactId>hudi-metaserver-server-bundle</artifactId>
 
     <properties>
         <main.basedir>${project.parent.basedir}</main.basedir>
+        <h2.version>1.4.200</h2.version>
     </properties>
 
     <dependencies>
         <dependency>
             <groupId>org.apache.hudi</groupId>
             <artifactId>hudi-metaserver-server</artifactId>
             <version>${project.version}</version>
+            <scope>compile</scope>
+        </dependency>
+        <dependency>
+            <groupId>com.h2database</groupId>
+            <artifactId>h2</artifactId>
+            <version>${h2.version}</version>
+            <scope>compile</scope>
+        </dependency>
+        <!-- Logging -->
+        <dependency>
+            <groupId>org.apache.logging.log4j</groupId>
+            <artifactId>log4j-api</artifactId>
+            <version>${log4j2.version}</version>

Review Comment:
   so the logging issue resolved? care to share the findings?



##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala:
##########
@@ -110,7 +110,8 @@ class DefaultSource extends RelationProvider
     }
     log.info("Obtained hudi table path: " + tablePath)
 
-    val metaClient = HoodieTableMetaClient.builder().setConf(fs.getConf).setBasePath(tablePath).build()
+    val metaClient = HoodieTableMetaClient.builder().setProperties(optParams.asJava)
+      .setConf(fs.getConf).setBasePath(tablePath).build()

Review Comment:
   what issue are we fixing with this? this should be separated out as another PR. not mixing it with metaserver bundle  testing setup



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org