[GitHub] [spark] beliefer opened a new pull request #24575: [SPARK-27670][SQL]Add HA for HiveThriftServer2 based on HiveServer2.

[GitBox <gi...@apache.org>]

beliefer opened a new pull request #24575: [SPARK-27670][SQL]Add HA for HiveThriftServer2 based on HiveServer2.
URL: https://github.com/apache/spark/pull/24575
 
 
   ## What changes were proposed in this pull request?
   
   `HiveThriftServer2` is a multi-session managed service based on `HiveServer2`, which provides centralized management services for Hive.
   
   Since `HiveThriftServer2` itself inherits from `HiveServer2`, `HiveServer2`'s own HA solution can also support `HiveThriftServer2`.
   
   `HiveThriftServer2` does not support **HA**. Why is this?
   
   The main method of `HiveServer2` is as follows:
   ```
     public static void main(String[] args) {
       HiveConf.setLoadHiveServer2Config(true);
       try {
         ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
         ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
    
         // Omit irrelevant code
    
         // Call the executor which will execute the appropriate command based on the parsed options
         oprocResponse.getServerOptionsExecutor().execute();
       } catch (LogInitializationException e) {
         LOG.error("Error initializing log: " + e.getMessage(), e);
         System.exit(-1);
       }
     }
   ```
   The above code first creates the `ServerOptionsProcessor` object and parses the parameters. The `parse` method parses the parameters and returns the `oprocResponse` object (type is `ServerOptionsProcessorResponse`). Then the object obtained by calling the `getServerOptionsExecutor` method of `oprocResponse` is actually `StartOptionExecutor`. Finally, the execute method of `StartOptionExecutor` is called. The implementation of StartOptionExecutor is as follows:
   ```
     static class StartOptionExecutor implements ServerOptionsExecutor {
       @Override
       public void execute() {
         try {
           startHiveServer2();
         } catch (Throwable t) {
           LOG.fatal("Error starting HiveServer2", t);
           System.exit(-1);
         }
       }
     }
   ```
   It can be seen that the `execute` method of `StartOptionExecutor` actually calls the `startHiveServer2` method, and the code related to HA in the `startHiveServer2` method is as follows:
   ```
           if (hiveConf.getBoolVar(ConfVars.HIVE_SERVER2_SUPPORT_DYNAMIC_SERVICE_DISCOVERY)) {
             server.addServerInstanceToZooKeeper(hiveConf);
           }
   ```
   You can see that the `addServerInstanceToZooKeeper` method of `HiveServer2` is called. The role of the `addServerInstanceToZooKeeper` is to create a persistent parent node as the HA namespace on the specified `ZooKeeper` cluster, and create a persistent node to save the `HiveServer2` instance information to the node.
   
   But `HiveThriftServer2` can not call the `execute` method of `StartOptionExecutor` as Spark has its own logic. So I added some reflection code as to call the `addServerInstanceToZooKeeper` method of `HiveServer2`.
   
   
   ## How was this patch tested?
   
   Exists UT.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org