[GitHub] [dolphinscheduler] KingSpring commented on issue #7094: [Bug] [dolphinscheduler-common] when the first RM configured down，ds can't get the active RM correctly

[GitBox <gi...@apache.org>]

KingSpring commented on issue #7094:
URL: https://github.com/apache/dolphinscheduler/issues/7094#issuecomment-986577429


   U may refrence this:
   1.env:
   yarn ha ：hadoop47，hadoop48
   ds:1.3.9
   
   cat ./conf/common.properties 
   # resourcemanager port, the default value is 8088 if not specified
   resource.manager.httpaddress.port=8088
   
   # if resourcemanager HA is enabled, please set the HA IPs; if resourcemanager is single, keep this value empty
   yarn.resourcemanager.ha.rm.ids=hadoop47,hadoop48
   
   # if resourcemanager HA is enabled or not use resourcemanager, please keep the default value; If resourcemanager is single, you only need to replace ds1 to actual resourcemanager hostname
   yarn.application.status.address=http://ds1:%s/ws/v1/cluster/apps/%s
   
   2.problem
   [ERROR] 2021-12-06 10:07:14.343  - [taskAppId=TASK-12-56-262]:[418] - yarn applications: application_1638416574447_0083 , query status failed, exception:{}
   java.lang.NullPointerException: null
       at org.apache.dolphinscheduler.common.utils.HadoopUtils.getApplicationStatus(HadoopUtils.java:423)
       at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.isSuccessOfYarnState(AbstractCommandExecutor.java:404)
       at org.apache.dolphinscheduler.server.worker.task.AbstractCommandExecutor.run(AbstractCommandExecutor.java:230)
       at org.apache.dolphinscheduler.server.worker.task.shell.ShellTask.handle(ShellTask.java:101)
       at org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread.run(TaskExecuteThread.java:139)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
   
   
   3.propose solution
   
   org.apache.dolphinscheduler.common.utils.HadoopUtils
   public static String getAcitveRMName(String rmIds) {
   
               String[] rmIdArr = rmIds.split(Constants.COMMA);
   
               int activeResourceManagerPort = PropertyUtils.getInt(Constants.HADOOP_RESOURCE_MANAGER_HTTPADDRESS_PORT, 8088);
   
               String yarnUrl = "http://%s:" + activeResourceManagerPort + "/ws/v1/cluster/info";
   
               String state = null;
               try {
                   /**
                    * send http get request to rm1
                    */
                   state = getRMState(String.format(yarnUrl, rmIdArr[0]));
   
                   if (Constants.HADOOP_RM_STATE_ACTIVE.equals(state)) {
                       return rmIdArr[0];
                   } else  {
                       state = getRMState(String.format(yarnUrl, rmIdArr[1]));
                       if (Constants.HADOOP_RM_STATE_ACTIVE.equals(state)) {
                           return rmIdArr[1];
                       }
                   }
                   return null;
   
               } catch (Exception e) {
                   state = getRMState(String.format(yarnUrl, rmIdArr[1]));
                   if (Constants.HADOOP_RM_STATE_ACTIVE.equals(state)) {
                       return rmIdArr[1];
                   }
               }
               return null;
           }


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org