You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/02/09 16:40:26 UTC

[GitHub] [hive] belugabehr opened a new pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

belugabehr opened a new pull request #1963:
URL: https://github.com/apache/hive/pull/1963


   <!--
   Thanks for sending a pull request!  Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
     2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
     3. Ensure you have added or run the appropriate tests for your PR: 
     4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX:  Your PR title ...'.
     5. Be sure to keep the PR description updated to reflect all changes.
     6. Please write your PR title to summarize what this PR proposes.
     7. If possible, provide a concise example to reproduce the issue for a faster review.
   
   -->
   
   ### What changes were proposed in this pull request?
   <!--
   Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue. 
   If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
     1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
     2. If you fix some SQL features, you can provide some references of other DBMSes.
     3. If there is design documentation, please add the link.
     4. If there is a discussion in the mailing list, please add the link.
   -->
   
   
   ### Why are the changes needed?
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you propose a new API, clarify the use case for a new API.
     2. If you fix a bug, you can clarify why it is a bug.
   -->
   
   
   ### Does this PR introduce _any_ user-facing change?
   <!--
   Note that it means *any* user-facing change including all aspects such as the documentation fix.
   If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description, screenshot and/or a reproducable example to show the behavior difference if possible.
   If possible, please also clarify if this is a user-facing change compared to the released Hive versions or within the unreleased branches such as master.
   If no, write 'No'.
   -->
   
   
   ### How was this patch tested?
   <!--
   If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
   If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why it was difficult to add.
   -->
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r576867163



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -236,6 +239,10 @@ public int execute() {
           throw new HiveException("Operation cancelled");
         }
 
+        // Log all the info required to find the various logs for this query
+        LOG.info("HS2 Host: [{}], Query ID: [{}], Dag ID: [{}], DAG Session ID: [{}]", getHostNameIP(), queryId,

Review comment:
       @pgaref Have you had a chance to think about this topic?  What do you think about getting this in for the on-prem solutions and then modify in a subsequent ticket for multi-HS2 hosts?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r592506951



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -236,6 +239,10 @@ public int execute() {
           throw new HiveException("Operation cancelled");
         }
 
+        // Log all the info required to find the various logs for this query
+        LOG.info("HS2 Host: [{}], Query ID: [{}], Dag ID: [{}], DAG Session ID: [{}]", getHostNameIP(), queryId,

Review comment:
       @pgaref I at least re-used the `hive-common` package feature here.  Please review once more.  Let not the perfect be the enemy of the good.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r593811046



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -253,7 +259,7 @@ public int execute() {
           counters = mergedCounters;
         } catch (Exception err) {
           // Don't fail execution due to counters - just don't print summary info
-          LOG.warn("Failed to get counters. Ignoring, summary info will be incomplete. " + err, err);
+          LOG.warn("Failed to get counters. Ignoring, summary info will be incomplete.", err);

Review comment:
       Nope.  Removing this was intentional.  SLF4J will log the entire Exception information, no need to print it twice.
   
   Edit: Or, perhaps you mean missing the anchor `{}` for err.  If the last variable is an Exception, SL4J will handle it specially, no need to explicitly call it out.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] pgaref commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
pgaref commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r585566958



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -236,6 +239,10 @@ public int execute() {
           throw new HiveException("Operation cancelled");
         }
 
+        // Log all the info required to find the various logs for this query
+        LOG.info("HS2 Host: [{}], Query ID: [{}], Dag ID: [{}], DAG Session ID: [{}]", getHostNameIP(), queryId,

Review comment:
       Hey @belugabehr  -- taking another look here and seems like SessionState could do the job.
   What about SessionState.getHiveServer2Host() ? 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr merged pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr merged pull request #1963:
URL: https://github.com/apache/hive/pull/1963


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r573979578



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -236,6 +239,10 @@ public int execute() {
           throw new HiveException("Operation cancelled");
         }
 
+        // Log all the info required to find the various logs for this query
+        LOG.info("HS2 Host: [{}], Query ID: [{}], Dag ID: [{}], DAG Session ID: [{}]", getHostNameIP(), queryId,

Review comment:
       Hey @pgaref, thanks for the valuable input.
   
   So, on one hand it may seem confusing as to the need to log the host name because if you are looking at this log file then of course the hostname is already known.  However, as I understand the code, these log messages are sent  (redirected) to the client via Thrift RPC APIs.  This is helpful information for debugging for the client as it's not otherwise clear which instance of HS2 is processing the query, if for example, a load balancer is between the client and HS2.  I had thought about including some sort of HS2 unique ID as well, but I didn't find such a capability in the project already and did not want to introduce here.
   
   However, these 4 pieces of information allow a client to report a problem to the admin and allow the admin to grab all the log files: HS2 and YARN TEZ DAG logs.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] pgaref commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
pgaref commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r573937582



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -236,6 +239,10 @@ public int execute() {
           throw new HiveException("Operation cancelled");
         }
 
+        // Log all the info required to find the various logs for this query
+        LOG.info("HS2 Host: [{}], Query ID: [{}], Dag ID: [{}], DAG Session ID: [{}]", getHostNameIP(), queryId,

Review comment:
       I would change that to something like "Executing query with Id {} from dag {} by session {} "
   Not sure I would add the hostName IP here as this can be quite confusing, especially when running mutiple containers on the same node (like kubernetes). Maybe we already have better info for that on a Task level ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r573064500



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -351,6 +356,17 @@ private String getUserNameForGroups(SessionState ss) {
     return ss.getUserName();
   }
 
+  private static String getHostNameIP() {
+    try {
+      InetAddress ip = InetAddress.getLocalHost();
+      String hostname = ip.getHostName();
+      return String.format("%s (%s)", hostname, ip.getHostAddress());
+    } catch (UnknownHostException e) {
+      LOG.debug("Unable to determine hostname", e);

Review comment:
       @miklosgergely The issue here is that anything that logs to this logger gets forwarded to the user's connection.  I don't want to print errors and stack traces to their connection.  If user starts getting "unknown" responses, they can enable debug logging and get it.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r593811046



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -253,7 +259,7 @@ public int execute() {
           counters = mergedCounters;
         } catch (Exception err) {
           // Don't fail execution due to counters - just don't print summary info
-          LOG.warn("Failed to get counters. Ignoring, summary info will be incomplete. " + err, err);
+          LOG.warn("Failed to get counters. Ignoring, summary info will be incomplete.", err);

Review comment:
       Nope.  Removing this was intentional.  SLF4J will log the entire Exception information, no need to print it twice.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #1963:
URL: https://github.com/apache/hive/pull/1963#issuecomment-776753469


   @pgaref Can you take a peek at this?  This is related to my Tez troubleshooting work.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] pgaref commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
pgaref commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r573796890



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -137,6 +139,8 @@ public int execute() {
     Context ctx = null;
     Ref<TezSessionState> sessionRef = Ref.from(null);
 
+    final String queryId = HiveConf.getVar(conf, HiveConf.ConfVars.HIVEQUERYID);

Review comment:
       I would probably move this to Task initialize method and reuse across method -- e.g., build, execute etc

##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -236,6 +239,10 @@ public int execute() {
           throw new HiveException("Operation cancelled");
         }
 
+        // Log all the info required to find the various logs for this query
+        LOG.info("HS2 Host: [{}], Query ID: [{}], Dag ID: [{}], DAG Session ID: [{}]", getHostNameIP(), queryId,

Review comment:
       I would change that to something like "Executing query with Id {} from dag {} by session {} "
   Not sure I would add the hostName IP here as this can be quite confusing, especially when running mutiple containers on the same node (like kubernetes). Not sure if we already have better info for that on a Task level.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r592501626



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -236,6 +239,10 @@ public int execute() {
           throw new HiveException("Operation cancelled");
         }
 
+        // Log all the info required to find the various logs for this query
+        LOG.info("HS2 Host: [{}], Query ID: [{}], Dag ID: [{}], DAG Session ID: [{}]", getHostNameIP(), queryId,

Review comment:
       I'd hate to tie anything to the session state.
   ```java
         // Need to remove this static hack. But this is the way currently to get a session.
         SessionState ss = SessionState.get();
   ```
   
   Let me see what else we can do.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] miklosgergely commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
miklosgergely commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r573049911



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -351,6 +356,17 @@ private String getUserNameForGroups(SessionState ss) {
     return ss.getUserName();
   }
 
+  private static String getHostNameIP() {
+    try {
+      InetAddress ip = InetAddress.getLocalHost();
+      String hostname = ip.getHostName();
+      return String.format("%s (%s)", hostname, ip.getHostAddress());
+    } catch (UnknownHostException e) {
+      LOG.debug("Unable to determine hostname", e);

Review comment:
       This should be at least info, or rather warn level. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] belugabehr commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r573980532



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -137,6 +139,8 @@ public int execute() {
     Context ctx = null;
     Ref<TezSessionState> sessionRef = Ref.from(null);
 
+    final String queryId = HiveConf.getVar(conf, HiveConf.ConfVars.HIVEQUERYID);

Review comment:
       In working with this code, I think the Hive query ID is passed around in a few different ways.  There can be follow up work to unify/simplify this, but I would not want to change it too much in this unrelated ticket.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] pgaref commented on a change in pull request #1963: HIVE-24758: Log Tez Task DAG ID, DAG Session ID, HS2 Hostname

Posted by GitBox <gi...@apache.org>.
pgaref commented on a change in pull request #1963:
URL: https://github.com/apache/hive/pull/1963#discussion_r592527788



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java
##########
@@ -253,7 +259,7 @@ public int execute() {
           counters = mergedCounters;
         } catch (Exception err) {
           // Don't fail execution due to counters - just don't print summary info
-          LOG.warn("Failed to get counters. Ignoring, summary info will be incomplete. " + err, err);
+          LOG.warn("Failed to get counters. Ignoring, summary info will be incomplete.", err);

Review comment:
       Missing {} here?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org