[jira] [Commented] (HIVE-11488) Add sessionId and queryId info to HS2 log

["Aihua Xu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-11488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999276#comment-14999276 ] 

Aihua Xu commented on HIVE-11488:
---------------------------------

Initial patch. The changes in this patch include: 1. the patch will take a queryId if the client passes in one; create a new one if it doesn't exist. 2. Set the sessionId and queryId to ThreadContext of Log4J so that Log4J will print the info if it's configured to do so in log4j configuration. %X is added to print the info saved in ThreadContext. {{<PatternLayout pattern="%d{ISO8601} %-5p [%t(%x)]: %c{2} (%F:%M(%L)) - %m %X%n" />}}.

Here is the sample output:
{noformat}
2015-11-10T15:08:12,358 INFO  [HiveServer2-Background-Pool: Thread-49([])]: exec.Utilities (Utilities.java:getBaseWork(392)) - PLAN PATH = file:/var/folders/00/_7st2p5x5bg1hvpxlgm455180000gp/T/axu/5d204817-9069-4770-b6b5-4d24052febcb/hive_2015-11-10_15-07-16_107_8637508181034550535-2/-mr-10008/e1ed455b-014f-4537-bfc6-a42ab8382ab8/map.xml {hive.query.id=axu_20151110150653_a55d11ee-88ab-4e43-a59f-0d1b76dc7d76, hive.session.id=5d204817-9069-4770-b6b5-4d24052febcb}
2015-11-10T15:08:12,359 INFO  [HiveServer2-Background-Pool: Thread-49([])]: exec.Utilities (Utilities.java:getBaseWork(392)) - PLAN PATH = file:/var/folders/00/_7st2p5x5bg1hvpxlgm455180000gp/T/axu/5d204817-9069-4770-b6b5-4d24052febcb/hive_2015-11-10_15-07-16_107_8637508181034550535-2/-mr-10008/e1ed455b-014f-4537-bfc6-a42ab8382ab8/reduce.xml {hive.query.id=axu_20151110150653_a55d11ee-88ab-4e43-a59f-0d1b76dc7d76, hive.session.id=5d204817-9069-4770-b6b5-4d24052febcb}
{noformat}

MapReduce job is not adding the support since the logs are separate from each other.

> Add sessionId and queryId info to HS2 log
> -----------------------------------------
>
>                 Key: HIVE-11488
>                 URL: https://issues.apache.org/jira/browse/HIVE-11488
>             Project: Hive
>          Issue Type: New Feature
>          Components: Logging
>    Affects Versions: 2.0.0
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>         Attachments: HIVE-11488.patch
>
>
> Session is critical for a multi-user system like Hive. Currently Hive doesn't log seessionId to the log file, which sometimes make debugging and analysis difficult when multiple activities are going on at the same time and the log from different sessions are mixed together.
> Currently, Hive already has the sessionId saved in SessionState and also there is another sessionId in SessionHandle (Seems not used and I'm still looking to understand it). Generally we should have one sessionId from the beginning in the client side and server side. Seems we have some work on that side first.
> The sessionId then can be added to log4j supported mapped diagnostic context (MDC) and can be configured to output to log file through the log4j property. MDC is per thread, so we need to add sessionId to the HS2 main thread and then it will be inherited by the child threads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)