[jira] [Commented] (HIVE-14394) Reduce excessive INFO level logging

["Sushanth Sowmyan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-14394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401311#comment-15401311 ] 

Sushanth Sowmyan commented on HIVE-14394:
-----------------------------------------

The recent event that predicated the filing of this bug is reports that we're filling up disks of the metastore box from recent hive installations, due to the excessive amount of metrics logging we generate. At first, this is not a hive problem, because the primary bit of extra logging is coming from HadoopMetrics2Reporter, which is a new library we depend on to publish our Codahale metrics to the Hadoop Metrics2 subsystem, and is easily fixed by changing the loglevel for that class to DEBUG, rather than retaining it on INFO. However, it becomes our problem for upgrade scenarios where the customer now has a manual change they need to make for smooth operations of their cluster because we upgraded a dependency.

Thus, we need to work with the maintainer ([~elserj] of HadoopMetrics2Reporter to get a new version of the library, with info-level logging reduced to debug-level logging. To this end, I have filed a github pull request : https://github.com/joshelser/dropwizard-hadoop-metrics2/pull/1 . 

However, in the meanwhile, we also need to make a patch available (even if we don't commit it to hive) to solve the problem those installations have. Thus, I'm uploading a temporary patch to move the Metrics2 Reporter to the hive project, with the fix, so that those that need it can use this patch. I will update this patch with a newer patch which bumps up the version number of the dependency with the log-level aspect changed once we have a newer library version with that.

> Reduce excessive INFO level logging
> -----------------------------------
>
>                 Key: HIVE-14394
>                 URL: https://issues.apache.org/jira/browse/HIVE-14394
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
>
> We need to cull down on the number of logs we generate in HMS and HS2 that are not needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)