[jira] [Created] (HIVE-7438) Counters and metrics

["Xuefu Zhang (JIRA)" <ji...@apache.org>]

Xuefu Zhang created HIVE-7438:
---------------------------------

             Summary: Counters and metrics
                 Key: HIVE-7438
                 URL: https://issues.apache.org/jira/browse/HIVE-7438
             Project: Hive
          Issue Type: Sub-task
          Components: Spark
            Reporter: Xuefu Zhang


Hive makes use of MapReduce counters for statistics and possibly for other purposes. For Hive on Spark, we should achieve the same functionality using Spark's accumulators.

Hive also collects metrics from MapReduce jobs traditionally. Spark job very likely publishes a different set of metrics, which, if made available, would help user to get insights into their spark jobs. Thus, we should obtain the metrics and make them available as we do for MapReduce.

This task therefore includes 1. identify Hive's existing functionality w.r.t. counters and metrics; 2. design and implement the same functionality in Spark.

Please refer to the design document for more information. https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark#HiveonSpark-CountersandMetrics



--
This message was sent by Atlassian JIRA
(v6.2#6252)