You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Chengxiang Li (JIRA)" <ji...@apache.org> on 2014/12/05 03:18:12 UTC

[jira] [Resolved] (HIVE-7552) Collect spark job statistic through spark metrics [Spark Branch]

     [ https://issues.apache.org/jira/browse/HIVE-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chengxiang Li resolved HIVE-7552.
---------------------------------
    Resolution: Implemented

> Collect spark job statistic through spark metrics [Spark Branch]
> ----------------------------------------------------------------
>
>                 Key: HIVE-7552
>                 URL: https://issues.apache.org/jira/browse/HIVE-7552
>             Project: Hive
>          Issue Type: New Feature
>          Components: Spark
>            Reporter: Chengxiang Li
>            Assignee: Na Yang
>
> MR/Tez use counters to collect job statistic information, while Spark does not use accumulator to do the same thing. Instead, Spark store task metrics information in TaskMetrics and send it back to scheduler. We  could get spark job statistic information through combine all TaskMetrics with SparkListener.
> NO PRECOMMIT TESTS. This is for spark-branch only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)