You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "Joel Baranick (JIRA)" <ji...@apache.org> on 2017/08/04 16:04:00 UTC
[jira] [Commented] (GOBBLIN-138) Task metrics are not saved to Job
History Database when running under Yarn
[ https://issues.apache.org/jira/browse/GOBBLIN-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114558#comment-16114558 ]
Joel Baranick commented on GOBBLIN-138:
---------------------------------------
[~abti] Seems like there are being stored now, but they show up in the properties table, not the metrics table.
> Task metrics are not saved to Job History Database when running under Yarn
> --------------------------------------------------------------------------
>
> Key: GOBBLIN-138
> URL: https://issues.apache.org/jira/browse/GOBBLIN-138
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-yarn
> Reporter: Joel Baranick
> Assignee: Abhishek Tiwari
> Labels: Bug:Generic, LaunchType:Yarn
>
> Task level metrics are not transmitted from the containers that tasks are running on back to the app_master. This means that task level metrics cannot be saved in the Job History Database. We should be able to store the task level metrics in the Job History Database just like when we run a standalone job.
>
> *Github Url* : https://github.com/linkedin/gobblin/issues/748
> *Github Reporter* : [~jbaranick]
> *Github Created At* : 2016-02-23T21:33:39Z
> *Github Updated At* : 2016-03-08T06:15:25Z
> h3. Comments
> ----
> *ydai1124* wrote on 2016-02-23T21:42:11Z : @kadaan We are deprecating the Job/Task metrics you are using. It is better to switch to Gobblin Metrics: https://github.com/linkedin/gobblin/wiki/Gobblin%20Metrics%20Architecture. It has more contents and more stable. But we don't have the reporter to report to database yet. You can implement your own reporter: https://github.com/linkedin/gobblin/wiki/Implementing%20New%20Reporters.
>
>
> *Github Url* : https://github.com/linkedin/gobblin/issues/748#issuecomment-187926718
> ----
> [~stakiar] wrote on 2016-02-24T00:07:38Z : @kadaan is this bug blocking the deployment of the `JobHistoryStore` or are you actually interesting in viewing the `TaskMetrics`? Just want to understand where the bug is.
> AS @ydai1124 mentioned we are moving away from `TaskMetrics` and `JobMetrics`, and at some point we want to remove the current way `TaskMetrics` and `JobMetrics` are written to the `JobHistoryStore`.
>
>
> *Github Url* : https://github.com/linkedin/gobblin/issues/748#issuecomment-187977305
> ----
> [~jbaranick] wrote on 2016-02-24T00:09:20Z : No, it is not blocking anything.
>
>
> *Github Url* : https://github.com/linkedin/gobblin/issues/748#issuecomment-187978192
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)