You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/10/17 04:59:00 UTC
[jira] [Work logged] (GOBBLIN-2166) GoT must fill in info required for RMAppSummaryEvent fields - azkabanexecid, azkabanprojectname, azkabanflowid, azkabanjobid
[ https://issues.apache.org/jira/browse/GOBBLIN-2166?focusedWorklogId=938565&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-938565 ]
ASF GitHub Bot logged work on GOBBLIN-2166:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 17/Oct/24 04:58
Start Date: 17/Oct/24 04:58
Worklog Time Spent: 10m
Work Description: abhishekmjain opened a new pull request, #4067:
URL: https://github.com/apache/gobblin/pull/4067
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
### JIRA
- [ ] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN-2166) issues and references them in the PR title. For example, "[GOBBLIN-2166] GoT must fill in info required for RMAppSummaryEvent fields"
- https://issues.apache.org/jira/browse/GOBBLIN-2166
### Description
- [ ] Here are some details about my PR, including screenshots (if applicable):
GoT doesn't send applicationTags while submitting yarn app. This PR adds the applicationTags to appSubmissionContext while yarn app submission.
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
Tested the change in a local azkaban project:
### Commits
- [ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
Issue Time Tracking
-------------------
Worklog Id: (was: 938565)
Remaining Estimate: 0h
Time Spent: 10m
> GoT must fill in info required for RMAppSummaryEvent fields - azkabanexecid, azkabanprojectname, azkabanflowid, azkabanjobid
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-2166
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2166
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Abhishek Jain
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> right now, it's not possible to find `RMAppSummaryEvent`s by any of the above named fields even though the `GaaS-Gobblin-Temporal-Azkaban` project is used by GoT execs
> because `azkabanprojectname` is not populated in events for any GoT execution (the way it IS for GoMR executions), the only way to locate `RMAppSummaryEvent`s for GoT executions is `appid`.
> *why does this matter?*
> a significant consequence of missing these fields is it thwarts joining `GaaSJobObservabilityEvent`s to `RMAppSummaryEvent`s. this severely complicates analysis, because the GaaS obs. event does NOT contain the YARN appid, only the AZ flow ID.
> since there is clearly an AZ execution involved, the solution is for GoT to set whatever props are required on the YARN app side, so YARN will emit fully-populated `RMAppSummaryEvent`s, with all of their `azkaban*` fields set.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)