[jira] [Work logged] (GOBBLIN-1620) Make yarn container allocation group by helix tag

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GOBBLIN-1620?focusedWorklogId=748004&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748004 ]

ASF GitHub Bot logged work on GOBBLIN-1620:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 25/Mar/22 21:08
            Start Date: 25/Mar/22 21:08
    Worklog Time Spent: 10m 
      Work Description: hanghangliu opened a new pull request #3487:
URL: https://github.com/apache/gobblin/pull/3487


   Dear Gobblin maintainers,
   
   Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
   
   
   ### JIRA
   - [ ] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
       - https://issues.apache.org/jira/projects/GOBBLIN/issues/GOBBLIN-1620
   
   
   ### Description
   - [ ] Here are some details about my PR, including screenshots (if applicable):
   - GobblinHelixJobLauncher can pick up helix tag and resource config for each Gobblin job run. These configs will be read by YarnAutoScalingManager and YarnService, thus allocated container can have helix instances with required tag and desired resources. Since helix instances can contain unique tags, tasks within each Gobblin job run will be assigned to containers with specific tag and resource.
   
   
   ### Tests
   - [ ] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
   Test in testing pipeline. 
   
   ### Commits
   - [ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
       1. Subject is separated from body by a blank line
       2. Subject is limited to 50 characters
       3. Subject does not end with a period
       4. Subject uses the imperative mood ("add", not "adding")
       5. Body wraps at 72 characters
       6. Body explains "what" and "why", not "how"
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 748004)
    Remaining Estimate: 0h
            Time Spent: 10m

> Make yarn container allocation group by helix tag
> -------------------------------------------------
>
>                 Key: GOBBLIN-1620
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1620
>             Project: Apache Gobblin
>          Issue Type: New Feature
>          Components: gobblin-cluster
>            Reporter: Hanghang Liu
>            Assignee: Hung Tran
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> This change enable Helix instances use the specific tag defined(if exists) in their Job/Workflow. This makes the Helix cluster management easier and more strait forward, as each job's instances can carry unique Helix tag. 
> In this case, for each Gobblin job run, helix tasks within this job will
> be assigned to containers with desired helix tag and resource. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)