[jira] [Comment Edited] (YARN-10547) Decouple job parsing logic from SLSRunner

["Szilard Nemeth (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17510839#comment-17510839 ] 

Szilard Nemeth edited comment on YARN-10547 at 3/22/22, 7:15 PM:
-----------------------------------------------------------------

Hi [~quapaw],
Thanks for your review.
Fixed the folllowing: 
1. SLSRunner#L488: Checked exception is not thrown.
2. As I only moved this part of the code, I don't think it's worth to do a refactor on top of moving the code here. The raw type of map could be fixed easily with Jackson type references: https://stackoverflow.com/questions/6846244/jackson-and-generic-type-reference
However, as the unparameterized Map is passed along to many places, the changes would cascade too deeply, so the fix won't be that simple.
See the method definition and all the called methods that receive the Map: 
{code}
  public static AMDefinitionSLS createFromSlsTrace(Map<?, ?> jsonJob,
      SLSRunner slsRunner) throws YarnException {
{code}

I do suggest a follow-up jira for this cleanup.


was (Author: snemeth):
Hi [~quapaw],
Thanks for your review.
Fixed the folllowing: 
1. SLSRunner#L488: Checked exception is not thrown.
2. As I only moved this part of the code, I don't think it's worth to do a refactor on top of moving the code here. The raw type of map could be fixed easily with Jackson type references: https://stackoverflow.com/questions/6846244/jackson-and-generic-type-reference
However, as the unparameterized Map is passed along to many places, the changes would cascade too deeply, so the fix won't be that simple.
See the method definition and all the called methods that receive the Map: 
{code}
  public static AMDefinitionSLS createFromSlsTrace(Map<?, ?> jsonJob,
      SLSRunner slsRunner) throws YarnException {
{code}

> Decouple job parsing logic from SLSRunner
> -----------------------------------------
>
>                 Key: YARN-10547
>                 URL: https://issues.apache.org/jira/browse/YARN-10547
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Szilard Nemeth
>            Assignee: Szilard Nemeth
>            Priority: Minor
>         Attachments: YARN-10547.001.patch, YARN-10547.002.patch, YARN-10547.003.patch, YARN-10547.004.patch, YARN-10547.005.patch
>
>
> SLSRunner has too many responsibilities.
> One of them is to parse the job details from the SLS input formats and launch the AMs and task containers.
> As a first step, the job parser logic could be decoupled from this class.
> There are 3 types of inputs: 
> - SLS trace
> - Synth
> - Rumen
> Their job parsing method are: 
> - SLS trace: https://github.com/apache/hadoop/blob/005b854f6bad66defafae0abf95dabc6c36ca8b1/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java#L479-L526
> - Synth: https://github.com/apache/hadoop/blob/005b854f6bad66defafae0abf95dabc6c36ca8b1/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java#L722-L790
> - Rumen: https://github.com/apache/hadoop/blob/005b854f6bad66defafae0abf95dabc6c36ca8b1/hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java#L651-L716



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org