[jira] [Commented] (YARN-10427) Duplicate Job IDs in SLS output

["Szilard Nemeth (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-10427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261287#comment-17261287 ] 

Szilard Nemeth commented on YARN-10427:
---------------------------------------

Hi [~werd.up],
I'm glad that you found it useful.


> Duplicate Job IDs in SLS output
> -------------------------------
>
>                 Key: YARN-10427
>                 URL: https://issues.apache.org/jira/browse/YARN-10427
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler-load-simulator
>    Affects Versions: 3.0.0, 3.3.0, 3.2.1, 3.4.0
>         Environment: I ran the attached inputs on my MacBook Pro, using Hadoop compiled from the latest trunk (as of commit 139a43e98e). I also tested against 3.2.1 and 3.3.0 release branches.
>  
>            Reporter: Drew Merrill
>            Assignee: Szilard Nemeth
>            Priority: Major
>         Attachments: YARN-10427-sls-scriptsandlogs.tar.gz, YARN-10427.001.patch, YARN-10427.002.patch, YARN-10427.003.patch, fair-scheduler.xml, inputsls.json, jobruntime.csv, jobruntime.csv, mapred-site.xml, sls-runner.xml, yarn-site.xml
>
>
> Hello, I'm hoping someone can help me resolve or understand some issues I've been having with the YARN Scheduler Load Simulator (SLS). I've been experimenting with SLS for several months now at work as we're trying to build a simulation model to characterize our enterprise Hadoop infrastructure for purposes of future capacity planning. In the process of attempting to verify and validate the SLS output, I've encountered a number of issues including runtime exceptions and bad output. The focus of this issue is the bad output. In all my simulation runs, the jobruntime.csv output seems to have one or more of the following problems: no output, duplicate job ids, and/or missing job ids.
>  
> Because of where I work, I'm unable to provide the exact inputs I typically use, but I'm able to reproduce the problem of the duplicate Job IDS using some simplified inputs and configuration files, which I've attached, along with the output I obtained.
>  
> The command I used to run the simulation:
> {{./runsls.sh --tracetype=SLS --tracelocation=./inputsls.json --output-dir=sls-run-1 --print-simulation --track-jobs=job_1,job_2,job_3,job_4,job_5,job_6,job_7,job_8,job_9,job_10}}
>  
> Can anyone help me understand what would cause the duplicate Job IDs in the output? Is this a bug in Hadoop or a problem with my inputs? Thanks in advance.
>  
> PS: This is my first issue I've ever opened so please be kind if I've missed something or am not understanding something obvious about the way Hadoop works. I'll gladly follow-up with more info as requested.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org