[jira] [Commented] (YARN-10427) Duplicate Job IDs in SLS output

["Drew Merrill (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-10427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235876#comment-17235876 ] 

Drew Merrill commented on YARN-10427:
-------------------------------------

_*Anyone?*_ *I'd really appreciate a response from someone on this.* _*A developer? A fellow user? A computer?*_

Have I not included enough info or the right info needed to investigate this? If so, please let me know!

*At the very least, can someone else please _{color:#FF0000}confirm{color}_ _{color:#FF0000}that the issue with duplicate Job IDs is reproducible?{color}_ {color:#FF0000}{color:#172b4d}It's frustrating and stressful not knowing if the problem is due to something that _I'm doing wrong_ or if it's a bug in Hadoop.{color}{color}*

*{color:#FF0000}{color:#172b4d}There's either a teachable moment here where I can learn what I'm doing wrong or else an opportunity to identify and fix a bug in Hadoop. Both are good outcomes!{color}{color}*

> Duplicate Job IDs in SLS output
> -------------------------------
>
>                 Key: YARN-10427
>                 URL: https://issues.apache.org/jira/browse/YARN-10427
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler-load-simulator
>    Affects Versions: 3.0.0, 3.3.0, 3.2.1, 3.4.0
>         Environment: I ran the attached inputs on my MacBook Pro, using Hadoop compiled from the latest trunk (as of commit 139a43e98e). I also tested against 3.2.1 and 3.3.0 release branches.
>  
>            Reporter: Drew Merrill
>            Priority: Major
>         Attachments: fair-scheduler.xml, inputsls.json, jobruntime.csv, mapred-site.xml, sls-runner.xml, yarn-site.xml
>
>
> Hello, I'm hoping someone can help me resolve or understand some issues I've been having with the YARN Scheduler Load Simulator (SLS). I've been experimenting with SLS for several months now at work as we're trying to build a simulation model to characterize our enterprise Hadoop infrastructure for purposes of future capacity planning. In the process of attempting to verify and validate the SLS output, I've encountered a number of issues including runtime exceptions and bad output. The focus of this issue is the bad output. In all my simulation runs, the jobruntime.csv output seems to have one or more of the following problems: no output, duplicate job ids, and/or missing job ids.
>  
> Because of where I work, I'm unable to provide the exact inputs I typically use, but I'm able to reproduce the problem of the duplicate Job IDS using some simplified inputs and configuration files, which I've attached, along with the output I obtained.
>  
> The command I used to run the simulation:
> {{./runsls.sh --tracetype=SLS --tracelocation=./inputsls.json --output-dir=sls-run-1 --print-simulation --track-jobs=job_1,job_2,job_3,job_4,job_5,job_6,job_7,job_8,job_9,job_10}}
>  
> Can anyone help me understand what would cause the duplicate Job IDs in the output? Is this a bug in Hadoop or a problem with my inputs? Thanks in advance.
>  
> PS: This is my first issue I've ever opened so please be kind if I've missed something or am not understanding something obvious about the way Hadoop works. I'll gladly follow-up with more info as requested.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org