[jira] [Updated] (MAPREDUCE-2596) Gridmix should notify job failures

["Amar Kamat (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amar Kamat updated MAPREDUCE-2596:
----------------------------------

    Attachment: gridmix-memory-emulation-v1.3.patch

Attaching a patch that summarizes the gridmix run. This is how the summary looks like:
{noformat}
Execution Summary:-
Input trace: file:/home/user/hadoop/mapreduce/job_201106081604_0006.json
Input trace signature: 621979b3d484a2d824bd5d9a721dc085
Total number of jobs in trace: 1
Expected input data size: N/A
Input data statistics: Input data size: 5.0m, Number of files: 1, Compression: true
Total number of jobs processed: 1
Total number of successful jobs: 1
Total number of failed jobs: 0
Total number of jobs failed after submission: 0
Total number of jobs failed before submission: 0
Total number of map tasks launched: 1
Total number of reduce task launched: 1
Gridmix start time: 6/23/11 12:37 PM
Gridmix end time: 6/23/11 12:40 PM
Gridmix simulation start time: 6/23/11 12:37 PM
Gridmix runtime: 2mins, 57sec
Time spent in initialization (data-gen etc): 2sec
Time spent in simulation: 2mins, 54sec
Gridmix configuration parameters: -Dgridmix.output.directory=output -Dmapreduce.job.ubertask.enable=false -Dgridmix.min.file.size=1024 -Dgridmix.compression-emulation.enable=true input file:///home/user/hadoop/mapreduce/job_201106081604_0006.json
Gridmix job submission policy: STRESS
Gridmix resolver: org.apache.hadoop.mapred.gridmix.SubmitterUserResolver

Cluster Summary:-
JobTracker: localhost:9520
FileSystem: localhost:9500
Number of blacklisted trackers: 0
Number of active trackers: 1
Max map task capacity: 2
Max reduce task capacity: 2
{noformat}

test-patch and gridmix junit tests passed.

Todo:
1. job level summary (current-jobid, original-jobid, runtime characteristics, input output characteristics etc)
2. ability to record the summary in a specified file and in a specified format (xml/json/csv)
3. Test this on a slightly larger cluster.

The first cut is intentionally kept simple.

> Gridmix should notify job failures
> ----------------------------------
>
>                 Key: MAPREDUCE-2596
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2596
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: benchmarks, contrib/gridmix
>    Affects Versions: 0.23.0
>            Reporter: Arun C Murthy
>            Assignee: Amar Kamat
>         Attachments: gridmix-memory-emulation-v1.3.patch
>
>
> Gridmix doesn't warn the user if any of the jobs in the mix fail... it probably should printout a summary of the jobs and other statistics at the end too.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira