[jira] Created: (MAPREDUCE-2153) Bring in more job configuration properties in to the trace file

["Ravi Gummadi (JIRA)" <ji...@apache.org>]

Bring in more job configuration properties in to the trace file
---------------------------------------------------------------

                 Key: MAPREDUCE-2153
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2153
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: tools/rumen
            Reporter: Ravi Gummadi


To emulate distributed cache usage in gridmix jobs, there are 9 configuration properties needed to be available in trace file: 
(1) mapreduce.job.cache.files
(2) mapreduce.job.cache.files.visibilities
(3) mapreduce.job.cache.files.filesizes
(4) mapreduce.job.cache.files.timestamps

(5) mapreduce.job.cache.archives
(6) mapreduce.job.cache.archives.visibilities
(7) mapreduce.job.cache.archives.filesizes
(8) mapreduce.job.cache.archives.timestamps

(9) mapreduce.job.cache.symlink.create

To emulate data compression in gridmix jobs, trace file should contain the following configuration properties:
(1) mapreduce.map.output.compress
(2) mapreduce.map.output.compress.codec
(3) mapreduce.output.fileoutputformat.compress
(4) mapreduce.output.fileoutputformat.compress.codec
(5) mapreduce.output.fileoutputformat.compress.type

Ideally, gridmix should set many job specific configuration properties like io.sort.mb, io.sort.factor, etc when running simulated jobs to get the same effect of original/real job in terms of spilled records, number of merges, etc.

TraceBuilder should bring in all these properties into the generated trace file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.