You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2013/04/19 13:11:15 UTC
[jira] [Commented] (HADOOP-9487) Deprecation warnings in
Configuration should go to their own log or otherwise be suppressible
[ https://issues.apache.org/jira/browse/HADOOP-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636276#comment-13636276 ]
Steve Loughran commented on HADOOP-9487:
----------------------------------------
an example trace, showing most of the work is being lost in deprecation warnings
{code}
57/xyts0qt105z1f1k0twk6rd8m0000gq/T/1366368940255-0
2013-04-19 11:55:40,288 [main] INFO mapReduceLayer.MapReduceLauncher (MapReduceLauncher.java:launchPig(224)) - 1 map-reduce job(s) waiting for submission.
2013-04-19 11:55:40,289 [main] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.tracker.http.address is deprecated. Instead, use mapreduce.jobtracker.http.address
2013-04-19 11:55:40,296 [JobControl] INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2013-04-19 11:55:40,310 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - dfs.df.interval is deprecated. Instead, use fs.df.interval
2013-04-19 11:55:40,310 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2013-04-19 11:55:40,310 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.shuffle.read.timeout is deprecated. Instead, use mapreduce.reduce.shuffle.read.timeout
2013-04-19 11:55:40,310 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - io.sort.spill.percent is deprecated. Instead, use mapreduce.map.sort.spill.percent
2013-04-19 11:55:40,311 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.parallel.copies is deprecated. Instead, use mapreduce.reduce.shuffle.parallelcopies
2013-04-19 11:55:40,311 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.submit.replication is deprecated. Instead, use mapreduce.client.submit.file.replication
2013-04-19 11:55:40,311 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.profile is deprecated. Instead, use mapreduce.task.profile
2013-04-19 11:55:40,311 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.timeout is deprecated. Instead, use mapreduce.task.timeout
2013-04-19 11:55:40,311 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - jobclient.completion.poll.interval is deprecated. Instead, use mapreduce.client.completion.pollinterval
2013-04-19 11:55:40,311 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.compression.codec is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.codec
2013-04-19 11:55:40,311 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.shuffle.merge.percent is deprecated. Instead, use mapreduce.reduce.shuffle.merge.percent
2013-04-19 11:55:40,312 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.max.attempts is deprecated. Instead, use mapreduce.map.maxattempts
2013-04-19 11:55:40,312 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.reduce.input.buffer.percent is deprecated. Instead, use mapreduce.reduce.input.buffer.percent
2013-04-19 11:55:40,312 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
2013-04-19 11:55:40,312 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.userlog.limit.kb is deprecated. Instead, use mapreduce.task.userlog.limit.kb
2013-04-19 11:55:40,312 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - fs.default.name is deprecated. Instead, use fs.defaultFS
2013-04-19 11:55:40,312 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.speculative.execution.slowNodeThreshold is deprecated. Instead, use mapreduce.job.speculative.slownodethreshold
2013-04-19 11:55:40,312 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.skip.map.max.skip.records is deprecated. Instead, use mapreduce.map.skip.maxrecords
2013-04-19 11:55:40,313 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
2013-04-19 11:55:40,313 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.profile.maps is deprecated. Instead, use mapreduce.task.profile.maps
2013-04-19 11:55:40,313 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
2013-04-19 11:55:40,313 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
2013-04-19 11:55:40,313 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.max.tracker.failures is deprecated. Instead, use mapreduce.job.maxtaskfailures.per.tracker
2013-04-19 11:55:40,313 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.jobtracker.split.metainfo.maxsize is deprecated. Instead, use mapreduce.job.split.metainfo.maxsize
2013-04-19 11:55:40,314 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - job.end.retry.attempts is deprecated. Instead, use mapreduce.job.end-notification.retry.attempts
2013-04-19 11:55:40,314 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
2013-04-19 11:55:40,314 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max
2013-04-19 11:55:40,314 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.child.log.level is deprecated. Instead, use mapreduce.reduce.log.level
2013-04-19 11:55:40,314 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.max.attempts is deprecated. Instead, use mapreduce.reduce.maxattempts
2013-04-19 11:55:40,314 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.output.compression.codec is deprecated. Instead, use mapreduce.map.output.compress.codec
2013-04-19 11:55:40,315 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.shuffle.input.buffer.percent is deprecated. Instead, use mapreduce.reduce.shuffle.input.buffer.percent
2013-04-19 11:55:40,315 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - keep.failed.task.files is deprecated. Instead, use mapreduce.task.files.preserve.failedtasks
2013-04-19 11:55:40,315 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.speculative.execution.slowTaskThreshold is deprecated. Instead, use mapreduce.job.speculative.slowtaskthreshold
2013-04-19 11:55:40,315 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.acls.enabled is deprecated. Instead, use mapreduce.cluster.acls.enabled
2013-04-19 11:55:40,315 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.skip.attempts.to.start.skipping is deprecated. Instead, use mapreduce.task.skip.start.attempts
2013-04-19 11:55:40,315 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
2013-04-19 11:55:40,315 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.name is deprecated. Instead, use mapreduce.job.name
2013-04-19 11:55:40,316 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - jobclient.output.filter is deprecated. Instead, use mapreduce.client.output.filter
2013-04-19 11:55:40,316 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
2013-04-19 11:55:40,316 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.speculative.execution.speculativeCap is deprecated. Instead, use mapreduce.job.speculative.speculativecap
2013-04-19 11:55:40,316 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - jobclient.progress.monitor.poll.interval is deprecated. Instead, use mapreduce.client.progressmonitor.pollinterval
2013-04-19 11:55:40,316 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.child.log.level is deprecated. Instead, use mapreduce.map.log.level
2013-04-19 11:55:40,316 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
2013-04-19 11:55:40,316 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.compression.type is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.type
2013-04-19 11:55:40,317 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.profile.reduces is deprecated. Instead, use mapreduce.task.profile.reduces
2013-04-19 11:55:40,317 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
2013-04-19 11:55:40,317 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - job.end.retry.interval is deprecated. Instead, use mapreduce.job.end-notification.retry.interval
2013-04-19 11:55:40,317 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
2013-04-19 11:55:40,317 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
2013-04-19 11:55:40,317 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.child.tmp is deprecated. Instead, use mapreduce.task.tmp.dir
2013-04-19 11:55:40,317 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2013-04-19 11:55:40,318 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use mapreduce.job.committer.setup.cleanup.needed
2013-04-19 11:55:40,318 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.queue.name is deprecated. Instead, use mapreduce.job.queuename
2013-04-19 11:55:40,318 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.skip.reduce.max.skip.groups is deprecated. Instead, use mapreduce.reduce.skip.maxgroups
2013-04-19 11:55:40,318 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.inmem.merge.threshold is deprecated. Instead, use mapreduce.reduce.merge.inmem.threshold
2013-04-19 11:55:40,318 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - topology.script.number.args is deprecated. Instead, use net.topology.script.number.args
2013-04-19 11:55:40,319 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.slowstart.completed.maps is deprecated. Instead, use mapreduce.job.reduce.slowstart.completedmaps
2013-04-19 11:55:40,319 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - dfs.umaskmode is deprecated. Instead, use fs.permissions.umask-mode
2013-04-19 11:55:40,319 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - topology.node.switch.mapping.impl is deprecated. Instead, use net.topology.node.switch.mapping.impl
2013-04-19 11:55:40,319 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress
2013-04-19 11:55:40,319 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.merge.recordsBeforeProgress is deprecated. Instead, use mapreduce.task.merge.progress.records
2013-04-19 11:55:40,319 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.shuffle.connect.timeout is deprecated. Instead, use mapreduce.reduce.shuffle.connect.timeout
2013-04-19 11:55:40,319 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2013-04-19 11:55:40,345 [JobControl] WARN mapreduce.JobSubmitter (JobSubmitter.java:copyAndConfigureFiles(146)) - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2013-04-19 11:55:40,360 [JobControl] WARN mapreduce.JobSubmitter (JobSubmitter.java:copyAndConfigureFiles(255)) - No job jar file set. User classes may not be found. See Job or Job#setJar(String).
2013-04-19 11:55:40,792 [main] INFO mapReduceLayer.MapReduceLauncher (MapReduceLauncher.java:launchPig(291)) - HadoopJobId: job_local2085479235_0001
2013-04-19 11:55:40,793 [main] INFO mapReduceLayer.MapReduceLauncher (MapReduceLauncher.java:launchPig(297)) - Processing aliases f,generated
2013-04-19 11:55:40,793 [main] INFO mapReduceLayer.MapReduceLauncher (MapReduceLauncher.java:launchPig(299)) - detailed locations: M: generated[1,12],generated[-1,-1],f[5,4] C: R:
2013-04-19 11:55:40,890 [JobControl] INFO input.FileInputFormat (FileInputFormat.java:listStatus(245)) - Total input paths to process : 1
2013-04-19 11:55:40,890 [JobControl] INFO util.MapRedUtil (MapRedUtil.java:getAllFileRecursively(213)) - Total input paths to process : 1
2013-04-19 11:55:42,235 [JobControl] INFO util.MapRedUtil (MapRedUtil.java:getCombinePigSplits(587)) - Total input paths (combined) to process : 2
2013-04-19 11:55:42,303 [JobControl] INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(389)) - number of splits:2
2013-04-19 11:55:42,337 [JobControl] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
2013-04-19 11:55:42,420 [JobControl] INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(459)) - Submitting tokens for job: job_local2085479235_0001
2013-04-19 11:55:42,456 [JobControl] WARN conf.Configuration (Configuration.java:loadProperty(2070)) - file:/tmp/hadoop/mapred/staging/stevel2085479235/.staging/job_local2085479235_0001/job.xml:an attempt to override final parameter: hadoop.tmp.dir; Ignoring.
2013-04-19 11:55:42,614 [JobControl] INFO mapreduce.Job (Job.java:submit(1222)) - The url to track the job: http://localhost:8080/
2013-04-19 11:55:42,617 [Thread-13] INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(319)) - OutputCommitter set in config null
2013-04-19 11:55:42,619 [main] INFO mapReduceLayer.MapReduceLauncher (MapReduceLauncher.java:notifyProgress(512)) - 0% complete
2013-04-19 11:55:42,630 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - dfs.df.interval is deprecated. Instead, use fs.df.interval
2013-04-19 11:55:42,630 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2013-04-19 11:55:42,630 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.shuffle.read.timeout is deprecated. Instead, use mapreduce.reduce.shuffle.read.timeout
2013-04-19 11:55:42,630 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - io.sort.spill.percent is deprecated. Instead, use mapreduce.map.sort.spill.percent
2013-04-19 11:55:42,631 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.parallel.copies is deprecated. Instead, use mapreduce.reduce.shuffle.parallelcopies
2013-04-19 11:55:42,631 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.submit.replication is deprecated. Instead, use mapreduce.client.submit.file.replication
2013-04-19 11:55:42,631 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.profile is deprecated. Instead, use mapreduce.task.profile
2013-04-19 11:55:42,631 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2013-04-19 11:55:42,631 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.timeout is deprecated. Instead, use mapreduce.task.timeout
2013-04-19 11:55:42,631 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - jobclient.completion.poll.interval is deprecated. Instead, use mapreduce.client.completion.pollinterval
2013-04-19 11:55:42,632 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.compression.codec is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.codec
2013-04-19 11:55:42,632 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.shuffle.merge.percent is deprecated. Instead, use mapreduce.reduce.shuffle.merge.percent
2013-04-19 11:55:42,632 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.max.attempts is deprecated. Instead, use mapreduce.map.maxattempts
2013-04-19 11:55:42,632 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.reduce.input.buffer.percent is deprecated. Instead, use mapreduce.reduce.input.buffer.percent
2013-04-19 11:55:42,632 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
2013-04-19 11:55:42,632 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.userlog.limit.kb is deprecated. Instead, use mapreduce.task.userlog.limit.kb
2013-04-19 11:55:42,632 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - fs.default.name is deprecated. Instead, use fs.defaultFS
2013-04-19 11:55:42,633 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.speculative.execution.slowNodeThreshold is deprecated. Instead, use mapreduce.job.speculative.slownodethreshold
2013-04-19 11:55:42,633 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.skip.map.max.skip.records is deprecated. Instead, use mapreduce.map.skip.maxrecords
2013-04-19 11:55:42,633 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use mapreduce.reduce.markreset.buffer.percent
2013-04-19 11:55:42,633 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
2013-04-19 11:55:42,633 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.profile.maps is deprecated. Instead, use mapreduce.task.profile.maps
2013-04-19 11:55:42,633 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
2013-04-19 11:55:42,634 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2013-04-19 11:55:42,634 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
2013-04-19 11:55:42,634 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.max.tracker.failures is deprecated. Instead, use mapreduce.job.maxtaskfailures.per.tracker
2013-04-19 11:55:42,634 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.jobtracker.split.metainfo.maxsize is deprecated. Instead, use mapreduce.job.split.metainfo.maxsize
2013-04-19 11:55:42,634 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - job.end.retry.attempts is deprecated. Instead, use mapreduce.job.end-notification.retry.attempts
2013-04-19 11:55:42,635 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
2013-04-19 11:55:42,635 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max
2013-04-19 11:55:42,635 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.child.log.level is deprecated. Instead, use mapreduce.reduce.log.level
2013-04-19 11:55:42,635 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.max.attempts is deprecated. Instead, use mapreduce.reduce.maxattempts
2013-04-19 11:55:42,635 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.output.compression.codec is deprecated. Instead, use mapreduce.map.output.compress.codec
2013-04-19 11:55:42,635 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.shuffle.input.buffer.percent is deprecated. Instead, use mapreduce.reduce.shuffle.input.buffer.percent
2013-04-19 11:55:42,636 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - keep.failed.task.files is deprecated. Instead, use mapreduce.task.files.preserve.failedtasks
2013-04-19 11:55:42,636 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.speculative.execution.slowTaskThreshold is deprecated. Instead, use mapreduce.job.speculative.slowtaskthreshold
2013-04-19 11:55:42,636 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.acls.enabled is deprecated. Instead, use mapreduce.cluster.acls.enabled
2013-04-19 11:55:42,636 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.skip.attempts.to.start.skipping is deprecated. Instead, use mapreduce.task.skip.start.attempts
2013-04-19 11:55:42,636 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
2013-04-19 11:55:42,636 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.name is deprecated. Instead, use mapreduce.job.name
2013-04-19 11:55:42,636 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - jobclient.output.filter is deprecated. Instead, use mapreduce.client.output.filter
2013-04-19 11:55:42,637 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
2013-04-19 11:55:42,637 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2013-04-19 11:55:42,637 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.speculative.execution.speculativeCap is deprecated. Instead, use mapreduce.job.speculative.speculativecap
2013-04-19 11:55:42,637 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - jobclient.progress.monitor.poll.interval is deprecated. Instead, use mapreduce.client.progressmonitor.pollinterval
2013-04-19 11:55:42,637 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.child.log.level is deprecated. Instead, use mapreduce.map.log.level
2013-04-19 11:55:42,637 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
2013-04-19 11:55:42,637 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.compression.type is deprecated. Instead, use mapreduce.output.fileoutputformat.compress.type
2013-04-19 11:55:42,638 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.profile.reduces is deprecated. Instead, use mapreduce.task.profile.reduces
2013-04-19 11:55:42,638 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
2013-04-19 11:55:42,638 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - job.end.retry.interval is deprecated. Instead, use mapreduce.job.end-notification.retry.interval
2013-04-19 11:55:42,638 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
2013-04-19 11:55:42,638 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
2013-04-19 11:55:42,638 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.child.tmp is deprecated. Instead, use mapreduce.task.tmp.dir
2013-04-19 11:55:42,638 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2013-04-19 11:55:42,638 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use mapreduce.job.committer.setup.cleanup.needed
2013-04-19 11:55:42,639 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.queue.name is deprecated. Instead, use mapreduce.job.queuename
2013-04-19 11:55:42,639 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.skip.reduce.max.skip.groups is deprecated. Instead, use mapreduce.reduce.skip.maxgroups
2013-04-19 11:55:42,639 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.inmem.merge.threshold is deprecated. Instead, use mapreduce.reduce.merge.inmem.threshold
2013-04-19 11:55:42,639 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - topology.script.number.args is deprecated. Instead, use net.topology.script.number.args
2013-04-19 11:55:42,639 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.reduce.slowstart.completed.maps is deprecated. Instead, use mapreduce.job.reduce.slowstart.completedmaps
2013-04-19 11:55:42,639 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - dfs.umaskmode is deprecated. Instead, use fs.permissions.umask-mode
2013-04-19 11:55:42,640 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - topology.node.switch.mapping.impl is deprecated. Instead, use net.topology.node.switch.mapping.impl
2013-04-19 11:55:42,640 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress
2013-04-19 11:55:42,640 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.merge.recordsBeforeProgress is deprecated. Instead, use mapreduce.task.merge.progress.records
2013-04-19 11:55:42,640 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.shuffle.connect.timeout is deprecated. Instead, use mapreduce.reduce.shuffle.connect.timeout
2013-04-19 11:55:42,640 [Thread-13] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2013-04-19 11:55:42,643 [Thread-13] INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(337)) - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter
2013-04-19 11:55:42,655 [Thread-13] INFO mapred.LocalJobRunner (LocalJobRunner.java:run(385)) - Waiting for map tasks
2013-04-19 11:55:42,655 [LocalJobRunner Map Task Executor #0] INFO mapred.LocalJobRunner (LocalJobRunner.java:run(212)) - Starting task: attempt_local2085479235_0001_m_000000_0
2013-04-19 11:55:42,687 [LocalJobRunner Map Task Executor #0] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
2013-04-19 11:55:42,687 [LocalJobRunner Map Task Executor #0] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - user.name is deprecated. Instead, use mapreduce.job.user.name
2013-04-19 11:55:42,687 [LocalJobRunner Map Task Executor #0] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
2013-04-19 11:55:42,688 [LocalJobRunner Map Task Executor #0] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.tip.id is deprecated. Instead, use mapreduce.task.id
2013-04-19 11:55:42,688 [LocalJobRunner Map Task Executor #0] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2013-04-19 11:55:42,689 [LocalJobRunner Map Task Executor #0] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.job.id is deprecated. Instead, use mapreduce.job.id
2013-04-19 11:55:42,695 [LocalJobRunner Map Task Executor #0] WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(823)) - mapred.work.output.dir is deprecated. Instead, use mapreduce.task.output.dir
2013-04-19 11:55:42,699 [LocalJobRunner Map Task Executor #0] INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(129)) - ProcfsBasedProcessTree currently is supported only on Linux.
2013-04-19 11:55:42,699 [LocalJobRunner Map Task Executor #0] INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : null
2013-04-19 11:55:42,703 [LocalJobRunner Map Task Executor #0] INFO mapred.MapTask (MapTask.java:runNewMapper(727)) - Processing split: Number of splits :1
Total Length = 131176
Input split[0]:
Length = 131176
Locations:
{code}
> Deprecation warnings in Configuration should go to their own log or otherwise be suppressible
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-9487
> URL: https://issues.apache.org/jira/browse/HADOOP-9487
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
>
> Running local pig jobs triggers large quantities of warnings about deprecated properties -something I don't care about as I'm not in a position to fix without delving into Pig.
> I can suppress them by changing the log level, but that can hide other warnings that may actually matter.
> If there was a special Configuration.deprecated log for all deprecation messages, this log could be suppressed by people who don't want noisy logs
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira