You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Cheolsoo Park <pi...@gmail.com> on 2013/12/02 05:54:36 UTC
Re: Unable to run PIG local as regular user (non-root)
Hi Daniel,
>From the console output, I can see Pig ran against Hadoop 1.x as root while
it ran against Hadoop 2.x as watrous. I don't know what your setup is like,
but please double check what version of Hadoop jars are present in
classpath. Running "pig -secretDebugCmd" should show HADOOP_CLASSPATH.
Apparently, your Pig jar is compiled against Hadoop 1.x, so it only works
against Hadoop 1.x (as root).
Thanks,
Cheolsoo
On Wed, Nov 27, 2013 at 11:01 AM, Watrous, Daniel
<da...@hp.com>wrote:
> Hi,
>
> I've been following the book Hadoop Definitive Guide and a few other
> resources while exploring PIG. I am able to run as root, but not as a
> regular user. I am running in local mode. I've found a few links, but they
> haven't helped me solve my issue. The most relevant seems to be:
> http://answers.mapr.com/questions/5135/cannot-run-pig-in-local-mode
>
> My first experiment is just to load in a text file and dump it to the
> console. The files exist and are owned by the user running PIG. I show the
> failed attempt as a regular user followed by a successful run as root on
> the same system. Any help is appreciated.
>
> Here's what I'm trying:
> [watrous@c0003913 ~]$ pig -x local
> 2013-11-27 18:53:18,691 [main] INFO org.apache.pig.Main - Apache Pig
> version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
> 2013-11-27 18:53:18,692 [main] INFO org.apache.pig.Main - Logging error
> messages to: /home/watrous/pig_1385578398689.log
> 2013-11-27 18:53:18,747 [main] INFO org.apache.pig.impl.util.Utils -
> Default bootup file /home/watrous/.pigbootup not found
> 2013-11-27 18:53:19,025 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2013-11-27 18:53:19,025 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
> deprecated. Instead, use mapreduce.jobtracker.address
> 2013-11-27 18:53:19,027 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: file:///
> 2013-11-27 18:53:19,031 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.used.genericoptionsparser is deprecated. Instead, use
> mapreduce.client.genericoptionsparser.used
> 2013-11-27 18:53:19,451 [main] WARN
> org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 2013-11-27 18:53:19,517 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - io.bytes.per.checksum
> is deprecated. Instead, use dfs.bytes-per-checksum
> 2013-11-27 18:53:19,519 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> grunt> ls /home/watrous/input/
> file:/home/watrous/input/sample.txt<r 1> 530
> grunt> cat /home/watrous/input/sample.txt
>
> 0067011990999991950051507004+68750+023550FM-12+038299999V0203301N00671220001CN9999999N9+00001+99999999999
>
> 0043011990999991950051512004+68750+023550FM-12+038299999V0203201N00671220001CN9999999N9+00221+99999999999
>
> 0043011990999991950051518004+68750+023550FM-12+038299999V0203201N00261220001CN9999999N9-00111+99999999999
>
> 0043012650999991949032412004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+01111+99999999999
>
> 0043012650999991949032418004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+00781+99999999999
> grunt> records = load '/home/watrous/input/sample.txt';
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - job.end.retry.interval
> is deprecated. Instead, use mapreduce.job.end-notification.retry.interval
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.retiredjobs.cache.size is deprecated. Instead, use
> mapreduce.jobtracker.retiredjobs.cache.size
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.profile.reduces is deprecated. Instead, use
> mapreduce.task.profile.reduces
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.reuse.jvm.num.tasks is deprecated. Instead, use
> mapreduce.job.jvm.numtasks
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.tracker.http.address is deprecated. Instead, use
> mapreduce.tasktracker.http.address
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.system.dir is
> deprecated. Instead, use mapreduce.jobtracker.system.dir
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.tracker.report.address is deprecated. Instead, use
> mapreduce.tasktracker.report.address
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.healthChecker.interval is deprecated. Instead, use
> mapreduce.tasktracker.healthchecker.interval
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.child.tmp is
> deprecated. Instead, use mapreduce.task.tmp.dir
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.taskmemorymanager.monitoring-interval is deprecated.
> Instead, use mapreduce.tasktracker.taskmemorymanager.monitoringinterval
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.shuffle.connect.timeout is deprecated. Instead, use
> mapreduce.reduce.shuffle.connect.timeout
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.speculative.execution.speculativeCap is deprecated. Instead, use
> mapreduce.job.speculative.speculativecap
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - io.sort.spill.percent
> is deprecated. Instead, use mapreduce.map.sort.spill.percent
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.shuffle.input.buffer.percent is deprecated. Instead, use
> mapreduce.reduce.shuffle.input.buffer.percent
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.skip.map.max.skip.records is deprecated. Instead, use
> mapreduce.map.skip.maxrecords
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.profile.maps is deprecated. Instead, use
> mapreduce.task.profile.maps
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.local.dir is
> deprecated. Instead, use mapreduce.cluster.local.dir
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.merge.recordsBeforeProgress is deprecated. Instead, use
> mapreduce.task.merge.progress.records
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.http.address is deprecated. Instead, use
> mapreduce.jobtracker.http.address
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.compress.map.output is deprecated. Instead, use
> mapreduce.map.output.compress
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.userlog.retain.hours is deprecated. Instead, use
> mapreduce.job.userlog.retain.hours
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.speculative.execution.slowNodeThreshold is deprecated. Instead, use
> mapreduce.job.speculative.slownodethreshold
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.reduce.tasks.maximum is deprecated. Instead, use
> mapreduce.tasktracker.reduce.tasks.maximum
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.restart.recover is deprecated. Instead, use
> mapreduce.jobtracker.restart.recover
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.child.log.level is deprecated. Instead, use
> mapreduce.reduce.log.level
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.inmem.merge.threshold is deprecated. Instead, use
> mapreduce.reduce.merge.inmem.threshold
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.acls.enabled is
> deprecated. Instead, use mapreduce.cluster.acls.enabled
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.dns.nameserver is deprecated. Instead, use
> mapreduce.tasktracker.dns.nameserver
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.submit.replication is deprecated. Instead, use
> mapreduce.client.submit.file.replication
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.map.child.log.level is deprecated. Instead, use
> mapreduce.map.log.level
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.map.max.attempts
> is deprecated. Instead, use mapreduce.map.maxattempts
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.shuffle.merge.percent is deprecated. Instead, use
> mapreduce.reduce.shuffle.merge.percent
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.jobhistory.lru.cache.size is deprecated. Instead, use
> mapreduce.jobtracker.jobhistory.lru.cache.size
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.job.queue.nameis deprecated. Instead, use mapreduce.job.queuename
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.persist.jobstatus.hours is deprecated. Instead, use
> mapreduce.jobtracker.persist.jobstatus.hours
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.healthChecker.script.timeout is deprecated. Instead, use
> mapreduce.tasktracker.healthchecker.script.timeout
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.map.tasks.maximum is deprecated. Instead, use
> mapreduce.tasktracker.map.tasks.maximum
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use
> mapreduce.reduce.markreset.buffer.percent
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> jobclient.completion.poll.interval is deprecated. Instead, use
> mapreduce.client.completion.pollinterval
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.persist.jobstatus.dir is deprecated. Instead, use
> mapreduce.jobtracker.persist.jobstatus.dir
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.slowstart.completed.maps is deprecated. Instead, use
> mapreduce.job.reduce.slowstart.completedmaps
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - io.sort.mb is
> deprecated. Instead, use mapreduce.task.io.sort.mb
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - dfs.umaskmode is
> deprecated. Instead, use fs.permissions.umask-mode
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.instrumentation is deprecated. Instead, use
> mapreduce.jobtracker.instrumentation
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> topology.node.switch.mapping.impl is deprecated. Instead, use
> net.topology.node.switch.mapping.impl
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.output.compression.type is deprecated. Instead, use
> mapreduce.output.fileoutputformat.compress.type
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.skip.attempts.to.start.skipping is deprecated. Instead, use
> mapreduce.task.skip.start.attempts
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.tracker.task-controller is deprecated. Instead, use
> mapreduce.tasktracker.taskcontroller
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.userlog.limit.kb
> is deprecated. Instead, use mapreduce.task.userlog.limit.kb
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use
> mapreduce.job.committer.setup.cleanup.needed
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.local.dir.minspacekill is deprecated. Instead, use
> mapreduce.tasktracker.local.dir.minspacekill
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapreduce.jobtracker.split.metainfo.maxsize is deprecated. Instead, use
> mapreduce.job.split.metainfo.maxsize
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> jobclient.progress.monitor.poll.interval is deprecated. Instead, use
> mapreduce.client.progressmonitor.pollinterval
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.min.split.size
> is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.output.compression.codec is deprecated. Instead, use
> mapreduce.output.fileoutputformat.compress.codec
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.task.profile is
> deprecated. Instead, use mapreduce.task.profile
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.parallel.copies is deprecated. Instead, use
> mapreduce.reduce.shuffle.parallelcopies
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - io.sort.factor is
> deprecated. Instead, use mapreduce.task.io.sort.factor
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.task.timeout is
> deprecated. Instead, use mapreduce.task.timeout
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.heartbeats.in.second is deprecated. Instead, use
> mapreduce.jobtracker.heartbeats.in.second
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.map.output.compression.codec is deprecated. Instead, use
> mapreduce.map.output.compress.codec
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.cache.levels is deprecated. Instead, use
> mapreduce.jobtracker.taskcache.levels
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.dns.interface is deprecated. Instead, use
> mapreduce.tasktracker.dns.interface
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.max.tracker.failures is deprecated. Instead, use
> mapreduce.job.maxtaskfailures.per.tracker
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - dfs.df.interval is
> deprecated. Instead, use fs.df.interval
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.tasks.sleeptime-before-sigkill is deprecated. Instead,
> use mapreduce.tasktracker.tasks.sleeptimebeforesigkill
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.max.tracker.blacklists is deprecated. Instead, use
> mapreduce.jobtracker.tasktracker.maxblacklists
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - jobclient.output.filter
> is deprecated. Instead, use mapreduce.client.output.filter
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.taskScheduler is deprecated. Instead, use
> mapreduce.jobtracker.taskscheduler
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - job.end.retry.attempts
> is deprecated. Instead, use mapreduce.job.end-notification.retry.attempts
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.speculative.execution.slowTaskThreshold is deprecated. Instead, use
> mapreduce.job.speculative.slowtaskthreshold
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.indexcache.mb is deprecated. Instead, use
> mapreduce.tasktracker.indexcache.mb
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> tasktracker.http.threads is deprecated. Instead, use
> mapreduce.tasktracker.http.threads
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.handler.count is deprecated. Instead, use
> mapreduce.jobtracker.handler.count
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - keep.failed.task.files
> is deprecated. Instead, use mapreduce.task.files.preserve.failedtasks
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.output.compress
> is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.job.history.block.size is deprecated. Instead, use
> mapreduce.jobtracker.jobhistory.block.size
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.skip.reduce.max.skip.groups is deprecated. Instead, use
> mapreduce.reduce.skip.maxgroups
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> topology.script.number.args is deprecated. Instead, use
> net.topology.script.number.args
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.local.dir.minspacestart is deprecated. Instead, use
> mapreduce.tasktracker.local.dir.minspacestart
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.maxtasks.per.job is deprecated. Instead, use
> mapreduce.jobtracker.maxtasks.perjob
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.max.attempts is deprecated. Instead, use
> mapreduce.reduce.maxattempts
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.shuffle.read.timeout is deprecated. Instead, use
> mapreduce.reduce.shuffle.read.timeout
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.instrumentation is deprecated. Instead, use
> mapreduce.tasktracker.instrumentation
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.expiry.interval is deprecated. Instead, use
> mapreduce.jobtracker.expire.trackers.interval
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.persist.jobstatus.active is deprecated. Instead, use
> mapreduce.jobtracker.persist.jobstatus.active
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.temp.dir is
> deprecated. Instead, use mapreduce.cluster.temp.dir
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapreduce.job.counters.limit is deprecated. Instead, use
> mapreduce.job.counters.max
> 2013-11-27 18:53:25,095 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.reduce.input.buffer.percent is deprecated. Instead, use
> mapreduce.reduce.input.buffer.percent
> grunt> dump records;
> 2013-11-27 18:53:31,060 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2013-11-27 18:53:31,118 [main] INFO
> org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
> {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune,
> DuplicateForEachColumnRewrite, GroupByConstParallelSetter,
> ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter,
> MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer,
> PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter],
> RULES_DISABLED=[FilterLogicExpressionSimplifier]}
> 2013-11-27 18:53:31,234 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> File concatenation threshold: 100 optimistic? false
> 2013-11-27 18:53:31,292 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2013-11-27 18:53:31,292 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2013-11-27 18:53:31,322 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - session.id is
> deprecated. Instead, use dfs.metrics.session-id
> 2013-11-27 18:53:31,323 [main] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 2013-11-27 18:53:31,367 [main] WARN
> org.apache.pig.backend.hadoop20.PigJobControl - falling back to default
> JobControl (not using hadoop 0.20 ?)
> java.lang.NoSuchFieldException: runnerState
> at java.lang.Class.getDeclaredField(Class.java:1899)
> at
> org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51)
> at
> org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:98)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:190)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
> at org.apache.pig.PigServer.storeEx(PigServer.java:978)
> at org.apache.pig.PigServer.store(PigServer.java:942)
> at org.apache.pig.PigServer.openIterator(PigServer.java:855)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.run(Main.java:541)
> at org.apache.pig.Main.main(Main.java:156)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> 2013-11-27 18:53:31,371 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
> to the job
> 2013-11-27 18:53:31,378 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2013-11-27 18:53:31,449 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2013-11-27 18:53:31,456 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Map only job, skipping reducer estimation
> 2013-11-27 18:53:31,499 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2013-11-27 18:53:31,515 [JobControl] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics
> with processName=JobTracker, sessionId= - already initialized
> 2013-11-27 18:53:31,524 [JobControl] ERROR
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl - Error while trying
> to run jobs.
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> at
> org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> at java.lang.Thread.run(Thread.java:679)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
> 2013-11-27 18:53:31,525 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2013-11-27 18:53:31,533 [main] WARN
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to
> stop immediately on failure.
> 2013-11-27 18:53:31,533 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - job null has failed! Stop running all dependent jobs
> 2013-11-27 18:53:31,534 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2013-11-27 18:53:31,545 [main] ERROR
> org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
> recreate exception from backend error: Unexpected System Error Occured:
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> at
> org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> at java.lang.Thread.run(Thread.java:679)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
>
> 2013-11-27 18:53:31,546 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2013-11-27 18:53:31,546 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats
> reported below may be incomplete
> 2013-11-27 18:53:31,547 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion PigVersion UserId StartedAt FinishedAt
> Features
> 2.2.0 0.12.0 watrous 2013-11-27 18:53:31 2013-11-27 18:53:31
> UNKNOWN
>
> Failed!
>
> Failed Jobs:
> JobId Alias Feature Message Outputs
> N/A records MAP_ONLY Message: Unexpected System Error Occured:
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> at
> org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> at java.lang.Thread.run(Thread.java:679)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
> file:/tmp/temp-884153250/tmp-1708053986,
>
> Input(s):
> Failed to read data from "/home/watrous/input/sample.txt"
>
> Output(s):
> Failed to produce result in "file:/tmp/temp-884153250/tmp-1708053986"
>
> Job DAG:
> null
>
>
> 2013-11-27 18:53:31,547 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed!
> 2013-11-27 18:53:31,553 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1066: Unable to open iterator for alias records
> Details at logfile: /home/watrous/pig_1385578398689.log
> grunt>
>
> The exact same process works fine it I'm root:
> [watrous@c0003913 ~]$ sudo su
> [root@c0003913 watrous]# pig -x local
> which: no hadoop in
> (/sbin:/bin:/usr/sbin:/usr/bin:/opt/mtp/bin:/home/watrous/pig-0.12.0/bin)
> 2013-11-27 18:58:34,166 [main] INFO org.apache.pig.Main - Apache Pig
> version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
> 2013-11-27 18:58:34,167 [main] INFO org.apache.pig.Main - Logging error
> messages to: /home/watrous/pig_1385578714163.log
> 2013-11-27 18:58:34,184 [main] INFO org.apache.pig.impl.util.Utils -
> Default bootup file /root/.pigbootup not found
> 2013-11-27 18:58:34,321 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: file:///
> grunt> records = load '/home/watrous/input/sample.txt';
> grunt> dump records;
> 2013-11-27 18:59:17,019 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2013-11-27 18:59:17,082 [main] INFO
> org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
> {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune,
> DuplicateForEachColumnRewrite, GroupByConstParallelSetter,
> ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter,
> MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer,
> PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter],
> RULES_DISABLED=[FilterLogicExpressionSimplifier]}
> 2013-11-27 18:59:17,185 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> File concatenation threshold: 100 optimistic? false
> 2013-11-27 18:59:17,236 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2013-11-27 18:59:17,237 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2013-11-27 18:59:17,278 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
> to the job
> 2013-11-27 18:59:17,301 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2013-11-27 18:59:17,338 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2013-11-27 18:59:17,344 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Map only job, skipping reducer estimation
> 2013-11-27 18:59:17,375 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2013-11-27 18:59:17,387 [JobControl] WARN
> org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 2013-11-27 18:59:17,392 [JobControl] WARN
> org.apache.hadoop.mapred.JobClient - No job jar file set. User classes
> may not be found. See JobConf(Class) or JobConf#setJar(String).
> 2013-11-27 18:59:17,450 [JobControl] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
> to process : 1
> 2013-11-27 18:59:17,450 [JobControl] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input paths to process : 1
> 2013-11-27 18:59:17,459 [JobControl] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input paths (combined) to process : 1
> 2013-11-27 18:59:17,812 [Thread-3] INFO
> org.apache.hadoop.util.ProcessTree - setsid exited with exit code 0
> 2013-11-27 18:59:17,862 [Thread-3] INFO org.apache.hadoop.mapred.Task -
> Using ResourceCalculatorPlugin :
> org.apache.hadoop.util.LinuxResourceCalculatorPlugin@79ca209e
> 2013-11-27 18:59:17,877 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_local_0001
> 2013-11-27 18:59:17,877 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Processing aliases records
> 2013-11-27 18:59:17,877 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - detailed locations: M: records[1,10] C: R:
> 2013-11-27 18:59:17,884 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2013-11-27 18:59:17,921 [Thread-3] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader
> - Current split being processed file:/home/watrous/input/sample.txt:0+530
> 2013-11-27 18:59:17,979 [Thread-3] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map
> - Aliases being processed per job phase (AliasName[line,offset]): M:
> records[1,10] C: R:
> 2013-11-27 18:59:17,995 [Thread-3] INFO org.apache.hadoop.mapred.Task -
> Task:attempt_local_0001_m_000000_0 is done. And is in the process of
> commiting
> 2013-11-27 18:59:18,000 [Thread-3] INFO
> org.apache.hadoop.mapred.LocalJobRunner -
> 2013-11-27 18:59:18,001 [Thread-3] INFO org.apache.hadoop.mapred.Task -
> Task attempt_local_0001_m_000000_0 is allowed to commit now
> 2013-11-27 18:59:18,004 [Thread-3] INFO
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output
> of task 'attempt_local_0001_m_000000_0' to
> file:/tmp/temp-1635315632/tmp501386857
> 2013-11-27 18:59:20,734 [Thread-3] INFO
> org.apache.hadoop.mapred.LocalJobRunner -
> 2013-11-27 18:59:20,751 [Thread-3] INFO org.apache.hadoop.mapred.Task -
> Task 'attempt_local_0001_m_000000_0' done.
> 2013-11-27 18:59:20,893 [main] WARN
> org.apache.pig.tools.pigstats.PigStatsUtil - Failed to get RunningJob for
> job job_local_0001
> 2013-11-27 18:59:20,896 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2013-11-27 18:59:20,896 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats
> reported below may be incomplete
> 2013-11-27 18:59:20,898 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion PigVersion UserId StartedAt FinishedAt
> Features
> 1.0.0 0.12.0 root 2013-11-27 18:59:17 2013-11-27 18:59:20
> UNKNOWN
>
> Success!
>
> Job Stats (time in seconds):
> JobId Alias Feature Outputs
> job_local_0001 records MAP_ONLY
> file:/tmp/temp-1635315632/tmp501386857,
>
> Input(s):
> Successfully read records from: "/home/watrous/input/sample.txt"
>
> Output(s):
> Successfully stored records in: "file:/tmp/temp-1635315632/tmp501386857"
>
> Job DAG:
> job_local_0001
>
>
> 2013-11-27 18:59:20,905 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Success!
> 2013-11-27 18:59:20,908 [main] WARN
> org.apache.pig.data.SchemaTupleBackend - SchemaTupleBackend has already
> been initialized
> 2013-11-27 18:59:20,911 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
> to process : 1
> 2013-11-27 18:59:20,911 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input paths to process : 1
>
> (0067011990999991950051507004+68750+023550FM-12+038299999V0203301N00671220001CN9999999N9+00001+99999999999)
>
> (0043011990999991950051512004+68750+023550FM-12+038299999V0203201N00671220001CN9999999N9+00221+99999999999)
>
> (0043011990999991950051518004+68750+023550FM-12+038299999V0203201N00261220001CN9999999N9-00111+99999999999)
>
> (0043012650999991949032412004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+01111+99999999999)
>
> (0043012650999991949032418004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+00781+99999999999)
> grunt>
>
Re: Unable to run PIG local as regular user (non-root)
Posted by Cheolsoo Park <pi...@gmail.com>.
Hi Daniel,
The 0.12 binary release is compiled with Hadoop 1.x, so you will need to
recompile Pig with Hadoop 2.x from source code. We also ran tests against
Hadoop 2.0.x before the 0.12 release.
Thanks,
Cheolsoo
On Mon, Dec 2, 2013 at 9:41 AM, Watrous, Daniel <da...@hp.com>wrote:
> Cheolsoo,
>
> I hadn't noticed that. I do have hadoop 2.2.0 installed. Does the PIG
> distribution include hadoop v1? I suppose that if my environment variables
> aren't set correctly for hadoop 2 as root, then it may default to version 1
> of hadoop. The reason for this is that the path to hadoop/bin is part of
> PATH as my user, but not as root.
>
> I have confirmed this by removing the path to hadoop binaries from my PATH
> environment variable and running it locally. Everything worked identically.
> To correct this permanently I should either move backward to version 1.x of
> hadoop or compile PIG with hadoop 2.x support. Has PIG been tested against
> hadoop 2.x?
>
> Daniel
>
> -----Original Message-----
> From: Cheolsoo Park [mailto:piaozhexiu@gmail.com]
> Sent: Sunday, December 01, 2013 9:55 PM
> To: user@pig.apache.org
> Subject: Re: Unable to run PIG local as regular user (non-root)
>
> Hi Daniel,
>
> From the console output, I can see Pig ran against Hadoop 1.x as root
> while it ran against Hadoop 2.x as watrous. I don't know what your setup is
> like, but please double check what version of Hadoop jars are present in
> classpath. Running "pig -secretDebugCmd" should show HADOOP_CLASSPATH.
>
> Apparently, your Pig jar is compiled against Hadoop 1.x, so it only works
> against Hadoop 1.x (as root).
>
> Thanks,
> Cheolsoo
>
>
>
> On Wed, Nov 27, 2013 at 11:01 AM, Watrous, Daniel
> <da...@hp.com>wrote:
>
> > Hi,
> >
> > I've been following the book Hadoop Definitive Guide and a few other
> > resources while exploring PIG. I am able to run as root, but not as a
> > regular user. I am running in local mode. I've found a few links, but
> > they haven't helped me solve my issue. The most relevant seems to be:
> > http://answers.mapr.com/questions/5135/cannot-run-pig-in-local-mode
> >
> > My first experiment is just to load in a text file and dump it to the
> > console. The files exist and are owned by the user running PIG. I show
> > the failed attempt as a regular user followed by a successful run as
> > root on the same system. Any help is appreciated.
> >
> > Here's what I'm trying:
> > [watrous@c0003913 ~]$ pig -x local
> > 2013-11-27 18:53:18,691 [main] INFO org.apache.pig.Main - Apache Pig
> > version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
> > 2013-11-27 18:53:18,692 [main] INFO org.apache.pig.Main - Logging
> > error messages to: /home/watrous/pig_1385578398689.log
> > 2013-11-27 18:53:18,747 [main] INFO org.apache.pig.impl.util.Utils -
> > Default bootup file /home/watrous/.pigbootup not found
> > 2013-11-27 18:53:19,025 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> > deprecated. Instead, use fs.defaultFS
> > 2013-11-27 18:53:19,025 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker
> > is deprecated. Instead, use mapreduce.jobtracker.address
> > 2013-11-27 18:53:19,027 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> > Connecting to hadoop file system at: file:///
> > 2013-11-27 18:53:19,031 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.used.genericoptionsparser is deprecated. Instead, use
> > mapreduce.client.genericoptionsparser.used
> > 2013-11-27 18:53:19,451 [main] WARN
> > org.apache.hadoop.util.NativeCodeLoader - Unable to load
> > native-hadoop library for your platform... using builtin-java classes
> > where applicable
> > 2013-11-27 18:53:19,517 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > io.bytes.per.checksum is deprecated. Instead, use
> > dfs.bytes-per-checksum
> > 2013-11-27 18:53:19,519 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> > deprecated. Instead, use fs.defaultFS
> > grunt> ls /home/watrous/input/
> > file:/home/watrous/input/sample.txt<r 1> 530
> > grunt> cat /home/watrous/input/sample.txt
> >
> >
> 0067011990999991950051507004+68750+023550FM-12+038299999V0203301N00671220001CN9999999N9+00001+99999999999
> >
> >
> 0043011990999991950051512004+68750+023550FM-12+038299999V0203201N00671220001CN9999999N9+00221+99999999999
> >
> >
> 0043011990999991950051518004+68750+023550FM-12+038299999V0203201N00261220001CN9999999N9-00111+99999999999
> >
> >
> 0043012650999991949032412004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+01111+99999999999
> >
> >
> 0043012650999991949032418004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+00781+99999999999
> > grunt> records = load '/home/watrous/input/sample.txt';
> > 2013-11-27 18:53:25,086 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > job.end.retry.interval is deprecated. Instead, use
> > mapreduce.job.end-notification.retry.interval
> > 2013-11-27 18:53:25,086 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.tracker.retiredjobs.cache.size is deprecated. Instead, use
> > mapreduce.jobtracker.retiredjobs.cache.size
> > 2013-11-27 18:53:25,086 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.task.profile.reduces is deprecated. Instead, use
> > mapreduce.task.profile.reduces
> > 2013-11-27 18:53:25,086 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.reuse.jvm.num.tasks is deprecated. Instead, use
> > mapreduce.job.jvm.numtasks
> > 2013-11-27 18:53:25,086 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> > mapreduce.reduce.speculative
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.task.tracker.http.address is deprecated. Instead, use
> > mapreduce.tasktracker.http.address
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - mapred.system.dir
> > is deprecated. Instead, use mapreduce.jobtracker.system.dir
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.task.tracker.report.address is deprecated. Instead, use
> > mapreduce.tasktracker.report.address
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.healthChecker.interval is deprecated. Instead, use
> > mapreduce.tasktracker.healthchecker.interval
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - mapred.child.tmp
> > is deprecated. Instead, use mapreduce.task.tmp.dir
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.taskmemorymanager.monitoring-interval is deprecated.
> > Instead, use
> > mapreduce.tasktracker.taskmemorymanager.monitoringinterval
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.shuffle.connect.timeout is deprecated. Instead, use
> > mapreduce.reduce.shuffle.connect.timeout
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.speculative.execution.speculativeCap is deprecated. Instead,
> > use mapreduce.job.speculative.speculativecap
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > io.sort.spill.percent is deprecated. Instead, use
> > mapreduce.map.sort.spill.percent
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.shuffle.input.buffer.percent is deprecated. Instead, use
> > mapreduce.reduce.shuffle.input.buffer.percent
> > 2013-11-27 18:53:25,087 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.skip.map.max.skip.records is deprecated. Instead, use
> > mapreduce.map.skip.maxrecords
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.task.profile.maps is deprecated. Instead, use
> > mapreduce.task.profile.maps
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - mapred.local.dir
> > is deprecated. Instead, use mapreduce.cluster.local.dir
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.merge.recordsBeforeProgress is deprecated. Instead, use
> > mapreduce.task.merge.progress.records
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.tracker.http.address is deprecated. Instead, use
> > mapreduce.jobtracker.http.address
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.compress.map.output is deprecated. Instead, use
> > mapreduce.map.output.compress
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.userlog.retain.hours is deprecated. Instead, use
> > mapreduce.job.userlog.retain.hours
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.speculative.execution.slowNodeThreshold is deprecated. Instead,
> > use mapreduce.job.speculative.slownodethreshold
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.reduce.tasks.maximum is deprecated. Instead, use
> > mapreduce.tasktracker.reduce.tasks.maximum
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.jobtracker.restart.recover is deprecated. Instead, use
> > mapreduce.jobtracker.restart.recover
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.reduce.child.log.level is deprecated. Instead, use
> > mapreduce.reduce.log.level
> > 2013-11-27 18:53:25,088 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.inmem.merge.threshold is deprecated. Instead, use
> > mapreduce.reduce.merge.inmem.threshold
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.acls.enabled is deprecated. Instead, use
> > mapreduce.cluster.acls.enabled
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.dns.nameserver is deprecated. Instead, use
> > mapreduce.tasktracker.dns.nameserver
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.submit.replication is deprecated. Instead, use
> > mapreduce.client.submit.file.replication
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.map.tasks.speculative.execution is deprecated. Instead, use
> > mapreduce.map.speculative
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.map.child.log.level is deprecated. Instead, use
> > mapreduce.map.log.level
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.map.max.attempts is deprecated. Instead, use
> > mapreduce.map.maxattempts
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.shuffle.merge.percent is deprecated. Instead, use
> > mapreduce.reduce.shuffle.merge.percent
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.tracker.jobhistory.lru.cache.size is deprecated. Instead,
> > use mapreduce.jobtracker.jobhistory.lru.cache.size
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.queue.nameis deprecated. Instead, use
> > mapreduce.job.queuename
> > 2013-11-27 18:53:25,089 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.tracker.persist.jobstatus.hours is deprecated. Instead, use
> > mapreduce.jobtracker.persist.jobstatus.hours
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.healthChecker.script.timeout is deprecated. Instead, use
> > mapreduce.tasktracker.healthchecker.script.timeout
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.map.tasks.maximum is deprecated. Instead, use
> > mapreduce.tasktracker.map.tasks.maximum
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use
> > mapreduce.reduce.markreset.buffer.percent
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > jobclient.completion.poll.interval is deprecated. Instead, use
> > mapreduce.client.completion.pollinterval
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.tracker.persist.jobstatus.dir is deprecated. Instead, use
> > mapreduce.jobtracker.persist.jobstatus.dir
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.reduce.slowstart.completed.maps is deprecated. Instead, use
> > mapreduce.job.reduce.slowstart.completedmaps
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - io.sort.mb is
> > deprecated. Instead, use mapreduce.task.io.sort.mb
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - dfs.umaskmode is
> > deprecated. Instead, use fs.permissions.umask-mode
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.jobtracker.instrumentation is deprecated. Instead, use
> > mapreduce.jobtracker.instrumentation
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > topology.node.switch.mapping.impl is deprecated. Instead, use
> > net.topology.node.switch.mapping.impl
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.output.compression.type is deprecated. Instead, use
> > mapreduce.output.fileoutputformat.compress.type
> > 2013-11-27 18:53:25,090 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.skip.attempts.to.start.skipping is deprecated. Instead, use
> > mapreduce.task.skip.start.attempts
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.task.tracker.task-controller is deprecated. Instead, use
> > mapreduce.tasktracker.taskcontroller
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.userlog.limit.kb is deprecated. Instead, use
> > mapreduce.task.userlog.limit.kb
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use
> > mapreduce.job.committer.setup.cleanup.needed
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - mapred.map.tasks
> > is deprecated. Instead, use mapreduce.job.maps
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.local.dir.minspacekill is deprecated. Instead, use
> > mapreduce.tasktracker.local.dir.minspacekill
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapreduce.jobtracker.split.metainfo.maxsize is deprecated. Instead,
> > use mapreduce.job.split.metainfo.maxsize
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > jobclient.progress.monitor.poll.interval is deprecated. Instead, use
> > mapreduce.client.progressmonitor.pollinterval
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.min.split.size is deprecated. Instead, use
> > mapreduce.input.fileinputformat.split.minsize
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.output.compression.codec is deprecated. Instead, use
> > mapreduce.output.fileoutputformat.compress.codec
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.task.profile is deprecated. Instead, use mapreduce.task.profile
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.reduce.parallel.copies is deprecated. Instead, use
> > mapreduce.reduce.shuffle.parallelcopies
> > 2013-11-27 18:53:25,091 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - io.sort.factor is
> > deprecated. Instead, use mapreduce.task.io.sort.factor
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.task.timeout is deprecated. Instead, use mapreduce.task.timeout
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.heartbeats.in.second is deprecated. Instead, use
> > mapreduce.jobtracker.heartbeats.in.second
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.map.output.compression.codec is deprecated. Instead, use
> > mapreduce.map.output.compress.codec
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.task.cache.levels is deprecated. Instead, use
> > mapreduce.jobtracker.taskcache.levels
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.dns.interface is deprecated. Instead, use
> > mapreduce.tasktracker.dns.interface
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.max.tracker.failures is deprecated. Instead, use
> > mapreduce.job.maxtaskfailures.per.tracker
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - dfs.df.interval is
> > deprecated. Instead, use fs.df.interval
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.tasks.sleeptime-before-sigkill is deprecated.
> > Instead, use mapreduce.tasktracker.tasks.sleeptimebeforesigkill
> > 2013-11-27 18:53:25,092 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.max.tracker.blacklists is deprecated. Instead, use
> > mapreduce.jobtracker.tasktracker.maxblacklists
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > jobclient.output.filter is deprecated. Instead, use
> > mapreduce.client.output.filter
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.jobtracker.taskScheduler is deprecated. Instead, use
> > mapreduce.jobtracker.taskscheduler
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > job.end.retry.attempts is deprecated. Instead, use
> > mapreduce.job.end-notification.retry.attempts
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.speculative.execution.slowTaskThreshold is deprecated. Instead,
> > use mapreduce.job.speculative.slowtaskthreshold
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.indexcache.mb is deprecated. Instead, use
> > mapreduce.tasktracker.indexcache.mb
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > tasktracker.http.threads is deprecated. Instead, use
> > mapreduce.tasktracker.http.threads
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.tracker.handler.count is deprecated. Instead, use
> > mapreduce.jobtracker.handler.count
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > keep.failed.task.files is deprecated. Instead, use
> > mapreduce.task.files.preserve.failedtasks
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.output.compress is deprecated. Instead, use
> > mapreduce.output.fileoutputformat.compress
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.jobtracker.job.history.block.size is deprecated. Instead, use
> > mapreduce.jobtracker.jobhistory.block.size
> > 2013-11-27 18:53:25,093 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.skip.reduce.max.skip.groups is deprecated. Instead, use
> > mapreduce.reduce.skip.maxgroups
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > topology.script.number.args is deprecated. Instead, use
> > net.topology.script.number.args
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.local.dir.minspacestart is deprecated. Instead, use
> > mapreduce.tasktracker.local.dir.minspacestart
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.jobtracker.maxtasks.per.job is deprecated. Instead, use
> > mapreduce.jobtracker.maxtasks.perjob
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.reduce.max.attempts is deprecated. Instead, use
> > mapreduce.reduce.maxattempts
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.shuffle.read.timeout is deprecated. Instead, use
> > mapreduce.reduce.shuffle.read.timeout
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.instrumentation is deprecated. Instead, use
> > mapreduce.tasktracker.instrumentation
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.tasktracker.expiry.interval is deprecated. Instead, use
> > mapreduce.jobtracker.expire.trackers.interval
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.tracker.persist.jobstatus.active is deprecated. Instead,
> > use mapreduce.jobtracker.persist.jobstatus.active
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - mapred.temp.dir is
> > deprecated. Instead, use mapreduce.cluster.temp.dir
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - hadoop.native.lib
> > is deprecated. Instead, use io.native.lib.available
> > 2013-11-27 18:53:25,094 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapreduce.job.counters.limit is deprecated. Instead, use
> > mapreduce.job.counters.max
> > 2013-11-27 18:53:25,095 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation -
> > mapred.job.reduce.input.buffer.percent is deprecated. Instead, use
> > mapreduce.reduce.input.buffer.percent
> > grunt> dump records;
> > 2013-11-27 18:53:31,060 [main] INFO
> > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> > script: UNKNOWN
> > 2013-11-27 18:53:31,118 [main] INFO
> > org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
> > {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune,
> > DuplicateForEachColumnRewrite, GroupByConstParallelSetter,
> > ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter,
> > MergeFilter, MergeForEach, NewPartitionFilterOptimizer,
> > PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter,
> > SplitFilter, StreamTypeCastInserter],
> > RULES_DISABLED=[FilterLogicExpressionSimplifier]}
> > 2013-11-27 18:53:31,234 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompile
> > r - File concatenation threshold: 100 optimistic? false
> > 2013-11-27 18:53:31,292 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> > yOptimizer
> > - MR plan size before optimization: 1
> > 2013-11-27 18:53:31,292 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> > yOptimizer
> > - MR plan size after optimization: 1
> > 2013-11-27 18:53:31,322 [main] INFO
> > org.apache.hadoop.conf.Configuration.deprecation - session.id is
> > deprecated. Instead, use dfs.metrics.session-id
> > 2013-11-27 18:53:31,323 [main] INFO
> > org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics
> > with processName=JobTracker, sessionId=
> > 2013-11-27 18:53:31,367 [main] WARN
> > org.apache.pig.backend.hadoop20.PigJobControl - falling back to
> > default JobControl (not using hadoop 0.20 ?)
> > java.lang.NoSuchFieldException: runnerState
> > at java.lang.Class.getDeclaredField(Class.java:1899)
> > at
> >
> org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:98)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:287)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:190)
> > at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
> > at
> > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
> > at org.apache.pig.PigServer.storeEx(PigServer.java:978)
> > at org.apache.pig.PigServer.store(PigServer.java:942)
> > at org.apache.pig.PigServer.openIterator(PigServer.java:855)
> > at
> > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
> > at
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
> > at
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
> > at
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
> > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> > at org.apache.pig.Main.run(Main.java:541)
> > at org.apache.pig.Main.main(Main.java:156)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:616)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> > 2013-11-27 18:53:31,371 [main] INFO
> > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are
> > added to the job
> > 2013-11-27 18:53:31,378 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> > lCompiler
> > - mapred.job.reduce.markreset.buffer.percent is not set, set to
> > default 0.3
> > 2013-11-27 18:53:31,449 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> > lCompiler
> > - Setting up single store job
> > 2013-11-27 18:53:31,456 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> > lCompiler
> > - Map only job, skipping reducer estimation
> > 2013-11-27 18:53:31,499 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - 1 map-reduce job(s) waiting for submission.
> > 2013-11-27 18:53:31,515 [JobControl] INFO
> > org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM
> > Metrics with processName=JobTracker, sessionId= - already initialized
> > 2013-11-27 18:53:31,524 [JobControl] ERROR
> > org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl - Error while
> > trying to run jobs.
> > java.lang.IncompatibleClassChangeError: Found interface
> > org.apache.hadoop.mapreduce.JobContext, but class was expected
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> > at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> > at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:416)
> > at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> > at
> >
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> > at
> >
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> > at
> > org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> > at java.lang.Thread.run(Thread.java:679)
> > at
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher$1.run(MapReduceLauncher.java:270)
> > 2013-11-27 18:53:31,525 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - 0% complete
> > 2013-11-27 18:53:31,533 [main] WARN
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig
> > to stop immediately on failure.
> > 2013-11-27 18:53:31,533 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - job null has failed! Stop running all dependent jobs
> > 2013-11-27 18:53:31,534 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - 100% complete
> > 2013-11-27 18:53:31,545 [main] ERROR
> > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
> > recreate exception from backend error: Unexpected System Error Occured:
> > java.lang.IncompatibleClassChangeError: Found interface
> > org.apache.hadoop.mapreduce.JobContext, but class was expected
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> > at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> > at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:416)
> > at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> > at
> >
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> > at
> >
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> > at
> > org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> > at java.lang.Thread.run(Thread.java:679)
> > at
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher$1.run(MapReduceLauncher.java:270)
> >
> > 2013-11-27 18:53:31,546 [main] ERROR
> > org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> > 2013-11-27 18:53:31,546 [main] INFO
> > org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode.
> > Stats reported below may be incomplete
> > 2013-11-27 18:53:31,547 [main] INFO
> > org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
> >
> > HadoopVersion PigVersion UserId StartedAt FinishedAt
> > Features
> > 2.2.0 0.12.0 watrous 2013-11-27 18:53:31 2013-11-27 18:53:31
> > UNKNOWN
> >
> > Failed!
> >
> > Failed Jobs:
> > JobId Alias Feature Message Outputs
> > N/A records MAP_ONLY Message: Unexpected System Error Occured:
> > java.lang.IncompatibleClassChangeError: Found interface
> > org.apache.hadoop.mapreduce.JobContext, but class was expected
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> > at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> > at
> >
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:416)
> > at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> > at
> >
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> > at
> >
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> > at
> > org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> > at java.lang.Thread.run(Thread.java:679)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
> > file:/tmp/temp-884153250/tmp-1708053986,
> >
> > Input(s):
> > Failed to read data from "/home/watrous/input/sample.txt"
> >
> > Output(s):
> > Failed to produce result in "file:/tmp/temp-884153250/tmp-1708053986"
> >
> > Job DAG:
> > null
> >
> >
> > 2013-11-27 18:53:31,547 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - Failed!
> > 2013-11-27 18:53:31,553 [main] ERROR org.apache.pig.tools.grunt.Grunt
> > - ERROR 1066: Unable to open iterator for alias records Details at
> > logfile: /home/watrous/pig_1385578398689.log
> > grunt>
> >
> > The exact same process works fine it I'm root:
> > [watrous@c0003913 ~]$ sudo su
> > [root@c0003913 watrous]# pig -x local
> > which: no hadoop in
> > (/sbin:/bin:/usr/sbin:/usr/bin:/opt/mtp/bin:/home/watrous/pig-0.12.0/b
> > in)
> > 2013-11-27 18:58:34,166 [main] INFO org.apache.pig.Main - Apache Pig
> > version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
> > 2013-11-27 18:58:34,167 [main] INFO org.apache.pig.Main - Logging
> > error messages to: /home/watrous/pig_1385578714163.log
> > 2013-11-27 18:58:34,184 [main] INFO org.apache.pig.impl.util.Utils -
> > Default bootup file /root/.pigbootup not found
> > 2013-11-27 18:58:34,321 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> > Connecting to hadoop file system at: file:///
> > grunt> records = load '/home/watrous/input/sample.txt'; dump records;
> > 2013-11-27 18:59:17,019 [main] INFO
> > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> > script: UNKNOWN
> > 2013-11-27 18:59:17,082 [main] INFO
> > org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
> > {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune,
> > DuplicateForEachColumnRewrite, GroupByConstParallelSetter,
> > ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter,
> > MergeFilter, MergeForEach, NewPartitionFilterOptimizer,
> > PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter,
> > SplitFilter, StreamTypeCastInserter],
> > RULES_DISABLED=[FilterLogicExpressionSimplifier]}
> > 2013-11-27 18:59:17,185 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompile
> > r - File concatenation threshold: 100 optimistic? false
> > 2013-11-27 18:59:17,236 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> > yOptimizer
> > - MR plan size before optimization: 1
> > 2013-11-27 18:59:17,237 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> > yOptimizer
> > - MR plan size after optimization: 1
> > 2013-11-27 18:59:17,278 [main] INFO
> > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are
> > added to the job
> > 2013-11-27 18:59:17,301 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> > lCompiler
> > - mapred.job.reduce.markreset.buffer.percent is not set, set to
> > default 0.3
> > 2013-11-27 18:59:17,338 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> > lCompiler
> > - Setting up single store job
> > 2013-11-27 18:59:17,344 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> > lCompiler
> > - Map only job, skipping reducer estimation
> > 2013-11-27 18:59:17,375 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - 1 map-reduce job(s) waiting for submission.
> > 2013-11-27 18:59:17,387 [JobControl] WARN
> > org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
> > library for your platform... using builtin-java classes where
> > applicable
> > 2013-11-27 18:59:17,392 [JobControl] WARN
> > org.apache.hadoop.mapred.JobClient - No job jar file set. User
> > classes may not be found. See JobConf(Class) or JobConf#setJar(String).
> > 2013-11-27 18:59:17,450 [JobControl] INFO
> > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input
> > paths to process : 1
> > 2013-11-27 18:59:17,450 [JobControl] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> > input paths to process : 1
> > 2013-11-27 18:59:17,459 [JobControl] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> > input paths (combined) to process : 1
> > 2013-11-27 18:59:17,812 [Thread-3] INFO
> > org.apache.hadoop.util.ProcessTree - setsid exited with exit code 0
> > 2013-11-27 18:59:17,862 [Thread-3] INFO org.apache.hadoop.mapred.Task
> > - Using ResourceCalculatorPlugin :
> > org.apache.hadoop.util.LinuxResourceCalculatorPlugin@79ca209e
> > 2013-11-27 18:59:17,877 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - HadoopJobId: job_local_0001
> > 2013-11-27 18:59:17,877 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - Processing aliases records
> > 2013-11-27 18:59:17,877 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - detailed locations: M: records[1,10] C: R:
> > 2013-11-27 18:59:17,884 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - 0% complete
> > 2013-11-27 18:59:17,921 [Thread-3] INFO
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecord
> > Reader
> > - Current split being processed
> > file:/home/watrous/input/sample.txt:0+530
> > 2013-11-27 18:59:17,979 [Thread-3] INFO
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnl
> > y$Map
> > - Aliases being processed per job phase (AliasName[line,offset]): M:
> > records[1,10] C: R:
> > 2013-11-27 18:59:17,995 [Thread-3] INFO org.apache.hadoop.mapred.Task
> > -
> > Task:attempt_local_0001_m_000000_0 is done. And is in the process of
> > commiting
> > 2013-11-27 18:59:18,000 [Thread-3] INFO
> > org.apache.hadoop.mapred.LocalJobRunner -
> > 2013-11-27 18:59:18,001 [Thread-3] INFO org.apache.hadoop.mapred.Task
> > - Task attempt_local_0001_m_000000_0 is allowed to commit now
> > 2013-11-27 18:59:18,004 [Thread-3] INFO
> > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved
> > output of task 'attempt_local_0001_m_000000_0' to
> > file:/tmp/temp-1635315632/tmp501386857
> > 2013-11-27 18:59:20,734 [Thread-3] INFO
> > org.apache.hadoop.mapred.LocalJobRunner -
> > 2013-11-27 18:59:20,751 [Thread-3] INFO org.apache.hadoop.mapred.Task
> > - Task 'attempt_local_0001_m_000000_0' done.
> > 2013-11-27 18:59:20,893 [main] WARN
> > org.apache.pig.tools.pigstats.PigStatsUtil - Failed to get RunningJob
> > for job job_local_0001
> > 2013-11-27 18:59:20,896 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - 100% complete
> > 2013-11-27 18:59:20,896 [main] INFO
> > org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode.
> > Stats reported below may be incomplete
> > 2013-11-27 18:59:20,898 [main] INFO
> > org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
> >
> > HadoopVersion PigVersion UserId StartedAt FinishedAt
> > Features
> > 1.0.0 0.12.0 root 2013-11-27 18:59:17 2013-11-27 18:59:20
> > UNKNOWN
> >
> > Success!
> >
> > Job Stats (time in seconds):
> > JobId Alias Feature Outputs
> > job_local_0001 records MAP_ONLY
> > file:/tmp/temp-1635315632/tmp501386857,
> >
> > Input(s):
> > Successfully read records from: "/home/watrous/input/sample.txt"
> >
> > Output(s):
> > Successfully stored records in: "file:/tmp/temp-1635315632/tmp501386857"
> >
> > Job DAG:
> > job_local_0001
> >
> >
> > 2013-11-27 18:59:20,905 [main] INFO
> >
> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> > Launcher
> > - Success!
> > 2013-11-27 18:59:20,908 [main] WARN
> > org.apache.pig.data.SchemaTupleBackend - SchemaTupleBackend has
> > already been initialized
> > 2013-11-27 18:59:20,911 [main] INFO
> > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input
> > paths to process : 1
> > 2013-11-27 18:59:20,911 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> > input paths to process : 1
> >
> > (0067011990999991950051507004+68750+023550FM-12+038299999V0203301N0067
> > 1220001CN9999999N9+00001+99999999999)
> >
> > (0043011990999991950051512004+68750+023550FM-12+038299999V0203201N0067
> > 1220001CN9999999N9+00221+99999999999)
> >
> > (0043011990999991950051518004+68750+023550FM-12+038299999V0203201N0026
> > 1220001CN9999999N9-00111+99999999999)
> >
> > (0043012650999991949032412004+62300+010750FM-12+048599999V0202701N0046
> > 1220001CN0500001N9+01111+99999999999)
> >
> > (0043012650999991949032418004+62300+010750FM-12+048599999V0202701N0046
> > 1220001CN0500001N9+00781+99999999999)
> > grunt>
> >
>
RE: Unable to run PIG local as regular user (non-root)
Posted by "Watrous, Daniel" <da...@hp.com>.
Cheolsoo,
I hadn't noticed that. I do have hadoop 2.2.0 installed. Does the PIG distribution include hadoop v1? I suppose that if my environment variables aren't set correctly for hadoop 2 as root, then it may default to version 1 of hadoop. The reason for this is that the path to hadoop/bin is part of PATH as my user, but not as root.
I have confirmed this by removing the path to hadoop binaries from my PATH environment variable and running it locally. Everything worked identically. To correct this permanently I should either move backward to version 1.x of hadoop or compile PIG with hadoop 2.x support. Has PIG been tested against hadoop 2.x?
Daniel
-----Original Message-----
From: Cheolsoo Park [mailto:piaozhexiu@gmail.com]
Sent: Sunday, December 01, 2013 9:55 PM
To: user@pig.apache.org
Subject: Re: Unable to run PIG local as regular user (non-root)
Hi Daniel,
>From the console output, I can see Pig ran against Hadoop 1.x as root while it ran against Hadoop 2.x as watrous. I don't know what your setup is like, but please double check what version of Hadoop jars are present in classpath. Running "pig -secretDebugCmd" should show HADOOP_CLASSPATH.
Apparently, your Pig jar is compiled against Hadoop 1.x, so it only works against Hadoop 1.x (as root).
Thanks,
Cheolsoo
On Wed, Nov 27, 2013 at 11:01 AM, Watrous, Daniel
<da...@hp.com>wrote:
> Hi,
>
> I've been following the book Hadoop Definitive Guide and a few other
> resources while exploring PIG. I am able to run as root, but not as a
> regular user. I am running in local mode. I've found a few links, but
> they haven't helped me solve my issue. The most relevant seems to be:
> http://answers.mapr.com/questions/5135/cannot-run-pig-in-local-mode
>
> My first experiment is just to load in a text file and dump it to the
> console. The files exist and are owned by the user running PIG. I show
> the failed attempt as a regular user followed by a successful run as
> root on the same system. Any help is appreciated.
>
> Here's what I'm trying:
> [watrous@c0003913 ~]$ pig -x local
> 2013-11-27 18:53:18,691 [main] INFO org.apache.pig.Main - Apache Pig
> version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
> 2013-11-27 18:53:18,692 [main] INFO org.apache.pig.Main - Logging
> error messages to: /home/watrous/pig_1385578398689.log
> 2013-11-27 18:53:18,747 [main] INFO org.apache.pig.impl.util.Utils -
> Default bootup file /home/watrous/.pigbootup not found
> 2013-11-27 18:53:19,025 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2013-11-27 18:53:19,025 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker
> is deprecated. Instead, use mapreduce.jobtracker.address
> 2013-11-27 18:53:19,027 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: file:///
> 2013-11-27 18:53:19,031 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.used.genericoptionsparser is deprecated. Instead, use
> mapreduce.client.genericoptionsparser.used
> 2013-11-27 18:53:19,451 [main] WARN
> org.apache.hadoop.util.NativeCodeLoader - Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2013-11-27 18:53:19,517 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> io.bytes.per.checksum is deprecated. Instead, use
> dfs.bytes-per-checksum
> 2013-11-27 18:53:19,519 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> grunt> ls /home/watrous/input/
> file:/home/watrous/input/sample.txt<r 1> 530
> grunt> cat /home/watrous/input/sample.txt
>
> 0067011990999991950051507004+68750+023550FM-12+038299999V0203301N00671220001CN9999999N9+00001+99999999999
>
> 0043011990999991950051512004+68750+023550FM-12+038299999V0203201N00671220001CN9999999N9+00221+99999999999
>
> 0043011990999991950051518004+68750+023550FM-12+038299999V0203201N00261220001CN9999999N9-00111+99999999999
>
> 0043012650999991949032412004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+01111+99999999999
>
> 0043012650999991949032418004+62300+010750FM-12+048599999V0202701N00461220001CN0500001N9+00781+99999999999
> grunt> records = load '/home/watrous/input/sample.txt';
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> job.end.retry.interval is deprecated. Instead, use
> mapreduce.job.end-notification.retry.interval
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.retiredjobs.cache.size is deprecated. Instead, use
> mapreduce.jobtracker.retiredjobs.cache.size
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.profile.reduces is deprecated. Instead, use
> mapreduce.task.profile.reduces
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.reuse.jvm.num.tasks is deprecated. Instead, use
> mapreduce.job.jvm.numtasks
> 2013-11-27 18:53:25,086 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.tracker.http.address is deprecated. Instead, use
> mapreduce.tasktracker.http.address
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.system.dir
> is deprecated. Instead, use mapreduce.jobtracker.system.dir
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.tracker.report.address is deprecated. Instead, use
> mapreduce.tasktracker.report.address
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.healthChecker.interval is deprecated. Instead, use
> mapreduce.tasktracker.healthchecker.interval
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.child.tmp
> is deprecated. Instead, use mapreduce.task.tmp.dir
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.taskmemorymanager.monitoring-interval is deprecated.
> Instead, use
> mapreduce.tasktracker.taskmemorymanager.monitoringinterval
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.shuffle.connect.timeout is deprecated. Instead, use
> mapreduce.reduce.shuffle.connect.timeout
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.speculative.execution.speculativeCap is deprecated. Instead,
> use mapreduce.job.speculative.speculativecap
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> io.sort.spill.percent is deprecated. Instead, use
> mapreduce.map.sort.spill.percent
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.shuffle.input.buffer.percent is deprecated. Instead, use
> mapreduce.reduce.shuffle.input.buffer.percent
> 2013-11-27 18:53:25,087 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.skip.map.max.skip.records is deprecated. Instead, use
> mapreduce.map.skip.maxrecords
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.profile.maps is deprecated. Instead, use
> mapreduce.task.profile.maps
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.local.dir
> is deprecated. Instead, use mapreduce.cluster.local.dir
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.merge.recordsBeforeProgress is deprecated. Instead, use
> mapreduce.task.merge.progress.records
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.http.address is deprecated. Instead, use
> mapreduce.jobtracker.http.address
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.compress.map.output is deprecated. Instead, use
> mapreduce.map.output.compress
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.userlog.retain.hours is deprecated. Instead, use
> mapreduce.job.userlog.retain.hours
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.speculative.execution.slowNodeThreshold is deprecated. Instead,
> use mapreduce.job.speculative.slownodethreshold
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.reduce.tasks.maximum is deprecated. Instead, use
> mapreduce.tasktracker.reduce.tasks.maximum
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.restart.recover is deprecated. Instead, use
> mapreduce.jobtracker.restart.recover
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.child.log.level is deprecated. Instead, use
> mapreduce.reduce.log.level
> 2013-11-27 18:53:25,088 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.inmem.merge.threshold is deprecated. Instead, use
> mapreduce.reduce.merge.inmem.threshold
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.acls.enabled is deprecated. Instead, use
> mapreduce.cluster.acls.enabled
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.dns.nameserver is deprecated. Instead, use
> mapreduce.tasktracker.dns.nameserver
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.submit.replication is deprecated. Instead, use
> mapreduce.client.submit.file.replication
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.map.child.log.level is deprecated. Instead, use
> mapreduce.map.log.level
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.map.max.attempts is deprecated. Instead, use
> mapreduce.map.maxattempts
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.shuffle.merge.percent is deprecated. Instead, use
> mapreduce.reduce.shuffle.merge.percent
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.jobhistory.lru.cache.size is deprecated. Instead,
> use mapreduce.jobtracker.jobhistory.lru.cache.size
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.queue.nameis deprecated. Instead, use
> mapreduce.job.queuename
> 2013-11-27 18:53:25,089 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.persist.jobstatus.hours is deprecated. Instead, use
> mapreduce.jobtracker.persist.jobstatus.hours
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.healthChecker.script.timeout is deprecated. Instead, use
> mapreduce.tasktracker.healthchecker.script.timeout
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.map.tasks.maximum is deprecated. Instead, use
> mapreduce.tasktracker.map.tasks.maximum
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use
> mapreduce.reduce.markreset.buffer.percent
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> jobclient.completion.poll.interval is deprecated. Instead, use
> mapreduce.client.completion.pollinterval
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.persist.jobstatus.dir is deprecated. Instead, use
> mapreduce.jobtracker.persist.jobstatus.dir
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.slowstart.completed.maps is deprecated. Instead, use
> mapreduce.job.reduce.slowstart.completedmaps
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - io.sort.mb is
> deprecated. Instead, use mapreduce.task.io.sort.mb
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - dfs.umaskmode is
> deprecated. Instead, use fs.permissions.umask-mode
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.instrumentation is deprecated. Instead, use
> mapreduce.jobtracker.instrumentation
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> topology.node.switch.mapping.impl is deprecated. Instead, use
> net.topology.node.switch.mapping.impl
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.output.compression.type is deprecated. Instead, use
> mapreduce.output.fileoutputformat.compress.type
> 2013-11-27 18:53:25,090 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.skip.attempts.to.start.skipping is deprecated. Instead, use
> mapreduce.task.skip.start.attempts
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.tracker.task-controller is deprecated. Instead, use
> mapreduce.tasktracker.taskcontroller
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.userlog.limit.kb is deprecated. Instead, use
> mapreduce.task.userlog.limit.kb
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.committer.job.setup.cleanup.needed is deprecated. Instead, use
> mapreduce.job.committer.setup.cleanup.needed
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.map.tasks
> is deprecated. Instead, use mapreduce.job.maps
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.local.dir.minspacekill is deprecated. Instead, use
> mapreduce.tasktracker.local.dir.minspacekill
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapreduce.jobtracker.split.metainfo.maxsize is deprecated. Instead,
> use mapreduce.job.split.metainfo.maxsize
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> jobclient.progress.monitor.poll.interval is deprecated. Instead, use
> mapreduce.client.progressmonitor.pollinterval
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.min.split.size is deprecated. Instead, use
> mapreduce.input.fileinputformat.split.minsize
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.output.compression.codec is deprecated. Instead, use
> mapreduce.output.fileoutputformat.compress.codec
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.profile is deprecated. Instead, use mapreduce.task.profile
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.parallel.copies is deprecated. Instead, use
> mapreduce.reduce.shuffle.parallelcopies
> 2013-11-27 18:53:25,091 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - io.sort.factor is
> deprecated. Instead, use mapreduce.task.io.sort.factor
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.timeout is deprecated. Instead, use mapreduce.task.timeout
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.heartbeats.in.second is deprecated. Instead, use
> mapreduce.jobtracker.heartbeats.in.second
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.map.output.compression.codec is deprecated. Instead, use
> mapreduce.map.output.compress.codec
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.task.cache.levels is deprecated. Instead, use
> mapreduce.jobtracker.taskcache.levels
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.dns.interface is deprecated. Instead, use
> mapreduce.tasktracker.dns.interface
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.max.tracker.failures is deprecated. Instead, use
> mapreduce.job.maxtaskfailures.per.tracker
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - dfs.df.interval is
> deprecated. Instead, use fs.df.interval
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.tasks.sleeptime-before-sigkill is deprecated.
> Instead, use mapreduce.tasktracker.tasks.sleeptimebeforesigkill
> 2013-11-27 18:53:25,092 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.max.tracker.blacklists is deprecated. Instead, use
> mapreduce.jobtracker.tasktracker.maxblacklists
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> jobclient.output.filter is deprecated. Instead, use
> mapreduce.client.output.filter
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.taskScheduler is deprecated. Instead, use
> mapreduce.jobtracker.taskscheduler
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> job.end.retry.attempts is deprecated. Instead, use
> mapreduce.job.end-notification.retry.attempts
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.speculative.execution.slowTaskThreshold is deprecated. Instead,
> use mapreduce.job.speculative.slowtaskthreshold
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.indexcache.mb is deprecated. Instead, use
> mapreduce.tasktracker.indexcache.mb
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> tasktracker.http.threads is deprecated. Instead, use
> mapreduce.tasktracker.http.threads
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.handler.count is deprecated. Instead, use
> mapreduce.jobtracker.handler.count
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> keep.failed.task.files is deprecated. Instead, use
> mapreduce.task.files.preserve.failedtasks
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.output.compress is deprecated. Instead, use
> mapreduce.output.fileoutputformat.compress
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.job.history.block.size is deprecated. Instead, use
> mapreduce.jobtracker.jobhistory.block.size
> 2013-11-27 18:53:25,093 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.skip.reduce.max.skip.groups is deprecated. Instead, use
> mapreduce.reduce.skip.maxgroups
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> topology.script.number.args is deprecated. Instead, use
> net.topology.script.number.args
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.local.dir.minspacestart is deprecated. Instead, use
> mapreduce.tasktracker.local.dir.minspacestart
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.jobtracker.maxtasks.per.job is deprecated. Instead, use
> mapreduce.jobtracker.maxtasks.perjob
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.reduce.max.attempts is deprecated. Instead, use
> mapreduce.reduce.maxattempts
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.shuffle.read.timeout is deprecated. Instead, use
> mapreduce.reduce.shuffle.read.timeout
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.instrumentation is deprecated. Instead, use
> mapreduce.tasktracker.instrumentation
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.tasktracker.expiry.interval is deprecated. Instead, use
> mapreduce.jobtracker.expire.trackers.interval
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.tracker.persist.jobstatus.active is deprecated. Instead,
> use mapreduce.jobtracker.persist.jobstatus.active
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - mapred.temp.dir is
> deprecated. Instead, use mapreduce.cluster.temp.dir
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - hadoop.native.lib
> is deprecated. Instead, use io.native.lib.available
> 2013-11-27 18:53:25,094 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapreduce.job.counters.limit is deprecated. Instead, use
> mapreduce.job.counters.max
> 2013-11-27 18:53:25,095 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation -
> mapred.job.reduce.input.buffer.percent is deprecated. Instead, use
> mapreduce.reduce.input.buffer.percent
> grunt> dump records;
> 2013-11-27 18:53:31,060 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2013-11-27 18:53:31,118 [main] INFO
> org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
> {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune,
> DuplicateForEachColumnRewrite, GroupByConstParallelSetter,
> ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter,
> MergeFilter, MergeForEach, NewPartitionFilterOptimizer,
> PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter,
> SplitFilter, StreamTypeCastInserter],
> RULES_DISABLED=[FilterLogicExpressionSimplifier]}
> 2013-11-27 18:53:31,234 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompile
> r - File concatenation threshold: 100 optimistic? false
> 2013-11-27 18:53:31,292 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> yOptimizer
> - MR plan size before optimization: 1
> 2013-11-27 18:53:31,292 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> yOptimizer
> - MR plan size after optimization: 1
> 2013-11-27 18:53:31,322 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - session.id is
> deprecated. Instead, use dfs.metrics.session-id
> 2013-11-27 18:53:31,323 [main] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics
> with processName=JobTracker, sessionId=
> 2013-11-27 18:53:31,367 [main] WARN
> org.apache.pig.backend.hadoop20.PigJobControl - falling back to
> default JobControl (not using hadoop 0.20 ?)
> java.lang.NoSuchFieldException: runnerState
> at java.lang.Class.getDeclaredField(Class.java:1899)
> at
> org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51)
> at
> org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:98)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:287)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:190)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
> at org.apache.pig.PigServer.storeEx(PigServer.java:978)
> at org.apache.pig.PigServer.store(PigServer.java:942)
> at org.apache.pig.PigServer.openIterator(PigServer.java:855)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.run(Main.java:541)
> at org.apache.pig.Main.main(Main.java:156)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> 2013-11-27 18:53:31,371 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are
> added to the job
> 2013-11-27 18:53:31,378 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> lCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to
> default 0.3
> 2013-11-27 18:53:31,449 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> lCompiler
> - Setting up single store job
> 2013-11-27 18:53:31,456 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> lCompiler
> - Map only job, skipping reducer estimation
> 2013-11-27 18:53:31,499 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 1 map-reduce job(s) waiting for submission.
> 2013-11-27 18:53:31,515 [JobControl] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM
> Metrics with processName=JobTracker, sessionId= - already initialized
> 2013-11-27 18:53:31,524 [JobControl] ERROR
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl - Error while
> trying to run jobs.
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> at
> org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> at java.lang.Thread.run(Thread.java:679)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher$1.run(MapReduceLauncher.java:270)
> 2013-11-27 18:53:31,525 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 0% complete
> 2013-11-27 18:53:31,533 [main] WARN
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig
> to stop immediately on failure.
> 2013-11-27 18:53:31,533 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - job null has failed! Stop running all dependent jobs
> 2013-11-27 18:53:31,534 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 100% complete
> 2013-11-27 18:53:31,545 [main] ERROR
> org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
> recreate exception from backend error: Unexpected System Error Occured:
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> at
> org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> at java.lang.Thread.run(Thread.java:679)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher$1.run(MapReduceLauncher.java:270)
>
> 2013-11-27 18:53:31,546 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2013-11-27 18:53:31,546 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode.
> Stats reported below may be incomplete
> 2013-11-27 18:53:31,547 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion PigVersion UserId StartedAt FinishedAt
> Features
> 2.2.0 0.12.0 watrous 2013-11-27 18:53:31 2013-11-27 18:53:31
> UNKNOWN
>
> Failed!
>
> Failed Jobs:
> JobId Alias Feature Message Outputs
> N/A records MAP_ONLY Message: Unexpected System Error Occured:
> java.lang.IncompatibleClassChangeError: Found interface
> org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
> at
> org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
> at java.lang.Thread.run(Thread.java:679)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
> file:/tmp/temp-884153250/tmp-1708053986,
>
> Input(s):
> Failed to read data from "/home/watrous/input/sample.txt"
>
> Output(s):
> Failed to produce result in "file:/tmp/temp-884153250/tmp-1708053986"
>
> Job DAG:
> null
>
>
> 2013-11-27 18:53:31,547 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - Failed!
> 2013-11-27 18:53:31,553 [main] ERROR org.apache.pig.tools.grunt.Grunt
> - ERROR 1066: Unable to open iterator for alias records Details at
> logfile: /home/watrous/pig_1385578398689.log
> grunt>
>
> The exact same process works fine it I'm root:
> [watrous@c0003913 ~]$ sudo su
> [root@c0003913 watrous]# pig -x local
> which: no hadoop in
> (/sbin:/bin:/usr/sbin:/usr/bin:/opt/mtp/bin:/home/watrous/pig-0.12.0/b
> in)
> 2013-11-27 18:58:34,166 [main] INFO org.apache.pig.Main - Apache Pig
> version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
> 2013-11-27 18:58:34,167 [main] INFO org.apache.pig.Main - Logging
> error messages to: /home/watrous/pig_1385578714163.log
> 2013-11-27 18:58:34,184 [main] INFO org.apache.pig.impl.util.Utils -
> Default bootup file /root/.pigbootup not found
> 2013-11-27 18:58:34,321 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: file:///
> grunt> records = load '/home/watrous/input/sample.txt'; dump records;
> 2013-11-27 18:59:17,019 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2013-11-27 18:59:17,082 [main] INFO
> org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
> {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune,
> DuplicateForEachColumnRewrite, GroupByConstParallelSetter,
> ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter,
> MergeFilter, MergeForEach, NewPartitionFilterOptimizer,
> PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter,
> SplitFilter, StreamTypeCastInserter],
> RULES_DISABLED=[FilterLogicExpressionSimplifier]}
> 2013-11-27 18:59:17,185 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompile
> r - File concatenation threshold: 100 optimistic? false
> 2013-11-27 18:59:17,236 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> yOptimizer
> - MR plan size before optimization: 1
> 2013-11-27 18:59:17,237 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQuer
> yOptimizer
> - MR plan size after optimization: 1
> 2013-11-27 18:59:17,278 [main] INFO
> org.apache.pig.tools.pigstats.ScriptState - Pig script settings are
> added to the job
> 2013-11-27 18:59:17,301 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> lCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to
> default 0.3
> 2013-11-27 18:59:17,338 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> lCompiler
> - Setting up single store job
> 2013-11-27 18:59:17,344 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobContro
> lCompiler
> - Map only job, skipping reducer estimation
> 2013-11-27 18:59:17,375 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 1 map-reduce job(s) waiting for submission.
> 2013-11-27 18:59:17,387 [JobControl] WARN
> org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
> library for your platform... using builtin-java classes where
> applicable
> 2013-11-27 18:59:17,392 [JobControl] WARN
> org.apache.hadoop.mapred.JobClient - No job jar file set. User
> classes may not be found. See JobConf(Class) or JobConf#setJar(String).
> 2013-11-27 18:59:17,450 [JobControl] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input
> paths to process : 1
> 2013-11-27 18:59:17,450 [JobControl] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input paths to process : 1
> 2013-11-27 18:59:17,459 [JobControl] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input paths (combined) to process : 1
> 2013-11-27 18:59:17,812 [Thread-3] INFO
> org.apache.hadoop.util.ProcessTree - setsid exited with exit code 0
> 2013-11-27 18:59:17,862 [Thread-3] INFO org.apache.hadoop.mapred.Task
> - Using ResourceCalculatorPlugin :
> org.apache.hadoop.util.LinuxResourceCalculatorPlugin@79ca209e
> 2013-11-27 18:59:17,877 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - HadoopJobId: job_local_0001
> 2013-11-27 18:59:17,877 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - Processing aliases records
> 2013-11-27 18:59:17,877 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - detailed locations: M: records[1,10] C: R:
> 2013-11-27 18:59:17,884 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 0% complete
> 2013-11-27 18:59:17,921 [Thread-3] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecord
> Reader
> - Current split being processed
> file:/home/watrous/input/sample.txt:0+530
> 2013-11-27 18:59:17,979 [Thread-3] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnl
> y$Map
> - Aliases being processed per job phase (AliasName[line,offset]): M:
> records[1,10] C: R:
> 2013-11-27 18:59:17,995 [Thread-3] INFO org.apache.hadoop.mapred.Task
> -
> Task:attempt_local_0001_m_000000_0 is done. And is in the process of
> commiting
> 2013-11-27 18:59:18,000 [Thread-3] INFO
> org.apache.hadoop.mapred.LocalJobRunner -
> 2013-11-27 18:59:18,001 [Thread-3] INFO org.apache.hadoop.mapred.Task
> - Task attempt_local_0001_m_000000_0 is allowed to commit now
> 2013-11-27 18:59:18,004 [Thread-3] INFO
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved
> output of task 'attempt_local_0001_m_000000_0' to
> file:/tmp/temp-1635315632/tmp501386857
> 2013-11-27 18:59:20,734 [Thread-3] INFO
> org.apache.hadoop.mapred.LocalJobRunner -
> 2013-11-27 18:59:20,751 [Thread-3] INFO org.apache.hadoop.mapred.Task
> - Task 'attempt_local_0001_m_000000_0' done.
> 2013-11-27 18:59:20,893 [main] WARN
> org.apache.pig.tools.pigstats.PigStatsUtil - Failed to get RunningJob
> for job job_local_0001
> 2013-11-27 18:59:20,896 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - 100% complete
> 2013-11-27 18:59:20,896 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode.
> Stats reported below may be incomplete
> 2013-11-27 18:59:20,898 [main] INFO
> org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion PigVersion UserId StartedAt FinishedAt
> Features
> 1.0.0 0.12.0 root 2013-11-27 18:59:17 2013-11-27 18:59:20
> UNKNOWN
>
> Success!
>
> Job Stats (time in seconds):
> JobId Alias Feature Outputs
> job_local_0001 records MAP_ONLY
> file:/tmp/temp-1635315632/tmp501386857,
>
> Input(s):
> Successfully read records from: "/home/watrous/input/sample.txt"
>
> Output(s):
> Successfully stored records in: "file:/tmp/temp-1635315632/tmp501386857"
>
> Job DAG:
> job_local_0001
>
>
> 2013-11-27 18:59:20,905 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduce
> Launcher
> - Success!
> 2013-11-27 18:59:20,908 [main] WARN
> org.apache.pig.data.SchemaTupleBackend - SchemaTupleBackend has
> already been initialized
> 2013-11-27 18:59:20,911 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input
> paths to process : 1
> 2013-11-27 18:59:20,911 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input paths to process : 1
>
> (0067011990999991950051507004+68750+023550FM-12+038299999V0203301N0067
> 1220001CN9999999N9+00001+99999999999)
>
> (0043011990999991950051512004+68750+023550FM-12+038299999V0203201N0067
> 1220001CN9999999N9+00221+99999999999)
>
> (0043011990999991950051518004+68750+023550FM-12+038299999V0203201N0026
> 1220001CN9999999N9-00111+99999999999)
>
> (0043012650999991949032412004+62300+010750FM-12+048599999V0202701N0046
> 1220001CN0500001N9+01111+99999999999)
>
> (0043012650999991949032418004+62300+010750FM-12+048599999V0202701N0046
> 1220001CN0500001N9+00781+99999999999)
> grunt>
>