You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Raman Yakkala <ry...@gmail.com> on 2010/08/27 18:55:01 UTC
PIG Jobs are constantly failing
Hi All,
My PIG jobs are failing since yesterday which was completed successfully in
the past. I would appreciate any pointers on the possible root cause. Here
is the console log from the job and the dump of my environment values.
Thanks,
Raman
==================
grunt> STORE trec INTO '/apps/sq/ryakkala/trec_fp_us_clicks_desc1' USING
PigStorage();
2010-08-27 07:03:16,240 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned for
odl: $1, $2, $4, $5, $6, $8
2010-08-27 07:03:16,240 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
for odl
2010-08-27 07:03:16,242 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned for
oil: $2, $3, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,
$19, $20, $21, $22, $23, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35,
$36, $37, $38, $39, $40, $41, $42, $43, $44, $45, $46, $47, $48, $49, $51,
$53, $55
2010-08-27 07:03:16,242 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
for oil
2010-08-27 07:03:16,243 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned for
qif: $0, $2
2010-08-27 07:03:16,243 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
for qif
2010-08-27 07:03:16,402 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
Store(
hdfs://srwaishdc1nn0001/apps/sq/ryakkala/trec_fp_us_clicks_desc1:PigStorage)
- 1-2213 Operator Key: 1-2213)
2010-08-27 07:03:16,442 [main] WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 4 time(s).
2010-08-27 07:03:16,446 [main] WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Encountered Warning MULTI_LEAF_MAP 2 time(s).
2010-08-27 07:03:16,461 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 5
2010-08-27 07:03:16,461 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 5
2010-08-27 07:03:16,507 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2010-08-27 07:03:17,861 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2010-08-27 07:03:17,916 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2010-08-27 07:03:19,150 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2010-08-27 07:03:19,191 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2010-08-27 07:03:20,395 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2010-08-27 07:03:20,421 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 3 map-reduce job(s) waiting for submission.
2010-08-27 07:03:20,424 [Thread-16] WARN org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2010-08-27 07:03:20,923 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2010-08-27 07:03:25,950 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8621/job.jar retrying...
2010-08-27 07:03:26,353 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8621/job.jar retrying...
2010-08-27 07:03:27,667 [Thread-16] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 7850
2010-08-27 07:03:54,399 [Thread-16] WARN org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2010-08-27 07:03:59,359 [Thread-16] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1300
2010-08-27 07:04:15,397 [Thread-16] WARN org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2010-08-27 07:04:20,525 [Thread-16] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 1
2010-08-27 07:04:20,574 [Thread-16] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 200
2010-08-27 07:04:26,184 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
2010-08-27 07:04:26,586 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
2010-08-27 07:04:26,988 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
2010-08-27 07:04:27,390 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
2010-08-27 07:04:27,792 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
2010-08-27 07:04:28,194 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
2010-08-27 07:04:28,596 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
2010-08-27 07:04:28,998 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
2010-08-27 07:04:34,438 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:34,840 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:35,242 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:35,644 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:36,046 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:36,448 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:36,850 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:37,252 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:37,654 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:38,056 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:38,458 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:38,860 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:39,262 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:39,664 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:40,066 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:40,468 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:40,870 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:41,272 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
2010-08-27 07:04:48,911 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_201007221306_8621
2010-08-27 07:04:48,911 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- More information at:
http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8621
2010-08-27 07:04:48,911 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_201007221306_8622
2010-08-27 07:04:48,911 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- More information at:
http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8622
2010-08-27 07:04:48,911 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_201007221306_8623
2010-08-27 07:04:48,911 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- More information at:
http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8623
2010-08-27 07:04:48,914 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1% complete
2010-08-27 07:05:03,709 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 4% complete
2010-08-27 07:05:08,257 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 6% complete
2010-08-27 07:05:08,760 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 8% complete
2010-08-27 07:05:10,777 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 10% complete
2010-08-27 07:05:11,282 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 12% complete
2010-08-27 07:05:13,304 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 15% complete
2010-08-27 07:05:18,342 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 17% complete
2010-08-27 07:05:23,429 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 18% complete
2010-08-27 07:05:28,472 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 20% complete
2010-08-27 07:05:46,630 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 28% complete
2010-08-27 07:06:04,019 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 31% complete
2010-08-27 07:07:34,303 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 41% complete
2010-08-27 07:13:44,914 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 58% complete
2010-08-27 07:20:03,257 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 59% complete
2010-08-27 07:20:28,389 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2010-08-27 07:20:29,578 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2010-08-27 07:20:29,581 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
2010-08-27 07:20:29,587 [Thread-44] WARN org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2010-08-27 07:20:29,954 [Thread-44] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 2000
2010-08-27 07:20:29,955 [Thread-44] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 2000
2010-08-27 07:20:31,040 [Thread-44] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 2000
2010-08-27 07:20:31,041 [Thread-44] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 2000
2010-08-27 07:20:40,285 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_201007221306_8626
2010-08-27 07:20:40,285 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- More information at:
http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8626
2010-08-27 07:22:15,269 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2010-08-27 07:22:16,468 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2010-08-27 07:22:16,473 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
2010-08-27 07:22:16,478 [Thread-54] WARN org.apache.hadoop.mapred.JobClient
- Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2010-08-27 07:22:16,832 [Thread-54] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 2000
2010-08-27 07:22:16,833 [Thread-54] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 2000
2010-08-27 07:22:17,721 [Thread-54] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 2000
2010-08-27 07:22:17,721 [Thread-54] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 2000
2010-08-27 07:22:29,084 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_201007221306_8627
2010-08-27 07:22:29,084 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- More information at:
http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8627
2010-08-27 07:32:29,974 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 92% complete
2010-08-27 07:35:35,239 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2010-08-27 07:35:35,240 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map reduce job(s) failed!
2010-08-27 07:35:39,699 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed to produce result in:
"hdfs://srwaishdc1nn0001/apps/sq/ryakkala/trec_fp_us_clicks_desc1"
2010-08-27 07:35:41,622 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Some jobs have failed! Stop running all dependent jobs
grunt>
grunt>
grunt>
grunt>
grunt> quit
[ryakkala@srwaishdc1gn0001 ~]$ env
HOSTNAME=srwaishdc1gn0001
SHELL=/bin/bash
TERM=vt100
HISTSIZE=1000
HADOOP_HOME=/usr/lib/hadoop
SSH_CLIENT=10.254.24.100 2212 22
KDE_NO_IPV6=1
PIGDIR=/export/home/ryakkala/hadoop/pig-0.7.0
PIG_OPTS=-Dmapred.child.java.opts=-Xmx2048m
SSH_TTY=/dev/pts/24
bis=/inbound/sq/bis
USER=ryakkala
LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
HADOOPSITEPATH=/etc/hadoop/conf/core-site.xml
OOZIE_URL=http://localhost:8080/oozie
SSH_AUTH_SOCK=/tmp/ssh-IzHeiw5668/agent.5668
KDEDIR=/usr
PATH=/export/home/ryakkala/hadoop/pig-0.7.0/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/export/home/ryakkala/bin
MAIL=/var/spool/mail/ryakkala
PWD=/export/home/ryakkala
INPUTRC=/etc/inputrc
JAVA_HOME=/usr/java/jdk1.6.0_16/
KDE_IS_PRELINKED=1
LANG=en_US.UTF-8
PIG_CLASSPATH=/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.3.jar:/etc/hadoop/conf
HADOOP_CONF_DIR=/usr/lib/hadoop/conf
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
HOME=/export/home/ryakkala
SHLVL=2
LOGNAME=ryakkala
SSH_CONNECTION=10.254.24.100 2212 10.110.217.53 22
PIG_HEAPSIZE=30000
LESSOPEN=|/usr/bin/lesspipe.sh %s
G_BROKEN_FILENAMES=1
_=/bin/env
Re: PIG Jobs are constantly failing
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Do you have space left on your cluster?
This is odd:
2010-08-27 07:03:25,950 [Thread-16] INFO org.apache.hadoop.hdfs.
DFSClient -
Could not complete file
/tmp/hadoop-hadoop/mapred/system/job_201007221306_8621/job.jar retrying...
(it can't even finish uploading the jar)
On Fri, Aug 27, 2010 at 10:21 AM, Jeff Zhang <zj...@gmail.com> wrote:
> I think you should look into the log of failed task
>
>
>
> On Fri, Aug 27, 2010 at 9:55 AM, Raman Yakkala <ry...@gmail.com> wrote:
> > Hi All,
> >
> > My PIG jobs are failing since yesterday which was completed successfully
> in
> > the past. I would appreciate any pointers on the possible root cause.
> Here
> > is the console log from the job and the dump of my environment values.
> >
> > Thanks,
> > Raman
> >
> > ==================
> > grunt> STORE trec INTO '/apps/sq/ryakkala/trec_fp_us_clicks_desc1' USING
> > PigStorage();
> > 2010-08-27 07:03:16,240 [main] INFO
> > org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned
> for
> > odl: $1, $2, $4, $5, $6, $8
> > 2010-08-27 07:03:16,240 [main] INFO
> > org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys
> pruned
> > for odl
> > 2010-08-27 07:03:16,242 [main] INFO
> > org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned
> for
> > oil: $2, $3, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,
> > $19, $20, $21, $22, $23, $26, $27, $28, $29, $30, $31, $32, $33, $34,
> $35,
> > $36, $37, $38, $39, $40, $41, $42, $43, $44, $45, $46, $47, $48, $49,
> $51,
> > $53, $55
> > 2010-08-27 07:03:16,242 [main] INFO
> > org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys
> pruned
> > for oil
> > 2010-08-27 07:03:16,243 [main] INFO
> > org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned
> for
> > qif: $0, $2
> > 2010-08-27 07:03:16,243 [main] INFO
> > org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys
> pruned
> > for qif
> > 2010-08-27 07:03:16,402 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
> > Store(
> >
> hdfs://srwaishdc1nn0001/apps/sq/ryakkala/trec_fp_us_clicks_desc1:PigStorage)
> > - 1-2213 Operator Key: 1-2213)
> > 2010-08-27 07:03:16,442 [main] WARN
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 4 time(s).
> > 2010-08-27 07:03:16,446 [main] WARN
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - Encountered Warning MULTI_LEAF_MAP 2 time(s).
> > 2010-08-27 07:03:16,461 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> > - MR plan size before optimization: 5
> > 2010-08-27 07:03:16,461 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> > - MR plan size after optimization: 5
> > 2010-08-27 07:03:16,507 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - mapred.job.reduce.markreset.buffer.percent is not set, set to default
> 0.3
> > 2010-08-27 07:03:17,861 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - Setting up single store job
> > 2010-08-27 07:03:17,916 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - mapred.job.reduce.markreset.buffer.percent is not set, set to default
> 0.3
> > 2010-08-27 07:03:19,150 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - Setting up single store job
> > 2010-08-27 07:03:19,191 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - mapred.job.reduce.markreset.buffer.percent is not set, set to default
> 0.3
> > 2010-08-27 07:03:20,395 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - Setting up single store job
> > 2010-08-27 07:03:20,421 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 3 map-reduce job(s) waiting for submission.
> > 2010-08-27 07:03:20,424 [Thread-16] WARN
> org.apache.hadoop.mapred.JobClient
> > - Use GenericOptionsParser for parsing the arguments. Applications should
> > implement Tool for the same.
> > 2010-08-27 07:03:20,923 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 0% complete
> > 2010-08-27 07:03:25,950 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8621/job.jar
> retrying...
> > 2010-08-27 07:03:26,353 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8621/job.jar
> retrying...
> > 2010-08-27 07:03:27,667 [Thread-16] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input
> > paths to process : 7850
> > 2010-08-27 07:03:54,399 [Thread-16] WARN
> org.apache.hadoop.mapred.JobClient
> > - Use GenericOptionsParser for parsing the arguments. Applications should
> > implement Tool for the same.
> > 2010-08-27 07:03:59,359 [Thread-16] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input
> > paths to process : 1300
> > 2010-08-27 07:04:15,397 [Thread-16] WARN
> org.apache.hadoop.mapred.JobClient
> > - Use GenericOptionsParser for parsing the arguments. Applications should
> > implement Tool for the same.
> > 2010-08-27 07:04:20,525 [Thread-16] INFO
> > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
> to
> > process : 1
> > 2010-08-27 07:04:20,574 [Thread-16] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input
> > paths to process : 200
> > 2010-08-27 07:04:26,184 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split
> retrying...
> > 2010-08-27 07:04:26,586 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split
> retrying...
> > 2010-08-27 07:04:26,988 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split
> retrying...
> > 2010-08-27 07:04:27,390 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split
> retrying...
> > 2010-08-27 07:04:27,792 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split
> retrying...
> > 2010-08-27 07:04:28,194 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split
> retrying...
> > 2010-08-27 07:04:28,596 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split
> retrying...
> > 2010-08-27 07:04:28,998 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split
> retrying...
> > 2010-08-27 07:04:34,438 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:34,840 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:35,242 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:35,644 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:36,046 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:36,448 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:36,850 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:37,252 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:37,654 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:38,056 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:38,458 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:38,860 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:39,262 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:39,664 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:40,066 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:40,468 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:40,870 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:41,272 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient
> -
> > Could not complete file
> > /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml
> retrying...
> > 2010-08-27 07:04:48,911 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - HadoopJobId: job_201007221306_8621
> > 2010-08-27 07:04:48,911 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - More information at:
> > http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8621
> > 2010-08-27 07:04:48,911 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - HadoopJobId: job_201007221306_8622
> > 2010-08-27 07:04:48,911 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - More information at:
> > http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8622
> > 2010-08-27 07:04:48,911 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - HadoopJobId: job_201007221306_8623
> > 2010-08-27 07:04:48,911 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - More information at:
> > http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8623
> > 2010-08-27 07:04:48,914 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 1% complete
> > 2010-08-27 07:05:03,709 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 4% complete
> > 2010-08-27 07:05:08,257 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 6% complete
> > 2010-08-27 07:05:08,760 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 8% complete
> > 2010-08-27 07:05:10,777 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 10% complete
> > 2010-08-27 07:05:11,282 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 12% complete
> > 2010-08-27 07:05:13,304 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 15% complete
> > 2010-08-27 07:05:18,342 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 17% complete
> > 2010-08-27 07:05:23,429 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 18% complete
> > 2010-08-27 07:05:28,472 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 20% complete
> > 2010-08-27 07:05:46,630 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 28% complete
> > 2010-08-27 07:06:04,019 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 31% complete
> > 2010-08-27 07:07:34,303 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 41% complete
> > 2010-08-27 07:13:44,914 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 58% complete
> > 2010-08-27 07:20:03,257 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 59% complete
> > 2010-08-27 07:20:28,389 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - mapred.job.reduce.markreset.buffer.percent is not set, set to default
> 0.3
> > 2010-08-27 07:20:29,578 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - Setting up single store job
> > 2010-08-27 07:20:29,581 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 1 map-reduce job(s) waiting for submission.
> > 2010-08-27 07:20:29,587 [Thread-44] WARN
> org.apache.hadoop.mapred.JobClient
> > - Use GenericOptionsParser for parsing the arguments. Applications should
> > implement Tool for the same.
> > 2010-08-27 07:20:29,954 [Thread-44] INFO
> > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
> to
> > process : 2000
> > 2010-08-27 07:20:29,955 [Thread-44] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input
> > paths to process : 2000
> > 2010-08-27 07:20:31,040 [Thread-44] INFO
> > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
> to
> > process : 2000
> > 2010-08-27 07:20:31,041 [Thread-44] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input
> > paths to process : 2000
> > 2010-08-27 07:20:40,285 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - HadoopJobId: job_201007221306_8626
> > 2010-08-27 07:20:40,285 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - More information at:
> > http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8626
> > 2010-08-27 07:22:15,269 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - mapred.job.reduce.markreset.buffer.percent is not set, set to default
> 0.3
> > 2010-08-27 07:22:16,468 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> > - Setting up single store job
> > 2010-08-27 07:22:16,473 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 1 map-reduce job(s) waiting for submission.
> > 2010-08-27 07:22:16,478 [Thread-54] WARN
> org.apache.hadoop.mapred.JobClient
> > - Use GenericOptionsParser for parsing the arguments. Applications should
> > implement Tool for the same.
> > 2010-08-27 07:22:16,832 [Thread-54] INFO
> > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
> to
> > process : 2000
> > 2010-08-27 07:22:16,833 [Thread-54] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input
> > paths to process : 2000
> > 2010-08-27 07:22:17,721 [Thread-54] INFO
> > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths
> to
> > process : 2000
> > 2010-08-27 07:22:17,721 [Thread-54] INFO
> > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
> input
> > paths to process : 2000
> > 2010-08-27 07:22:29,084 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - HadoopJobId: job_201007221306_8627
> > 2010-08-27 07:22:29,084 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - More information at:
> > http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8627
> > 2010-08-27 07:32:29,974 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 92% complete
> > 2010-08-27 07:35:35,239 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 100% complete
> > 2010-08-27 07:35:35,240 [main] ERROR
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - 1 map reduce job(s) failed!
> > 2010-08-27 07:35:39,699 [main] ERROR
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - Failed to produce result in:
> > "hdfs://srwaishdc1nn0001/apps/sq/ryakkala/trec_fp_us_clicks_desc1"
> > 2010-08-27 07:35:41,622 [main] INFO
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> > - Some jobs have failed! Stop running all dependent jobs
> > grunt>
> > grunt>
> > grunt>
> > grunt>
> > grunt> quit
> > [ryakkala@srwaishdc1gn0001 ~]$ env
> > HOSTNAME=srwaishdc1gn0001
> > SHELL=/bin/bash
> > TERM=vt100
> > HISTSIZE=1000
> > HADOOP_HOME=/usr/lib/hadoop
> > SSH_CLIENT=10.254.24.100 2212 22
> > KDE_NO_IPV6=1
> > PIGDIR=/export/home/ryakkala/hadoop/pig-0.7.0
> > PIG_OPTS=-Dmapred.child.java.opts=-Xmx2048m
> > SSH_TTY=/dev/pts/24
> > bis=/inbound/sq/bis
> > USER=ryakkala
> >
> LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
> > HADOOPSITEPATH=/etc/hadoop/conf/core-site.xml
> > OOZIE_URL=http://localhost:8080/oozie
> > SSH_AUTH_SOCK=/tmp/ssh-IzHeiw5668/agent.5668
> > KDEDIR=/usr
> >
> PATH=/export/home/ryakkala/hadoop/pig-0.7.0/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/export/home/ryakkala/bin
> > MAIL=/var/spool/mail/ryakkala
> > PWD=/export/home/ryakkala
> > INPUTRC=/etc/inputrc
> > JAVA_HOME=/usr/java/jdk1.6.0_16/
> > KDE_IS_PRELINKED=1
> > LANG=en_US.UTF-8
> >
> PIG_CLASSPATH=/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.3.jar:/etc/hadoop/conf
> > HADOOP_CONF_DIR=/usr/lib/hadoop/conf
> > SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
> > HOME=/export/home/ryakkala
> > SHLVL=2
> > LOGNAME=ryakkala
> > SSH_CONNECTION=10.254.24.100 2212 10.110.217.53 22
> > PIG_HEAPSIZE=30000
> > LESSOPEN=|/usr/bin/lesspipe.sh %s
> > G_BROKEN_FILENAMES=1
> > _=/bin/env
> >
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: PIG Jobs are constantly failing
Posted by Jeff Zhang <zj...@gmail.com>.
I think you should look into the log of failed task
On Fri, Aug 27, 2010 at 9:55 AM, Raman Yakkala <ry...@gmail.com> wrote:
> Hi All,
>
> My PIG jobs are failing since yesterday which was completed successfully in
> the past. I would appreciate any pointers on the possible root cause. Here
> is the console log from the job and the dump of my environment values.
>
> Thanks,
> Raman
>
> ==================
> grunt> STORE trec INTO '/apps/sq/ryakkala/trec_fp_us_clicks_desc1' USING
> PigStorage();
> 2010-08-27 07:03:16,240 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned for
> odl: $1, $2, $4, $5, $6, $8
> 2010-08-27 07:03:16,240 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
> for odl
> 2010-08-27 07:03:16,242 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned for
> oil: $2, $3, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,
> $19, $20, $21, $22, $23, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35,
> $36, $37, $38, $39, $40, $41, $42, $43, $44, $45, $46, $47, $48, $49, $51,
> $53, $55
> 2010-08-27 07:03:16,242 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
> for oil
> 2010-08-27 07:03:16,243 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - Columns pruned for
> qif: $0, $2
> 2010-08-27 07:03:16,243 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
> for qif
> 2010-08-27 07:03:16,402 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
> Store(
> hdfs://srwaishdc1nn0001/apps/sq/ryakkala/trec_fp_us_clicks_desc1:PigStorage)
> - 1-2213 Operator Key: 1-2213)
> 2010-08-27 07:03:16,442 [main] WARN
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 4 time(s).
> 2010-08-27 07:03:16,446 [main] WARN
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Encountered Warning MULTI_LEAF_MAP 2 time(s).
> 2010-08-27 07:03:16,461 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 5
> 2010-08-27 07:03:16,461 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 5
> 2010-08-27 07:03:16,507 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2010-08-27 07:03:17,861 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2010-08-27 07:03:17,916 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2010-08-27 07:03:19,150 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2010-08-27 07:03:19,191 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2010-08-27 07:03:20,395 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2010-08-27 07:03:20,421 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 3 map-reduce job(s) waiting for submission.
> 2010-08-27 07:03:20,424 [Thread-16] WARN org.apache.hadoop.mapred.JobClient
> - Use GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> 2010-08-27 07:03:20,923 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2010-08-27 07:03:25,950 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8621/job.jar retrying...
> 2010-08-27 07:03:26,353 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8621/job.jar retrying...
> 2010-08-27 07:03:27,667 [Thread-16] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 7850
> 2010-08-27 07:03:54,399 [Thread-16] WARN org.apache.hadoop.mapred.JobClient
> - Use GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> 2010-08-27 07:03:59,359 [Thread-16] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1300
> 2010-08-27 07:04:15,397 [Thread-16] WARN org.apache.hadoop.mapred.JobClient
> - Use GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> 2010-08-27 07:04:20,525 [Thread-16] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 1
> 2010-08-27 07:04:20,574 [Thread-16] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 200
> 2010-08-27 07:04:26,184 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
> 2010-08-27 07:04:26,586 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
> 2010-08-27 07:04:26,988 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
> 2010-08-27 07:04:27,390 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
> 2010-08-27 07:04:27,792 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
> 2010-08-27 07:04:28,194 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
> 2010-08-27 07:04:28,596 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
> 2010-08-27 07:04:28,998 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.split retrying...
> 2010-08-27 07:04:34,438 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:34,840 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:35,242 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:35,644 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:36,046 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:36,448 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:36,850 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:37,252 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:37,654 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:38,056 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:38,458 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:38,860 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:39,262 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:39,664 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:40,066 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:40,468 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:40,870 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:41,272 [Thread-16] INFO org.apache.hadoop.hdfs.DFSClient -
> Could not complete file
> /tmp/hadoop-hadoop/mapred/system/job_201007221306_8623/job.xml retrying...
> 2010-08-27 07:04:48,911 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_201007221306_8621
> 2010-08-27 07:04:48,911 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - More information at:
> http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8621
> 2010-08-27 07:04:48,911 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_201007221306_8622
> 2010-08-27 07:04:48,911 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - More information at:
> http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8622
> 2010-08-27 07:04:48,911 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_201007221306_8623
> 2010-08-27 07:04:48,911 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - More information at:
> http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8623
> 2010-08-27 07:04:48,914 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1% complete
> 2010-08-27 07:05:03,709 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 4% complete
> 2010-08-27 07:05:08,257 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 6% complete
> 2010-08-27 07:05:08,760 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 8% complete
> 2010-08-27 07:05:10,777 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 10% complete
> 2010-08-27 07:05:11,282 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 12% complete
> 2010-08-27 07:05:13,304 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 15% complete
> 2010-08-27 07:05:18,342 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 17% complete
> 2010-08-27 07:05:23,429 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 18% complete
> 2010-08-27 07:05:28,472 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 20% complete
> 2010-08-27 07:05:46,630 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 28% complete
> 2010-08-27 07:06:04,019 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 31% complete
> 2010-08-27 07:07:34,303 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 41% complete
> 2010-08-27 07:13:44,914 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 58% complete
> 2010-08-27 07:20:03,257 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 59% complete
> 2010-08-27 07:20:28,389 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2010-08-27 07:20:29,578 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2010-08-27 07:20:29,581 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2010-08-27 07:20:29,587 [Thread-44] WARN org.apache.hadoop.mapred.JobClient
> - Use GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> 2010-08-27 07:20:29,954 [Thread-44] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 2000
> 2010-08-27 07:20:29,955 [Thread-44] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 2000
> 2010-08-27 07:20:31,040 [Thread-44] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 2000
> 2010-08-27 07:20:31,041 [Thread-44] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 2000
> 2010-08-27 07:20:40,285 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_201007221306_8626
> 2010-08-27 07:20:40,285 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - More information at:
> http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8626
> 2010-08-27 07:22:15,269 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2010-08-27 07:22:16,468 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2010-08-27 07:22:16,473 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2010-08-27 07:22:16,478 [Thread-54] WARN org.apache.hadoop.mapred.JobClient
> - Use GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> 2010-08-27 07:22:16,832 [Thread-54] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 2000
> 2010-08-27 07:22:16,833 [Thread-54] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 2000
> 2010-08-27 07:22:17,721 [Thread-54] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 2000
> 2010-08-27 07:22:17,721 [Thread-54] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 2000
> 2010-08-27 07:22:29,084 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_201007221306_8627
> 2010-08-27 07:22:29,084 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - More information at:
> http://srwaishdc1jn0001:50030/jobdetails.jsp?jobid=job_201007221306_8627
> 2010-08-27 07:32:29,974 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 92% complete
> 2010-08-27 07:35:35,239 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2010-08-27 07:35:35,240 [main] ERROR
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map reduce job(s) failed!
> 2010-08-27 07:35:39,699 [main] ERROR
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed to produce result in:
> "hdfs://srwaishdc1nn0001/apps/sq/ryakkala/trec_fp_us_clicks_desc1"
> 2010-08-27 07:35:41,622 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Some jobs have failed! Stop running all dependent jobs
> grunt>
> grunt>
> grunt>
> grunt>
> grunt> quit
> [ryakkala@srwaishdc1gn0001 ~]$ env
> HOSTNAME=srwaishdc1gn0001
> SHELL=/bin/bash
> TERM=vt100
> HISTSIZE=1000
> HADOOP_HOME=/usr/lib/hadoop
> SSH_CLIENT=10.254.24.100 2212 22
> KDE_NO_IPV6=1
> PIGDIR=/export/home/ryakkala/hadoop/pig-0.7.0
> PIG_OPTS=-Dmapred.child.java.opts=-Xmx2048m
> SSH_TTY=/dev/pts/24
> bis=/inbound/sq/bis
> USER=ryakkala
> LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
> HADOOPSITEPATH=/etc/hadoop/conf/core-site.xml
> OOZIE_URL=http://localhost:8080/oozie
> SSH_AUTH_SOCK=/tmp/ssh-IzHeiw5668/agent.5668
> KDEDIR=/usr
> PATH=/export/home/ryakkala/hadoop/pig-0.7.0/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/export/home/ryakkala/bin
> MAIL=/var/spool/mail/ryakkala
> PWD=/export/home/ryakkala
> INPUTRC=/etc/inputrc
> JAVA_HOME=/usr/java/jdk1.6.0_16/
> KDE_IS_PRELINKED=1
> LANG=en_US.UTF-8
> PIG_CLASSPATH=/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.3.jar:/etc/hadoop/conf
> HADOOP_CONF_DIR=/usr/lib/hadoop/conf
> SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
> HOME=/export/home/ryakkala
> SHLVL=2
> LOGNAME=ryakkala
> SSH_CONNECTION=10.254.24.100 2212 10.110.217.53 22
> PIG_HEAPSIZE=30000
> LESSOPEN=|/usr/bin/lesspipe.sh %s
> G_BROKEN_FILENAMES=1
> _=/bin/env
>
--
Best Regards
Jeff Zhang