Re: Job Jar file does not exist

[Rohini Palaniswamy <ro...@gmail.com>]

Do you have any cron job that cleans up /tmp directory?

On Fri, Nov 23, 2012 at 7:32 PM, Jieru Shi <cr...@gmail.com> wrote:

> HI Jagat Singh
> I have the permission to write file there.
> My scrip consists of Loops,which means several job will be created.
> It is weird such error happens irregularly, sometimes first job will fail,
> sometimes certain
> job will fail after several successful jobs.
>
>
>
> 2012/11/23 Jagat Singh <ja...@gmail.com>
>
> > First check i would do is the permission check of this temp folder.
> >
> >
> > On Sat, Nov 24, 2012 at 2:19 PM, Jieru Shi <cr...@gmail.com> wrote:
> >
> > > Hi
> > > I'm using embeded Pig to implement graph algorithm.
> > > It is fine when I worked in local mode, but when I worked on hadoop
> > > cluster,
> > > there always popped up some error message like: (Please see the last
> few
> > > lines)
> > >
> > > 2012-11-23 22:00:00,651 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler - creating jar
> file
> > > Job4116346741117365374.jar
> > > 2012-11-23 22:00:09,418 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler - jar file
> > > Job4116346741117365374.jar created
> > > 2012-11-23 22:00:09,423 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler - Setting up
> multi
> > > store job
> > > 2012-11-23 22:00:09,431 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler -
> > > BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=296
> > > 2012-11-23 22:00:09,431 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**JobControlCompiler - Neither
> PARALLEL
> > > nor default parallelism is set for this job. Setting number of reducers
> > to
> > > 1
> > > 2012-11-23 22:00:09,442 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**MapReduceLauncher - 1 map-reduce
> > job(s)
> > > waiting for submission.
> > > 2012-11-23 22:00:09,949 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**MapReduceLauncher - job null has
> > failed!
> > > Stop running all dependent jobs
> > > 2012-11-23 22:00:09,949 [main] INFO org.apache.pig.backend.hadoop.**
> > > executionengine.**mapReduceLayer.**MapReduceLauncher - 100% complete
> > > 2012-11-23 22:00:09,992 [main] ERROR
> > > org.apache.pig.tools.pigstats.**SimplePigStats
> > > - ERROR 6015: During execution, encountered a Hadoop error.
> > > 2012-11-23 22:00:09,993 [main] ERROR
> > > org.apache.pig.tools.pigstats.**PigStatsUtil
> > > - 1 map reduce job(s) failed!
> > > 2012-11-23 22:00:09,994 [main] INFO
> > > org.apache.pig.tools.pigstats.**SimplePigStats
> > > - Script Statistics:
> > >
> > > HadoopVersion    PigVersion    UserId    StartedAt    FinishedAt
> Features
> > > 0.20.1    0.10.0    jierus    2012-11-23 21:52:38    2012-11-23
> 22:00:09
> > >  HASH_JOIN,GROUP_BY,DISTINCT,**FILTER,UNION
> > >
> > > Some jobs have failed! Stop running all dependent jobs
> > > Failed Jobs:
> > > JobId    Alias    Feature    Message    Outputs
> > > N/A    vec_comp,vec_comp_final,vec_**comp_tmp HASH_JOIN,MULTI_QUERY
> > >  Message: java.io.FileNotFoundException: File
> > > /tmp/Job4116346741117365374.**jar
> > > does not exist.
> > >     at org.apache.hadoop.fs.**RawLocalFileSystem.**getFileStatus(**
> > > RawLocalFileSystem.java:361)
> > >     at org.apache.hadoop.fs.**FilterFileSystem.**getFileStatus(**
> > > FilterFileSystem.java:245)
> > >     at org.apache.hadoop.fs.FileUtil.**copy(FileUtil.java:192)
> > >     at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(**
> > > FileSystem.java:1184)
> > >     at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(**
> > > FileSystem.java:1160)
> > >     at org.apache.hadoop.fs.**FileSystem.copyFromLocalFile(**
> > > FileSystem.java:1132)
> > >
> > > Does anybody know which part of my code or operation is wrong?
> > >
> >
>