You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by an...@nokia.com on 2011/05/22 16:14:34 UTC
please help - Caused by: java.io.IOException: Filesystem closed
Hello all
sometimes my pig job fails to run successfully (stack trace below)
In fact the last 2 days it has been the same stack trace
Googling for it gave this
http://lucene.472066.n3.nabble.com/what-does-it-mean-java-io-IOException-Filesystem-closed-td1829780.html
which talks about OS open file limit
Has anybody else encountered similar issues ?
Any advise/suggestions that you have of what could be causing this and what I should do?
2042: Internal error. Unable to translate logical plan to physical plan
ERROR 6002: Unable to obtain a temporary path.
Caused by: java.io.IOException: Filesystem closed
Issue is pressing, so any help is appreciated
thanks and regards
Anindita
> Full Exception Stacktrace:
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias pop_short_csv
> at org.apache.pig.PigServer.store(PigServer.java:577)
> at org.apache.pig.PigServer.store(PigServer.java:533)
> at com.nokia.productscores.ScoresTrendsPigJob.run(ScoresTrendsPigJob.java:316)
> at com.nokia.productscores.ScoresTrendsPigJob.runJob(ScoresTrendsPigJob.java:137)
> at com.nokia.productscores.ProductScoresImpl.compute(ProductScoresImpl.java:113)
> at com.nokia.relevancy.scheduler.quartz.ComputerProductScoresJob.execute(ComputerProductScoresJob.java:32)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:199)
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.ja
> va:549) Caused by:
> org.apache.pig.backend.executionengine.ExecException: ERROR 2042: Internal error. Unable to translate logical plan to physical plan.
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:289)
> at org.apache.pig.PigServer.compilePp(PigServer.java:917)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:832)
> at org.apache.pig.PigServer.store(PigServer.java:569)
> ... 7 more
> Caused by: org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogicalToPhysicalTranslatorException: ERROR 6002: Unable to obtain a temporary path.
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:1464)
> at org.apache.pig.impl.logicalLayer.LOSplit.visit(LOSplit.java:118)
> at org.apache.pig.impl.logicalLayer.LOSplit.visit(LOSplit.java:40)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:283)
> ... 10 more
> Caused by: java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:226)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:617)
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:453)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648)
> at org.apache.pig.backend.hadoop.datastorage.HPath.exists(HPath.java:106)
> at org.apache.pig.impl.io.FileLocalizer.getTemporaryPath(FileLocalizer.java:465)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:1448)
> ... 15 more