You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org> on 2012/04/06 21:05:25 UTC
[jira] [Commented] (PIG-2635) PigServer fails to run mapreduce
against hadoop 0.23.
[ https://issues.apache.org/jira/browse/PIG-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248605#comment-13248605 ]
Daniel Dai commented on PIG-2635:
---------------------------------
[2] looks familiar, did you build pig using -Dhadoopversion=23?
> PigServer fails to run mapreduce against hadoop 0.23.
> -----------------------------------------------------
>
> Key: PIG-2635
> URL: https://issues.apache.org/jira/browse/PIG-2635
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.2
> Reporter: Brian Tieman
>
> Running Pig scripts through Grunt works without hiccup but I have a utility program that runs commands through PigServer. When running that way, I run into two problems.
> The first, which I can hack around, is that I need to pass in to PigServer the hbase configuration. When I do this with ConfigurationUtil.toProperties(hbaseConfig), the new properties get passed in-not the deprecated ones. Later on down the road, Pig looks for fs.default.name which is the deprecated property for fs.defaultfs and throws [1] The hack is to back map the fs.default.name property to the value of fs.defaultfs after calling ConfigurationUtil.toProperties();
> That brings me to the second problem which I haven't been able to hack around - When PigServer runs, it throws the exception[2]. Poking through the code, there is no longer an org.apache.hadoop.mapred.jobcontrol.JobControl.addJob() method distributed with hadoop 0.23-only an org.apache.hadoop.mapred.jobcontrol.JobControl.addJobs() method-which also appears to be deprecated.
> Again, Grunt works fine-just PigServer shows these problems. Has anyone resolved this? Or should I log a bug?
> Thanks!
> [1]
> java.lang.NullPointerException
> at java.net.URI$Parser.parse(URI.java:3003)
> at java.net.URI.<init>(URI.java:578)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.setHdfsServers(QueryParser.java:485)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.LoadClause(QueryParser.java:1618)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1366)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:1013)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:800)
> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1612)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:558)
> at com.cerner.kepler.pig.console.KeyDump.dumpKeys(KeyDump.java:227)
> [2]
> Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Lja
> va/lang/String;
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:258)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
> at org.apache.pig.PigServer.storeEx(PigServer.java:885)
> at org.apache.pig.PigServer.store(PigServer.java:827)
> at com.cerner.kepler.pig.console.KeyDump.dumpKeys(KeyDump.java:231)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira