You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2010/07/06 21:27:50 UTC
[jira] Commented: (PIG-1481) PigServer throws exception if it
cannot find hadoop-site.xml or core-site.xml
[ https://issues.apache.org/jira/browse/PIG-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885650#action_12885650 ]
Daniel Dai commented on PIG-1481:
---------------------------------
You will need hadoop conf file even in MiniCluster. Pig will build a conf file for MiniCluster, which is located in "${user.home}"/pigtest/conf. Put that in your classpath. If you don't have hadoop conf file and run Pig in mapreduce mode (no "-x local"), Pig is running in agnostic mode, some operation success and some fail (such as order by). So this check is necessary. This is the case [PIG-1338|https://issues.apache.org/jira/browse/PIG-1338] address.
> PigServer throws exception if it cannot find hadoop-site.xml or core-site.xml
> -----------------------------------------------------------------------------
>
> Key: PIG-1481
> URL: https://issues.apache.org/jira/browse/PIG-1481
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Sameer M
>
> Hi
> We've been using the Hadoop MiniCluster to do unit testing of our pig scripts in the following way.
> MiniCluster minicluster = MiniCluster.buildCluster(2,2);
> pigServer = new PigServer(ExecType.MAPREDUCE, minicluster.getProperties());
> This has been working fine for 0.6 and 0.7.
> However in the trunk (0.8) looks like there is change due to which an exception is thrown if hadoop-site.xml or core-site.xml is not found in the classpath.
> org.apache.pig.backend.executionengine.ExecException: ERROR 4010: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:149)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:114)
> at org.apache.pig.impl.PigContext.connect(PigContext.java:177)
> at org.apache.pig.PigServer.<init>(PigServer.java:215)
> at org.apache.pig.PigServer.<init>(PigServer.java:204)
> at org.apache.pig.PigServer.<init>(PigServer.java:200)
> The problem seems to be org.apache.pig.backend.hadoop.executionengine.HExecutionEngine: 148
> if( hadoop_site == null && core_site == null ) {
> throw new ExecException("Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath)." +
> "If you plan to use local mode, please put -x local option in command line",
> 4010);
> }
> We would like to use the mapreduce mode but with the minicluster and have a lot of unit test with that setup.
> Can this check be removed from this level ?
> Thanks
> Sameer
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.