You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Cheolsoo Park (JIRA)" <ji...@apache.org> on 2013/02/26 01:32:13 UTC
[jira] [Resolved] (PIG-3145) Parameters in core-site.xml and
mapred-site.xml are not correctly substituted
[ https://issues.apache.org/jira/browse/PIG-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park resolved PIG-3145.
--------------------------------
Resolution: Fixed
Committed to trunk again. Thank you Santhosh for the review!
> Parameters in core-site.xml and mapred-site.xml are not correctly substituted
> -----------------------------------------------------------------------------
>
> Key: PIG-3145
> URL: https://issues.apache.org/jira/browse/PIG-3145
> Project: Pig
> Issue Type: Bug
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.12
>
> Attachments: PIG-3145-2.patch, PIG-3145.patch
>
>
> To reproduce the issue, please do the following:
> # Parameterize the address of name node in core-site.xml.
> {code}
> <property>
> <name>fs.default.name</name>
> <value>hdfs://${foo}:8020</value>
> </property>
> {code}
> # Set the value of "foo" via -D option.
> {code}
> export PIG_OPTS="-Dfoo=mr1-0.cheolsoo.com"
> {code}
> # Pig fails with the following error.
> {code}
> 2013-01-28 18:54:02,786 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://${foo}:8020
> 2013-01-28 18:54:02,805 [main] ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error. null
> Details at logfile: /home/cheolsoo/pig-cdh/pig_1359428042522.log
> {code}
> Note that the parameter $\{foo\} in core-site.xml is not expanded. This is because the addresses of name node and job tracker are read directly from core-site.xml instead of reading via Configuration.get().
> {code:title=HExecutionEngine.java}
> // properties is Java Properties
> cluster = properties.getProperty(JOB_TRACKER_LOCATION);
> nameNode = properties.getProperty(FILE_SYSTEM_LOCATION);
> {code}
> Replacing these lines with Configuration.get() fixes the issue.
> {code:title=HExecutionEngine.java}
> // jc is Hadoop Configuration
> cluster = jc.get(JOB_TRACKER_LOCATION);
> nameNode = jc.get(FILE_SYSTEM_LOCATION);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira