[jira] [Reopened] (PIG-3135) HExecutionEngine should look for resources in user passed Properties

["Cheolsoo Park (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Cheolsoo Park reopened PIG-3135:
--------------------------------


[~prkommireddi], very sorry for reopening the jira, but this makes unit test fail.

Note that {{ant clean test -Dtestcase=TestHExecutionEngine}} always passes. But when you run the test as part of unit test (i.e. ant clean test), it fails.

To reproduce, please do the following:
# Run a unit test that uses mini-cluster (e.g. TestAccumulator).
{code}
ant clean test -Dtestcase=TestAccumulator
{code}
# Run TestHExecutionEngine *without ant clean*.
{code}
ant test -Dtestcase=TestHExecutionEngine
{code}
This will fail with the following errors in logs:
{code}
java.net.ConnectException: Call to localhost.localdomain/127.0.0.1:59119 failed on connection exception: java.net.ConnectException: Connection refused
{code}
In fact, the same issue was discussed in [PIG-2769|https://issues.apache.org/jira/browse/PIG-2769?focusedCommentId=13541974&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13541974] before. What I don't fully understand is why this happens even though you're doing [what Rohini suggests|https://issues.apache.org/jira/browse/PIG-2769?focusedCommentId=13547105&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13547105].

For the time being, I will back out the patch to stabilize the build. As soon as you update the patch, I will test it again. Does this make sense?
                
> HExecutionEngine should look for resources in user passed Properties
> --------------------------------------------------------------------
>
>                 Key: PIG-3135
>                 URL: https://issues.apache.org/jira/browse/PIG-3135
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.10.0
>            Reporter: Prashant Kommireddi
>            Assignee: Prashant Kommireddi
>             Fix For: 0.12
>
>         Attachments: PIG-3135_1.patch, PIG-3135.patch
>
>
> Looking at this snippet:
> {code}
>     private void init(Properties properties) throws ExecException {
>           .
>           .
>           .
>             // Check existence of hadoop-site.xml or core-site.xml
>             Configuration testConf = new Configuration();
>             ClassLoader cl = testConf.getClassLoader();
>             URL hadoop_site = cl.getResource( HADOOP_SITE );
>             URL core_site = cl.getResource( CORE_SITE );
>            
>             if( hadoop_site == null && core_site == null ) {
>                 throw new ExecException("Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath)." +
>                         " If you plan to use local mode, please put -x local option in command line",
>                         4010);
>             }
> {code}
> This assumes the resources (*-site.xml) are set on the classpath, but this will not always be the case when run with Pig's Java APIs. One could want to programatically set the resources and the code here should additionally check if they are available in there. 
> Example: When a Configuration object is created and resources are added before passing it on to Pig.
> {code}
> Configuration conf = new Configuration(false);
> conf.addResource("foo/core-site.xml");
> conf.addResource("bar/hadoop-site.xml");
> PigServer pServer = new PigServer(ExecType.MAPREDUCE, conf);
> {code}
> The above conf is not used right now to obtain resources.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira