[jira] [Updated] (PIG-2978) TestLoadStoreFuncLifeCycle fails with hadoop-2.0.x

["Cheolsoo Park (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Cheolsoo Park updated PIG-2978:
-------------------------------

    Attachment: PIG-2978.patch

The error is thrown because the Hadoop-2.0.x Job class overrides the toString() method, and it checks whether the JobState is running or not.
{code}
  @Override
  public String toString() {
    ensureState(JobState.RUNNING);
    ...
{code}
In hadoop-1.0.x, String.valueOf(<Job object>) is printed as something like this:
{code}
org.apache.hadoop.mapreduce.Job@7ee49dcd
{code}
So I replaced String.valueOf(<Job object>) with Job.class.getName().

With this change, the test now runs with hadoop-2.0.x but fails because the no. of store func instances is 5, whereas it is 4 with hadoop-1.0.x. So I replaced the assertion as follows:
{code}
Storer.count <= (Util.isHadoop2_0() ? 5 : 4)
{code}
It would be good if we could identify why it is different between hadoop-1.0.x and hadoop-2.0.x.

                
> TestLoadStoreFuncLifeCycle fails with hadoop-2.0.x
> --------------------------------------------------
>
>                 Key: PIG-2978
>                 URL: https://issues.apache.org/jira/browse/PIG-2978
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Cheolsoo Park
>             Fix For: 0.11
>
>         Attachments: PIG-2978.patch
>
>
> To reproduce, please run:
> {code}
> ant clean test -Dtestcase=TestLoadStoreFuncLifeCycle -Dhadoopversion=23
> {code}
> This fails with the following error:
> {code}
> Error during parsing. Job in state DEFINE instead of RUNNING
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Job in state DEFINE instead of RUNNING
>     at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
>     at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
>     at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
>     at org.apache.pig.PigServer.registerQuery(PigServer.java:529)
>     at org.apache.pig.TestLoadStoreFuncLifeCycle.testLoadStoreFunc(TestLoadStoreFuncLifeCycle.java:332)
> Caused by: Failed to parse: Job in state DEFINE instead of RUNNING
>     at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:193)
>     at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> Caused by: java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
>     at org.apache.hadoop.mapreduce.Job.ensureState(Job.java:292)
>     at org.apache.hadoop.mapreduce.Job.toString(Job.java:456)
>     at java.lang.String.valueOf(String.java:2826)
>     at org.apache.pig.TestLoadStoreFuncLifeCycle.logCaller(TestLoadStoreFuncLifeCycle.java:270)
>     at org.apache.pig.TestLoadStoreFuncLifeCycle.access$000(TestLoadStoreFuncLifeCycle.java:41)
>     at org.apache.pig.TestLoadStoreFuncLifeCycle$InstrumentedStorage.logCaller(TestLoadStoreFuncLifeCycle.java:54)
>     at org.apache.pig.TestLoadStoreFuncLifeCycle$InstrumentedStorage.getSchema(TestLoadStoreFuncLifeCycle.java:115)
>     at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:174)
>     at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:88)
>     at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:839)
>     at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3236)
>     at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1315)
>     at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:799)
>     at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:517)
>     at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:392)
>     at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira