[jira] [Updated] (PIG-2484) Fix several e2e test failures/aborts for 23

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-2484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rohini Palaniswamy updated PIG-2484:
------------------------------------

    Attachment: PIG-2484-4-branch0.9.patch

Patch for Branch 0.9. 

- Removed ignore for Scripting_1 and Scripting_2 as PIG-2761 fixes the issue.
- Included the benchmark patch from PIG-2711 to make testing faster.
- Additional tests fixed from what is mentioned in the description. Will create a patch for these for 0.10 and trunk on a separate jira.
1)ClassResolution
   - Fully qualified UDF names were not used.
2)Native_x_Local 
   - This was failing as local.conf did not set mapredjars
3)Jython_Macro_1_Local
    - - Was failing because the input file was the data directory itself instead
of studenttab10K. In case of mapred and local, the number of files under data
directory was different so Jython_Macro_1 passed with the benchmark created during mapred, but
Jython_Macro_1_local failed with the benchmark from mapred.

Still Failing Tests not addressed in the patch:
1)Jython_CompileBindRun_3_local
   - Problem exists with 0.10 also. It launches 3 threads to launch multiple jobs. But with LocalJobRunner it does not work as all write to test/e2e/pig/testdist/build/test/mapred/local/localRunner/job_local_0001.xml and parsing it fails with org.xml.sax.SAXParseException: Content is not allowed in trailing section. 

Additional steps for checkin:
  test/e2e/pig/lib/hadoop-0.23.0-streaming.jar is not part of the patch. Copy file from branch-0.10 and svn add before committing the patch.

                
> Fix several e2e test failures/aborts for 23
> -------------------------------------------
>
>                 Key: PIG-2484
>                 URL: https://issues.apache.org/jira/browse/PIG-2484
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.9.2, 0.10.0, 0.11
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.10.0, 0.11
>
>         Attachments: PIG-2484-1.patch, PIG-2484-2.patch, PIG-2484-3.patch, PIG-2484-4-branch0.9.patch
>
>
> There are still a couple of e2e test aborts/failures for hadoop23. Most of them are due to test infrastructure, minor backward incompatibility change in 23, or recent changes in Pig. Here is a list:
> Scripting_1/Scripting_2: MAPREDUCE-3700
> Native_3: 23 test need a hadoop23-steaming.jar
> MonitoredUDF_1: Seems related to guava upgrade (PIG-2460), Pig's guava is newer than hadoop23's
> UdfException_1, UdfException_2, UdfException_3, UdfException_4: Error message change
> Checkin_2, GroupAggFunc_7, GroupAggFunc_9, GroupAggFunc_12, GroupAggFunc_13, Types_6, Scalar_1: float precision
> Limit_2: The specific output records change, test infrastructure should allow this

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira