[jira] [Commented] (PIG-4552) Fix TestForEachNestedPlanLocal for Spark engine

["Mohit Sabharwal (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-4552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544941#comment-14544941 ] 

Mohit Sabharwal commented on PIG-4552:
--------------------------------------

FYI: [~kellyzly], [~praveenr019], [~xuefuz]

CROSS operation is implemented in two flavors in Pig:
1) Regular CROSS using GFCross UDF
2) Nested CROSS using POCross

PIG-4549 addresses 1)
This patch fixes unit tests related to 2).

No work is required to implement 2) in Spark even though POCross
is a physical operator, because POCross can only appear inside 
nested foreach and, hence, will be handled directly as part of 
execution of foreach nested physical plans.

This patch fixes Nested CROSS related tests in 
TestForEachNestedPlanLocal.

I also added some minor cleanup of SparkLauncher to the patch/

> Fix TestForEachNestedPlanLocal for Spark engine
> -----------------------------------------------
>
>                 Key: PIG-4552
>                 URL: https://issues.apache.org/jira/browse/PIG-4552
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>    Affects Versions: spark-branch
>            Reporter: Mohit Sabharwal
>            Assignee: Mohit Sabharwal
>             Fix For: spark-branch
>
>
> TestForEachNestedPlanLocal tests with Nested CROSS are failing due to different GROUP BY ordering between MR and Spark.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)