[jira] [Updated] (PIG-4927) Support stop.on.failure in spark mode

["liyunzhang_intel (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-4927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

liyunzhang_intel updated PIG-4927:
----------------------------------
    Attachment: PIG-4927.patch

Before we skip TestGrunt#testStopOnFailure in spark mode
{code}
 @Test
    public void testStopOnFailure() throws Throwable {
        Assume.assumeTrue("Skip this test for TEZ", Util.isMapredExecType(cluster.getExecType()));
...
}
{code}

After merged with trunk code, this case is not skipped anymore. After test, this case fails due to stop.on.failure property is not supported in spark mode.

Changes in PIG-4927:
1. check pigContext.getProperties().getProperty("stop.on.failure","false") in JobGraphBuilder#sparkOperToRDD
2. once a failure is found,throw ExecException 
3. because we wrap ExecException to RuntimeException in JobGraphBuilder#visitSparkOp.  so we need unwrap in TestGrunt#testStopOnFailure.

> Support stop.on.failure in spark mode
> -------------------------------------
>
>                 Key: PIG-4927
>                 URL: https://issues.apache.org/jira/browse/PIG-4927
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>         Attachments: PIG-4927.patch
>
>
> After merge with trunk, following unit test fails because we don't support stop.on.failure property now.
> TestGrunt#testStopOnFailure



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)