[jira] [Commented] (PIG-4613) Fix unit test failures about TestAssert

["kexianda (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611338#comment-14611338 ] 

kexianda commented on PIG-4613:
-------------------------------

In this nagative UT case, the Assert should fail, and an exception is thrown by org.apache.pig.builtin.Assert
In spark mode, spark scheduler catch it and re-throw out. 
then, in following function: 
{code:title=PigServer.java|borderStyle=solid}
openIterator() {
	
	if (job.getStatus() == JOB_STATUS.COMPLETED) {
        return job.getResults();
    } else if (job.getStatus() == JOB_STATUS.FAILED  //in MR mode, job.getException()==null !
            && job.getException() != null) { 
        // throw the backend exception in the failed case
        Exception e = job.getException();
        int errCode = 1066;
        String msg = "Unable to open iterator for alias " + id
                + ". Backend error : " + e.getMessage();
        throw new FrontendException(msg, errCode, PigException.INPUT, e); // Pig throw a wrapped exeception here in Spark mode
    } else {
        throw new IOException("Job terminated with anomalous status " // Pig throw an exception here in MR mode
                + job.getStatus().toString());
    }
}
{code}
We can see that the exception thrown from spark engine can be catched by Pig.
>From my understanding, spark engine's behavior is reasonalbe.  

But why job.getException()== null in MR engine?
org.apache.pig.builtin.Assert will throw an exception as expected. But MR's jobrunner LocalJobRunner(local mode) will not notify the MR JobController of that exception.
That's why job.getException() == null.

> Fix unit test failures about TestAssert
> ---------------------------------------
>
>                 Key: PIG-4613
>                 URL: https://issues.apache.org/jira/browse/PIG-4613
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: kexianda
>            Assignee: kexianda
>             Fix For: spark-branch
>
>         Attachments: PIG-4613.patch
>
>
> UT failed at following cases:
> org.apache.pig.test.TestAssert.testNegativeWithoutFetch
> org.apache.pig.test.TestAssert.testNegative



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)