You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Nandor Kollar (JIRA)" <ji...@apache.org> on 2017/11/29 15:55:01 UTC
[jira] [Commented] (PIG-5318) Unit test failures on Pig on Spark
with Spark 2.2
[ https://issues.apache.org/jira/browse/PIG-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270990#comment-16270990 ]
Nandor Kollar commented on PIG-5318:
------------------------------------
[~szita], [~kellyzly] could you please have review?
> Unit test failures on Pig on Spark with Spark 2.2
> -------------------------------------------------
>
> Key: PIG-5318
> URL: https://issues.apache.org/jira/browse/PIG-5318
> Project: Pig
> Issue Type: Bug
> Components: spark
> Reporter: Nandor Kollar
> Assignee: Nandor Kollar
> Attachments: PIG-5318_1.patch
>
>
> There are sever failing cases when executing the unit tests with Spark 2.2:
> {code}
> org.apache.pig.test.TestAssert#testNegativeWithoutFetch
> org.apache.pig.test.TestAssert#testNegative
> org.apache.pig.test.TestEvalPipeline2#testNonStandardDataWithoutFetch
> org.apache.pig.test.TestScalarAliases#testScalarErrMultipleRowsInInput
> org.apache.pig.test.TestStore#testCleanupOnFailureMultiStore
> org.apache.pig.test.TestStoreInstances#testBackendStoreCommunication
> org.apache.pig.test.TestStoreLocal#testCleanupOnFailureMultiStore
> {code}
> All of these are related to fixes/changes in Spark.
> TestAssert, TestScalarAliases and TestEvalPipeline2 failures could be fixed by asserting on the message of the exception's root cause, looks like on Spark 2.2 the exception is wrapped into an additional layer.
> TestStore and TestStoreLocal failure are also a test related problems: looks like SPARK-7953 is fixed in Spark 2.2
> The root cause of TestStoreInstances is yet to be found out.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)