You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Syed Zulfiqar Ali (JIRA)" <ji...@apache.org> on 2015/10/26 07:19:27 UTC
[jira] [Commented] (PIG-4616) Fix UT errors of TestPigRunner in
Spark mode
[ https://issues.apache.org/jira/browse/PIG-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973798#comment-14973798 ]
Syed Zulfiqar Ali commented on PIG-4616:
----------------------------------------
TestPigRunner depends heavily on SparkPigStats API for validating the executed PIG script (..and underlying Spark jobs). Some of these tests are failing due to unimplemented methods in SparkPigStats API (Eg: getRecordWritten(), getBytesWritten() etc).
I dug into few tests and found that though underlying Spark jobs are doing correct computation but SparkPigStats API is not returning the correct stats and metrics, hence tests are failing.
> Fix UT errors of TestPigRunner in Spark mode
> --------------------------------------------
>
> Key: PIG-4616
> URL: https://issues.apache.org/jira/browse/PIG-4616
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: Xianda Ke
> Assignee: Xianda Ke
> Fix For: spark-branch
>
>
> Following unit test failed:
> org.apache.pig.test.TestPigRunner.returnCodeTest
> org.apache.pig.test.TestPigRunner.testEmptyFileCounter
> org.apache.pig.test.TestPigRunner.testDisablePigCounters2
> org.apache.pig.test.TestPigRunner.simpleTest
> org.apache.pig.test.TestPigRunner.simpleTest2
> org.apache.pig.test.TestPigRunner.MQDepJobFailedTest
> org.apache.pig.test.TestPigRunner.scriptsInDfsTest
> org.apache.pig.test.TestPigRunner.testGetHadoopCounters
> org.apache.pig.test.TestPigRunner.simpleMultiQueryTest
> org.apache.pig.test.TestPigRunner.testDuplicateCounterName
> org.apache.pig.test.TestPigRunner.testRegisterExternalJar
> org.apache.pig.test.TestPigRunner.simpleMultiQueryTest2
> org.apache.pig.test.TestPigRunner.testDuplicateCounterName2
> org.apache.pig.test.TestPigRunner.returnCodeTest2
> org.apache.pig.test.TestPigRunner.orderByTest
> org.apache.pig.test.TestPigRunner.testDisablePigCounters
> org.apache.pig.test.TestPigRunner.testLongCounterName
> org.apache.pig.test.TestPigRunner.testEmptyFileCounter2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)