You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by lulynn_2008 <lu...@163.com> on 2011/09/06 04:16:36 UTC
will open jira for pig-0.8.1 test cases, please help to check some
problems
Hello,
Based on previous discussion with Thejas Nair about pig-0.8.1 unit test. I think jira should be opened for the following unit test(my research indicate the cause is HashMap does not guarantee ordering):
org.apache.pig.test.TestDataModel
org.apache.pig.test.TestLogToPhyCompiler
org.apache.pig.test.TestMRCompiler
org.apache.pig.test.TestMergeJoinOuter
org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor
org.apache.pig.test.TestPruneColumn
org.apache.pig.test.TestUnionOnSchema
org.apache.pig.test.TestPushDownForeachFlatten
org.apache.pig.test.TestTypeCheckingValidator
Actually, there are more three unit test failed, and I can not find the cause, please help to check the problems:(with sun java version "1.6.0_19" and Apache Ant(TM) version 1.8.2):
[junit] Running org.apache.pig.test.TestJobSubmission
[junit] Tests run: 6, Failures: 1, Errors: 5, Time elapsed: 0.213 sec
Testcase: testJobControlCompilerErr took 0.01 sec
FAILED
org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
junit.framework.AssertionFailedError: org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:215)
at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
at org.apache.pig.test.TestJobSubmission.testJobControlCompilerErr(TestJobSubmission.java:450)
Testcase: testDefaultParallel took 0 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.TestJobSubmission.testDefaultParallel(TestJobSubmission.java:480)
Testcase: testDefaultParallelInSort took 0 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSort(TestJobSubmission.java:506)
Testcase: testDefaultParallelInSkewJoin took 0 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSkewJoin(TestJobSubmission.java:538)
Testcase: testReducerNumEstimation took 0 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:213)
at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
at org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:573)
Testcase: testReducerNumEstimationForOrderBy took 0 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.TestJobSubmission.testReducerNumEstimationForOrderBy(TestJobSubmission.java:638)
[junit] Running org.apache.pig.test.TestPigContext
[junit] Tests run: 5, Failures: 0, Errors: 4, Time elapsed: 25.258 sec
Testcase: testSetProperties_way_num01 took 6.941 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.Util.deleteFile(Util.java:355)
at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
at org.apache.pig.test.TestPigContext.testSetProperties_way_num01(TestPigContext.java:79)
Testcase: testSetProperties_way_num02 took 6.489 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.Util.deleteFile(Util.java:355)
at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
at org.apache.pig.test.TestPigContext.testSetProperties_way_num02(TestPigContext.java:90)
Testcase: testSetProperties_way_num03 took 6.445 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.Util.deleteFile(Util.java:355)
at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
at org.apache.pig.test.TestPigContext.testSetProperties_way_num03(TestPigContext.java:102)
Testcase: testHadoopExceptionCreation took 0.001 sec
Testcase: testImportList took 1.251 sec
Caused an ERROR
null
java.lang.NullPointerException
at org.apache.pig.test.TestPigContext.testImportList(TestPigContext.java:166)
[junit] Running org.apache.pig.test.TestSecondarySort
[junit] Tests run: 23, Failures: 0, Errors: 1, Time elapsed: 266.962 sec
Testcase: testNestedSortEndToEnd3 took 63.569 sec
Caused an ERROR
Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
at org.apache.pig.PigServer.openIterator(PigServer.java:753)
at org.apache.pig.test.TestSecondarySort.testNestedSortEndToEnd3(TestSecondarySort.java:550)
Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:357)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:237)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
Re:Re: Re: will open jira for pig-0.8.1 test cases, please help to
check some problems
Posted by lulynn_2008 <lu...@163.com>.
Hello Daniel Thejas,
I opened a jira for pig-0.8.1 unit test:https://issues.apache.org/jira/browse/PIG-2281
I put all my analysis information into description part. Not find good way to fix it. Any suggestion?
At 2011-09-07 11:51:16,"Daniel Dai" <da...@hortonworks.com> wrote:
>do:
>svn co http://svn.apache.org/repos/asf/pig/trunk
>
>and then run test:
>ant test
>
>Daniel
>
>2011/9/6 lulynn_2008 <lu...@163.com>:
>> Hello Daniel,
>> what do you mean by "
>> checkout from trunk and try
>> "?
>>
>>
>>
>>
>>
>> At 2011-09-07 01:30:16,"Daniel Dai" <da...@hortonworks.com> wrote:
>>>Some test failures are caused by ant 1.8 (See PIG-2172). Also we use
>>>ArrayList instead of HashMap in Pig 0.9. Can you checkout from trunk
>>>and try?
>>>
>>>Daniel
>>>
>>>2011/9/5 lulynn_2008 <lu...@163.com>:
>>>> Hello,
>>>> Based on previous discussion with Thejas Nair about pig-0.8.1 unit test. I think jira should be opened for the following unit test(my research indicate the cause is HashMap does not guarantee ordering):
>>>> org.apache.pig.test.TestDataModel
>>>> org.apache.pig.test.TestLogToPhyCompiler
>>>> org.apache.pig.test.TestMRCompiler
>>>> org.apache.pig.test.TestMergeJoinOuter
>>>> org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor
>>>> org.apache.pig.test.TestPruneColumn
>>>> org.apache.pig.test.TestUnionOnSchema
>>>> org.apache.pig.test.TestPushDownForeachFlatten
>>>> org.apache.pig.test.TestTypeCheckingValidator
>>>> Actually, there are more three unit test failed, and I can not find the cause, please help to check the problems:(with sun java version "1.6.0_19" and Apache Ant(TM) version 1.8.2):
>>>>
>>>>
>>>> [junit] Running org.apache.pig.test.TestJobSubmission
>>>> [junit] Tests run: 6, Failures: 1, Errors: 5, Time elapsed: 0.213 sec
>>>>
>>>> Testcase: testJobControlCompilerErr took 0.01 sec
>>>>
>>>> FAILED
>>>>
>>>> org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
>>>>
>>>> junit.framework.AssertionFailedError: org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
>>>>
>>>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:215)
>>>>
>>>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
>>>>
>>>> at org.apache.pig.test.TestJobSubmission.testJobControlCompilerErr(TestJobSubmission.java:450)
>>>>
>>>>
>>>>
>>>> Testcase: testDefaultParallel took 0 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.TestJobSubmission.testDefaultParallel(TestJobSubmission.java:480)
>>>>
>>>>
>>>>
>>>> Testcase: testDefaultParallelInSort took 0 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSort(TestJobSubmission.java:506)
>>>>
>>>>
>>>>
>>>> Testcase: testDefaultParallelInSkewJoin took 0 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSkewJoin(TestJobSubmission.java:538)
>>>>
>>>>
>>>>
>>>> Testcase: testReducerNumEstimation took 0 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:213)
>>>>
>>>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
>>>>
>>>> at org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:573)
>>>>
>>>>
>>>>
>>>> Testcase: testReducerNumEstimationForOrderBy took 0 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.TestJobSubmission.testReducerNumEstimationForOrderBy(TestJobSubmission.java:638)
>>>>
>>>> [junit] Running org.apache.pig.test.TestPigContext
>>>> [junit] Tests run: 5, Failures: 0, Errors: 4, Time elapsed: 25.258 sec
>>>>
>>>> Testcase: testSetProperties_way_num01 took 6.941 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>>>
>>>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>>>
>>>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num01(TestPigContext.java:79)
>>>>
>>>>
>>>>
>>>> Testcase: testSetProperties_way_num02 took 6.489 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>>>
>>>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>>>
>>>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num02(TestPigContext.java:90)
>>>>
>>>>
>>>>
>>>> Testcase: testSetProperties_way_num03 took 6.445 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>>>
>>>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>>>
>>>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num03(TestPigContext.java:102)
>>>>
>>>>
>>>>
>>>> Testcase: testHadoopExceptionCreation took 0.001 sec
>>>>
>>>> Testcase: testImportList took 1.251 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> null
>>>>
>>>> java.lang.NullPointerException
>>>>
>>>> at org.apache.pig.test.TestPigContext.testImportList(TestPigContext.java:166)
>>>>
>>>> [junit] Running org.apache.pig.test.TestSecondarySort
>>>>
>>>> [junit] Tests run: 23, Failures: 0, Errors: 1, Time elapsed: 266.962 sec
>>>>
>>>> Testcase: testNestedSortEndToEnd3 took 63.569 sec
>>>>
>>>> Caused an ERROR
>>>>
>>>> Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>>>
>>>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>>>
>>>> at org.apache.pig.PigServer.openIterator(PigServer.java:753)
>>>>
>>>> at org.apache.pig.test.TestSecondarySort.testNestedSortEndToEnd3(TestSecondarySort.java:550)
>>>>
>>>> Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>>>
>>>> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
>>>>
>>>> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:357)
>>>>
>>>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:237)
>>>>
>>>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
>>>>
>>>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
>>>>
>>>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>>>
>>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>>>>
>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>>>>
>>>>
>>>>
>>
Re: Re: will open jira for pig-0.8.1 test cases, please help to check
some problems
Posted by Daniel Dai <da...@hortonworks.com>.
do:
svn co http://svn.apache.org/repos/asf/pig/trunk
and then run test:
ant test
Daniel
2011/9/6 lulynn_2008 <lu...@163.com>:
> Hello Daniel,
> what do you mean by "
> checkout from trunk and try
> "?
>
>
>
>
>
> At 2011-09-07 01:30:16,"Daniel Dai" <da...@hortonworks.com> wrote:
>>Some test failures are caused by ant 1.8 (See PIG-2172). Also we use
>>ArrayList instead of HashMap in Pig 0.9. Can you checkout from trunk
>>and try?
>>
>>Daniel
>>
>>2011/9/5 lulynn_2008 <lu...@163.com>:
>>> Hello,
>>> Based on previous discussion with Thejas Nair about pig-0.8.1 unit test. I think jira should be opened for the following unit test(my research indicate the cause is HashMap does not guarantee ordering):
>>> org.apache.pig.test.TestDataModel
>>> org.apache.pig.test.TestLogToPhyCompiler
>>> org.apache.pig.test.TestMRCompiler
>>> org.apache.pig.test.TestMergeJoinOuter
>>> org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor
>>> org.apache.pig.test.TestPruneColumn
>>> org.apache.pig.test.TestUnionOnSchema
>>> org.apache.pig.test.TestPushDownForeachFlatten
>>> org.apache.pig.test.TestTypeCheckingValidator
>>> Actually, there are more three unit test failed, and I can not find the cause, please help to check the problems:(with sun java version "1.6.0_19" and Apache Ant(TM) version 1.8.2):
>>>
>>>
>>> [junit] Running org.apache.pig.test.TestJobSubmission
>>> [junit] Tests run: 6, Failures: 1, Errors: 5, Time elapsed: 0.213 sec
>>>
>>> Testcase: testJobControlCompilerErr took 0.01 sec
>>>
>>> FAILED
>>>
>>> org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
>>>
>>> junit.framework.AssertionFailedError: org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
>>>
>>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:215)
>>>
>>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
>>>
>>> at org.apache.pig.test.TestJobSubmission.testJobControlCompilerErr(TestJobSubmission.java:450)
>>>
>>>
>>>
>>> Testcase: testDefaultParallel took 0 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.TestJobSubmission.testDefaultParallel(TestJobSubmission.java:480)
>>>
>>>
>>>
>>> Testcase: testDefaultParallelInSort took 0 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSort(TestJobSubmission.java:506)
>>>
>>>
>>>
>>> Testcase: testDefaultParallelInSkewJoin took 0 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSkewJoin(TestJobSubmission.java:538)
>>>
>>>
>>>
>>> Testcase: testReducerNumEstimation took 0 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:213)
>>>
>>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
>>>
>>> at org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:573)
>>>
>>>
>>>
>>> Testcase: testReducerNumEstimationForOrderBy took 0 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.TestJobSubmission.testReducerNumEstimationForOrderBy(TestJobSubmission.java:638)
>>>
>>> [junit] Running org.apache.pig.test.TestPigContext
>>> [junit] Tests run: 5, Failures: 0, Errors: 4, Time elapsed: 25.258 sec
>>>
>>> Testcase: testSetProperties_way_num01 took 6.941 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>>
>>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>>
>>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num01(TestPigContext.java:79)
>>>
>>>
>>>
>>> Testcase: testSetProperties_way_num02 took 6.489 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>>
>>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>>
>>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num02(TestPigContext.java:90)
>>>
>>>
>>>
>>> Testcase: testSetProperties_way_num03 took 6.445 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>>
>>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>>
>>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num03(TestPigContext.java:102)
>>>
>>>
>>>
>>> Testcase: testHadoopExceptionCreation took 0.001 sec
>>>
>>> Testcase: testImportList took 1.251 sec
>>>
>>> Caused an ERROR
>>>
>>> null
>>>
>>> java.lang.NullPointerException
>>>
>>> at org.apache.pig.test.TestPigContext.testImportList(TestPigContext.java:166)
>>>
>>> [junit] Running org.apache.pig.test.TestSecondarySort
>>>
>>> [junit] Tests run: 23, Failures: 0, Errors: 1, Time elapsed: 266.962 sec
>>>
>>> Testcase: testNestedSortEndToEnd3 took 63.569 sec
>>>
>>> Caused an ERROR
>>>
>>> Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>>
>>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>>
>>> at org.apache.pig.PigServer.openIterator(PigServer.java:753)
>>>
>>> at org.apache.pig.test.TestSecondarySort.testNestedSortEndToEnd3(TestSecondarySort.java:550)
>>>
>>> Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>>
>>> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
>>>
>>> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:357)
>>>
>>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:237)
>>>
>>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
>>>
>>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
>>>
>>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>>
>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>>>
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>>>
>>>
>>>
>
Re:Re: will open jira for pig-0.8.1 test cases, please help to
check some problems
Posted by lulynn_2008 <lu...@163.com>.
Hello Daniel,
what do you mean by "
checkout from trunk and try
"?
At 2011-09-07 01:30:16,"Daniel Dai" <da...@hortonworks.com> wrote:
>Some test failures are caused by ant 1.8 (See PIG-2172). Also we use
>ArrayList instead of HashMap in Pig 0.9. Can you checkout from trunk
>and try?
>
>Daniel
>
>2011/9/5 lulynn_2008 <lu...@163.com>:
>> Hello,
>> Based on previous discussion with Thejas Nair about pig-0.8.1 unit test. I think jira should be opened for the following unit test(my research indicate the cause is HashMap does not guarantee ordering):
>> org.apache.pig.test.TestDataModel
>> org.apache.pig.test.TestLogToPhyCompiler
>> org.apache.pig.test.TestMRCompiler
>> org.apache.pig.test.TestMergeJoinOuter
>> org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor
>> org.apache.pig.test.TestPruneColumn
>> org.apache.pig.test.TestUnionOnSchema
>> org.apache.pig.test.TestPushDownForeachFlatten
>> org.apache.pig.test.TestTypeCheckingValidator
>> Actually, there are more three unit test failed, and I can not find the cause, please help to check the problems:(with sun java version "1.6.0_19" and Apache Ant(TM) version 1.8.2):
>>
>>
>> [junit] Running org.apache.pig.test.TestJobSubmission
>> [junit] Tests run: 6, Failures: 1, Errors: 5, Time elapsed: 0.213 sec
>>
>> Testcase: testJobControlCompilerErr took 0.01 sec
>>
>> FAILED
>>
>> org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
>>
>> junit.framework.AssertionFailedError: org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
>>
>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:215)
>>
>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
>>
>> at org.apache.pig.test.TestJobSubmission.testJobControlCompilerErr(TestJobSubmission.java:450)
>>
>>
>>
>> Testcase: testDefaultParallel took 0 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.TestJobSubmission.testDefaultParallel(TestJobSubmission.java:480)
>>
>>
>>
>> Testcase: testDefaultParallelInSort took 0 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSort(TestJobSubmission.java:506)
>>
>>
>>
>> Testcase: testDefaultParallelInSkewJoin took 0 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSkewJoin(TestJobSubmission.java:538)
>>
>>
>>
>> Testcase: testReducerNumEstimation took 0 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:213)
>>
>> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
>>
>> at org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:573)
>>
>>
>>
>> Testcase: testReducerNumEstimationForOrderBy took 0 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.TestJobSubmission.testReducerNumEstimationForOrderBy(TestJobSubmission.java:638)
>>
>> [junit] Running org.apache.pig.test.TestPigContext
>> [junit] Tests run: 5, Failures: 0, Errors: 4, Time elapsed: 25.258 sec
>>
>> Testcase: testSetProperties_way_num01 took 6.941 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>
>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>
>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num01(TestPigContext.java:79)
>>
>>
>>
>> Testcase: testSetProperties_way_num02 took 6.489 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>
>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>
>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num02(TestPigContext.java:90)
>>
>>
>>
>> Testcase: testSetProperties_way_num03 took 6.445 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>>
>> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>>
>> at org.apache.pig.test.TestPigContext.testSetProperties_way_num03(TestPigContext.java:102)
>>
>>
>>
>> Testcase: testHadoopExceptionCreation took 0.001 sec
>>
>> Testcase: testImportList took 1.251 sec
>>
>> Caused an ERROR
>>
>> null
>>
>> java.lang.NullPointerException
>>
>> at org.apache.pig.test.TestPigContext.testImportList(TestPigContext.java:166)
>>
>> [junit] Running org.apache.pig.test.TestSecondarySort
>>
>> [junit] Tests run: 23, Failures: 0, Errors: 1, Time elapsed: 266.962 sec
>>
>> Testcase: testNestedSortEndToEnd3 took 63.569 sec
>>
>> Caused an ERROR
>>
>> Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>
>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>
>> at org.apache.pig.PigServer.openIterator(PigServer.java:753)
>>
>> at org.apache.pig.test.TestSecondarySort.testNestedSortEndToEnd3(TestSecondarySort.java:550)
>>
>> Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>>
>> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
>>
>> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:357)
>>
>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:237)
>>
>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
>>
>> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
>>
>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>>
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>>
>>
>>
Re: will open jira for pig-0.8.1 test cases, please help to check
some problems
Posted by Daniel Dai <da...@hortonworks.com>.
Some test failures are caused by ant 1.8 (See PIG-2172). Also we use
ArrayList instead of HashMap in Pig 0.9. Can you checkout from trunk
and try?
Daniel
2011/9/5 lulynn_2008 <lu...@163.com>:
> Hello,
> Based on previous discussion with Thejas Nair about pig-0.8.1 unit test. I think jira should be opened for the following unit test(my research indicate the cause is HashMap does not guarantee ordering):
> org.apache.pig.test.TestDataModel
> org.apache.pig.test.TestLogToPhyCompiler
> org.apache.pig.test.TestMRCompiler
> org.apache.pig.test.TestMergeJoinOuter
> org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor
> org.apache.pig.test.TestPruneColumn
> org.apache.pig.test.TestUnionOnSchema
> org.apache.pig.test.TestPushDownForeachFlatten
> org.apache.pig.test.TestTypeCheckingValidator
> Actually, there are more three unit test failed, and I can not find the cause, please help to check the problems:(with sun java version "1.6.0_19" and Apache Ant(TM) version 1.8.2):
>
>
> [junit] Running org.apache.pig.test.TestJobSubmission
> [junit] Tests run: 6, Failures: 1, Errors: 5, Time elapsed: 0.213 sec
>
> Testcase: testJobControlCompilerErr took 0.01 sec
>
> FAILED
>
> org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
>
> junit.framework.AssertionFailedError: org.apache.pig.backend.executionengine.ExecException: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).If you plan to use local mode, please put -x local option in command line -- a = load 'input';
>
> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:215)
>
> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
>
> at org.apache.pig.test.TestJobSubmission.testJobControlCompilerErr(TestJobSubmission.java:450)
>
>
>
> Testcase: testDefaultParallel took 0 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.TestJobSubmission.testDefaultParallel(TestJobSubmission.java:480)
>
>
>
> Testcase: testDefaultParallelInSort took 0 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSort(TestJobSubmission.java:506)
>
>
>
> Testcase: testDefaultParallelInSkewJoin took 0 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.TestJobSubmission.testDefaultParallelInSkewJoin(TestJobSubmission.java:538)
>
>
>
> Testcase: testReducerNumEstimation took 0 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:213)
>
> at org.apache.pig.test.utils.LogicalPlanTester.buildPlan(LogicalPlanTester.java:90)
>
> at org.apache.pig.test.TestJobSubmission.testReducerNumEstimation(TestJobSubmission.java:573)
>
>
>
> Testcase: testReducerNumEstimationForOrderBy took 0 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.TestJobSubmission.testReducerNumEstimationForOrderBy(TestJobSubmission.java:638)
>
> [junit] Running org.apache.pig.test.TestPigContext
> [junit] Tests run: 5, Failures: 0, Errors: 4, Time elapsed: 25.258 sec
>
> Testcase: testSetProperties_way_num01 took 6.941 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>
> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>
> at org.apache.pig.test.TestPigContext.testSetProperties_way_num01(TestPigContext.java:79)
>
>
>
> Testcase: testSetProperties_way_num02 took 6.489 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>
> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>
> at org.apache.pig.test.TestPigContext.testSetProperties_way_num02(TestPigContext.java:90)
>
>
>
> Testcase: testSetProperties_way_num03 took 6.445 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.Util.deleteFile(Util.java:355)
>
> at org.apache.pig.test.TestPigContext.registerAndStore(TestPigContext.java:247)
>
> at org.apache.pig.test.TestPigContext.testSetProperties_way_num03(TestPigContext.java:102)
>
>
>
> Testcase: testHadoopExceptionCreation took 0.001 sec
>
> Testcase: testImportList took 1.251 sec
>
> Caused an ERROR
>
> null
>
> java.lang.NullPointerException
>
> at org.apache.pig.test.TestPigContext.testImportList(TestPigContext.java:166)
>
> [junit] Running org.apache.pig.test.TestSecondarySort
>
> [junit] Tests run: 23, Failures: 0, Errors: 1, Time elapsed: 266.962 sec
>
> Testcase: testNestedSortEndToEnd3 took 63.569 sec
>
> Caused an ERROR
>
> Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for aliasE. Backend error : org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>
> at org.apache.pig.PigServer.openIterator(PigServer.java:753)
>
> at org.apache.pig.test.TestSecondarySort.testNestedSortEndToEnd3(TestSecondarySort.java:550)
>
> Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
>
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
>
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:357)
>
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:237)
>
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
>
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
>
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>
>
>