You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Ying He (JIRA)" <ji...@apache.org> on 2009/10/20 21:33:59 UTC
[jira] Created: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
explain and dump not working with two UDFs inside inner plan of foreach
-----------------------------------------------------------------------
Key: PIG-1030
URL: https://issues.apache.org/jira/browse/PIG-1030
Project: Pig
Issue Type: Bug
Reporter: Ying He
this scprit does not work
register /homes/yinghe/owl/string.jar;
a = load '/user/yinghe/a.txt' as (id, color);
b = group a all;
c = foreach b {
d = distinct a.color;
generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
}
the udfs are regular, not algebraic.
then if I call "dump c;" or "explain c", I would get this error message.
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
The error only occurs forn the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771691#action_12771691 ]
Pradeep Kamath commented on PIG-1030:
-------------------------------------
In the following code :
{code}
750 List<PhysicalOperator> preds = mPlan.getPredecessors(leaf);
751 if (preds.size() > 1) {
752 sawNonAlgebraic = true;
753 }
{code}
Shouldn't there be a return after setting sawNonAlgebraic since we shouldn't proceed to make more checks?
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Attachments: PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771385#action_12771385 ]
Hadoop QA commented on PIG-1030:
--------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12423476/PIG-1030.patch
against trunk revision 830757.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/126/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/126/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/126/console
This message is automatically generated.
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Attachments: PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1030:
------------------------------
Status: Patch Available (was: Open)
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Attachments: PIG-1030.patch, PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding reassigned PIG-1030:
---------------------------------
Assignee: Richard Ding
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Ying He (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ying He updated PIG-1030:
-------------------------
Description:
this scprit does not work
register /homes/yinghe/owl/string.jar;
a = load '/user/yinghe/a.txt' as (id, color);
b = group a all;
c = foreach b {
d = distinct a.color;
generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
}
the udfs are regular, not algebraic.
then if I call "dump c;" or "explain c", I would get this error message.
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
was:
this scprit does not work
register /homes/yinghe/owl/string.jar;
a = load '/user/yinghe/a.txt' as (id, color);
b = group a all;
c = foreach b {
d = distinct a.color;
generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
}
the udfs are regular, not algebraic.
then if I call "dump c;" or "explain c", I would get this error message.
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
The error only occurs forn the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772123#action_12772123 ]
Hadoop QA commented on PIG-1030:
--------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12423708/PIG-1030.patch
against trunk revision 831402.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/36/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/36/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/36/console
This message is automatically generated.
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Attachments: PIG-1030.patch, PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1030:
------------------------------
Status: Open (was: Patch Available)
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Attachments: PIG-1030.patch, PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-1030:
--------------------------------
Resolution: Fixed
Fix Version/s: 0.6.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
+1, patch committed, Thanks Richard!
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Fix For: 0.6.0
>
> Attachments: PIG-1030.patch, PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1030:
------------------------------
Attachment: PIG-1030.patch
Add returns as suggested.
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Attachments: PIG-1030.patch, PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1030:
------------------------------
Attachment: PIG-1030.patch
The problem is that the code assums to combine only in the case where there is only one PODistinct which is the only input to an aggregate function. This patch disables the combiner if an aggregate function in foreach statements have multiple inputs and one of them is PODistinct.
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Attachments: PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1030) explain and dump not working with two
UDFs inside inner plan of foreach
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1030:
------------------------------
Status: Patch Available (was: Open)
> explain and dump not working with two UDFs inside inner plan of foreach
> -----------------------------------------------------------------------
>
> Key: PIG-1030
> URL: https://issues.apache.org/jira/browse/PIG-1030
> Project: Pig
> Issue Type: Bug
> Reporter: Ying He
> Assignee: Richard Ding
> Attachments: PIG-1030.patch
>
>
> this scprit does not work
> register /homes/yinghe/owl/string.jar;
> a = load '/user/yinghe/a.txt' as (id, color);
> b = group a all;
> c = foreach b {
> d = distinct a.color;
> generate group, string.BagCount2(d), string.ColumnLen2(d, 0);
> }
> the udfs are regular, not algebraic.
> then if I call "dump c;" or "explain c", I would get this error message.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2019: Expected to find plan with single leaf. Found 2 leaves.
> The error only occurs for the first time, after getting this error, if I call "dump c" or "explain c" again, it would succeed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.