You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Richard Ding (JIRA)" <ji...@apache.org> on 2011/04/18 22:53:05 UTC
[jira] [Created] (PIG-1999) Macro alias masker should consider
schema context
Macro alias masker should consider schema context
--------------------------------------------------
Key: PIG-1999
URL: https://issues.apache.org/jira/browse/PIG-1999
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Richard Ding
Assignee: Richard Ding
Fix For: 0.9.0
Macro alias masker doesn't consider the current schema context. This results errors when deciding with alias to mask. Here is an example:
{code}
define toBytearray(in, intermediate) returns e {
a = load '$in' as (name:chararray, age:long, gpa: float);
b = group a by name;
c = foreach b generate a, (1,2,3);
store c into '$intermediate' using BinStorage();
d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c));
$e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
};
f = toBytearray ('data', 'output1');
{code}
Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the current schema.
The workaround is to not use alias as as names in the schema definition.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-1999) Macro alias masker should consider
schema context
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1999:
------------------------------
Attachment: PIG-1999_2.patch
Rebase the patch.
> Macro alias masker should consider schema context
> --------------------------------------------------
>
> Key: PIG-1999
> URL: https://issues.apache.org/jira/browse/PIG-1999
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1999_1.patch, PIG-1999_2.patch
>
>
> Macro alias masker doesn't consider the current schema context. This results errors when deciding with alias to mask. Here is an example:
> {code}
> define toBytearray(in, intermediate) returns e {
> a = load '$in' as (name:chararray, age:long, gpa: float);
> b = group a by name;
> c = foreach b generate a, (1,2,3);
> store c into '$intermediate' using BinStorage();
> d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c));
> $e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
> };
>
> f = toBytearray ('data', 'output1');
> {code}
> Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the current schema.
> The workaround is to not use alias as as names in the schema definition.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-1999) Macro alias masker should consider
schema context
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027735#comment-13027735 ]
Richard Ding commented on PIG-1999:
-----------------------------------
test-patch result:
{code}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{code}
> Macro alias masker should consider schema context
> --------------------------------------------------
>
> Key: PIG-1999
> URL: https://issues.apache.org/jira/browse/PIG-1999
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1999_1.patch, PIG-1999_2.patch
>
>
> Macro alias masker doesn't consider the current schema context. This results errors when deciding with alias to mask. Here is an example:
> {code}
> define toBytearray(in, intermediate) returns e {
> a = load '$in' as (name:chararray, age:long, gpa: float);
> b = group a by name;
> c = foreach b generate a, (1,2,3);
> store c into '$intermediate' using BinStorage();
> d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c));
> $e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
> };
>
> f = toBytearray ('data', 'output1');
> {code}
> Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the current schema.
> The workaround is to not use alias as as names in the schema definition.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-1999) Macro alias masker should consider
schema context
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1999:
------------------------------
Attachment: PIG-1999_1.patch
> Macro alias masker should consider schema context
> --------------------------------------------------
>
> Key: PIG-1999
> URL: https://issues.apache.org/jira/browse/PIG-1999
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1999_1.patch
>
>
> Macro alias masker doesn't consider the current schema context. This results errors when deciding with alias to mask. Here is an example:
> {code}
> define toBytearray(in, intermediate) returns e {
> a = load '$in' as (name:chararray, age:long, gpa: float);
> b = group a by name;
> c = foreach b generate a, (1,2,3);
> store c into '$intermediate' using BinStorage();
> d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c));
> $e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
> };
>
> f = toBytearray ('data', 'output1');
> {code}
> Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the current schema.
> The workaround is to not use alias as as names in the schema definition.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-1999) Macro alias masker should consider
schema context
Posted by "Xuefu Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027938#comment-13027938 ]
Xuefu Zhang commented on PIG-1999:
----------------------------------
+1 Patch PIG-1999_2.patch looks good.
> Macro alias masker should consider schema context
> --------------------------------------------------
>
> Key: PIG-1999
> URL: https://issues.apache.org/jira/browse/PIG-1999
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1999_1.patch, PIG-1999_2.patch
>
>
> Macro alias masker doesn't consider the current schema context. This results errors when deciding with alias to mask. Here is an example:
> {code}
> define toBytearray(in, intermediate) returns e {
> a = load '$in' as (name:chararray, age:long, gpa: float);
> b = group a by name;
> c = foreach b generate a, (1,2,3);
> store c into '$intermediate' using BinStorage();
> d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c));
> $e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
> };
>
> f = toBytearray ('data', 'output1');
> {code}
> Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the current schema.
> The workaround is to not use alias as as names in the schema definition.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-1999) Macro alias masker should consider
schema context
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding resolved PIG-1999.
-------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Unit tests pass. Patch committed to trunk and 0.9 branch.
> Macro alias masker should consider schema context
> --------------------------------------------------
>
> Key: PIG-1999
> URL: https://issues.apache.org/jira/browse/PIG-1999
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.9.0
>
> Attachments: PIG-1999_1.patch, PIG-1999_2.patch
>
>
> Macro alias masker doesn't consider the current schema context. This results errors when deciding with alias to mask. Here is an example:
> {code}
> define toBytearray(in, intermediate) returns e {
> a = load '$in' as (name:chararray, age:long, gpa: float);
> b = group a by name;
> c = foreach b generate a, (1,2,3);
> store c into '$intermediate' using BinStorage();
> d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c));
> $e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
> };
>
> f = toBytearray ('data', 'output1');
> {code}
> Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the current schema.
> The workaround is to not use alias as as names in the schema definition.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-1999) Macro alias masker should consider
schema context
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023457#comment-13023457 ]
Richard Ding commented on PIG-1999:
-----------------------------------
The proposed solution is, in 0.9 release, disallow user defined schema inside a macro to have names that conflict with alias in the macro. Parser will throw an exception when it detects a name conflict in macro.
This is not a perfect solution. But this kind of name-conflict is not common in Pig scripts and any conflict can be easily fixed with clear error messages.
As mentioned before, we need schema context to solve this problem. Now schema resolution happens after parsing. If needed, we'll address this problem in future release.
> Macro alias masker should consider schema context
> --------------------------------------------------
>
> Key: PIG-1999
> URL: https://issues.apache.org/jira/browse/PIG-1999
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.9.0
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.9.0
>
>
> Macro alias masker doesn't consider the current schema context. This results errors when deciding with alias to mask. Here is an example:
> {code}
> define toBytearray(in, intermediate) returns e {
> a = load '$in' as (name:chararray, age:long, gpa: float);
> b = group a by name;
> c = foreach b generate a, (1,2,3);
> store c into '$intermediate' using BinStorage();
> d = load '$intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c));
> $e = foreach d generate COUNT(b), t2.a, t2.b, t2.c;
> };
>
> f = toBytearray ('data', 'output1');
> {code}
> Now the alias masker mistakes b in COUNT(b) as an alias instead of b in the current schema.
> The workaround is to not use alias as as names in the schema definition.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira