You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2011/09/08 00:01:09 UTC
[jira] [Created] (PIG-2267) Make the name of the columns in schema
optional
Make the name of the columns in schema optional
-----------------------------------------------
Key: PIG-2267
URL: https://issues.apache.org/jira/browse/PIG-2267
Project: Pig
Issue Type: Bug
Components: parser
Affects Versions: 0.9.0
Reporter: Ashutosh Chauhan
Following two works:
{code}
a = load 'data';
a = load 'data' as (f1:int);
{code}
Following doesn't:
{code}
a = load 'data' as (int);
{code}
It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2267) Make the name of the columns in schema
optional
Posted by "Jonathan Coveney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2267:
----------------------------------
Fix Version/s: 0.11
Affects Version/s: 0.11
0.10
Status: Patch Available (was: Open)
ant test-commit works, but I need to add tests specific to this (though I will say that the tests we have definitely fail if you mess up the grammar!)
> Make the name of the columns in schema optional
> -----------------------------------------------
>
> Key: PIG-2267
> URL: https://issues.apache.org/jira/browse/PIG-2267
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9.0, 0.10, 0.11
> Reporter: Ashutosh Chauhan
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
>
> Following two works:
> {code}
> a = load 'data';
> a = load 'data' as (f1:int);
> {code}
> Following doesn't:
> {code}
> a = load 'data' as (int);
> {code}
> It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2267) Make the name of the columns in schema
optional
Posted by "Jonathan Coveney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2267:
----------------------------------
Attachment: PIG2267.patch
> Make the name of the columns in schema optional
> -----------------------------------------------
>
> Key: PIG-2267
> URL: https://issues.apache.org/jira/browse/PIG-2267
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9.0, 0.10, 0.11
> Reporter: Ashutosh Chauhan
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG2267.patch
>
>
> Following two works:
> {code}
> a = load 'data';
> a = load 'data' as (f1:int);
> {code}
> Following doesn't:
> {code}
> a = load 'data' as (int);
> {code}
> It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2267) Make the name of the columns in
schema optional
Posted by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182143#comment-13182143 ]
Daniel Dai commented on PIG-2267:
---------------------------------
+1. Pretty good change. The next step is to simply the syntax for cast as well (eg, foreach a generate (bag{tuple(int,double)})a0).
Will commit once tests pass.
> Make the name of the columns in schema optional
> -----------------------------------------------
>
> Key: PIG-2267
> URL: https://issues.apache.org/jira/browse/PIG-2267
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9.0, 0.10, 0.11
> Reporter: Ashutosh Chauhan
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG2267.patch, PIG2267_1.patch
>
>
> Following two works:
> {code}
> a = load 'data';
> a = load 'data' as (f1:int);
> {code}
> Following doesn't:
> {code}
> a = load 'data' as (int);
> {code}
> It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2267) Make the name of the columns in schema
optional
Posted by "Jonathan Coveney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2267:
----------------------------------
Attachment: PIG2267_1.patch
A slightly more recent version of this with some tests.
> Make the name of the columns in schema optional
> -----------------------------------------------
>
> Key: PIG-2267
> URL: https://issues.apache.org/jira/browse/PIG-2267
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9.0, 0.10, 0.11
> Reporter: Ashutosh Chauhan
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG2267.patch, PIG2267_1.patch
>
>
> Following two works:
> {code}
> a = load 'data';
> a = load 'data' as (f1:int);
> {code}
> Following doesn't:
> {code}
> a = load 'data' as (int);
> {code}
> It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2267) Make the name of the columns in
schema optional
Posted by "Jonathan Coveney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181783#comment-13181783 ]
Jonathan Coveney commented on PIG-2267:
---------------------------------------
Bump :)
> Make the name of the columns in schema optional
> -----------------------------------------------
>
> Key: PIG-2267
> URL: https://issues.apache.org/jira/browse/PIG-2267
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9.0, 0.10, 0.11
> Reporter: Ashutosh Chauhan
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG2267.patch, PIG2267_1.patch
>
>
> Following two works:
> {code}
> a = load 'data';
> a = load 'data' as (f1:int);
> {code}
> Following doesn't:
> {code}
> a = load 'data' as (int);
> {code}
> It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (PIG-2267) Make the name of the columns in schema
optional
Posted by "Jonathan Coveney (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney reassigned PIG-2267:
-------------------------------------
Assignee: Jonathan Coveney
> Make the name of the columns in schema optional
> -----------------------------------------------
>
> Key: PIG-2267
> URL: https://issues.apache.org/jira/browse/PIG-2267
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9.0
> Reporter: Ashutosh Chauhan
> Assignee: Jonathan Coveney
>
> Following two works:
> {code}
> a = load 'data';
> a = load 'data' as (f1:int);
> {code}
> Following doesn't:
> {code}
> a = load 'data' as (int);
> {code}
> It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2267) Make the name of the columns in schema
optional
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2267:
----------------------------
Resolution: Fixed
Release Note:
Name part of the schema is optional in as clause in load and foreach statement. For example:
a = load 'data' as (int);
a = load 'data' as ({(int,int)});
y = foreach x generate x0 as int;
y = foreach x generate x0 as {(int, int)};
Pig will generate dummy name if the name part is missing.
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Unit test pass.
test-patch:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 6 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] -1 javac. The applied patch generated 898 javac compiler warnings (more than the trunk's current 894 warnings).
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] -1 release audit. The applied patch generated 510 release audit warnings (more than the trunk's current 502 warnings).
javac warning is caused by antlr generated code. I added Apache header to all new files, so ignore release audit warning.
Patch committed to trunk. Thanks Jonathan!
> Make the name of the columns in schema optional
> -----------------------------------------------
>
> Key: PIG-2267
> URL: https://issues.apache.org/jira/browse/PIG-2267
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9.0, 0.10, 0.11
> Reporter: Ashutosh Chauhan
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG2267.patch, PIG2267_1.patch
>
>
> Following two works:
> {code}
> a = load 'data';
> a = load 'data' as (f1:int);
> {code}
> Following doesn't:
> {code}
> a = load 'data' as (int);
> {code}
> It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2267) Make the name of the columns in
schema optional
Posted by "Jonathan Coveney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171470#comment-13171470 ]
Jonathan Coveney commented on PIG-2267:
---------------------------------------
Find attached a patch that works with trunk. Note, I still need to write some new tests (I'm still deciding what sort of test is meaningful), but it passes ant test-commit.
Bags get a schema bag_0 and up, tuples tuple_0 and up, maps map_0 and up, and the rest get val_ and up (I originally did int_, long_, and so on, but julien brought up a good point: it's important to be able to change an int to a long, for example, without breaking the script).
Here are some examples of output:
{code}
a = load 'data' as (tuple(int));
{code}
yields
{code}
a: {tuple_0: (val_0: int)}
{code}
{code}
a = load 'data' as (int,int,int,long,bytearray);
{code}
yields
{code}
a: {val_0: int,val_1: int,val_2: int,val_3: long,val_4: bytearray}
{code}
{code}
a = load 'data' as ({(int,int,int)},{(int,int,int)},{(int,int,int)});
{code}
yields
{code}
a: {bag_0: {(val_0: int,val_1: int,val_2: int)},bag_1: {(val_0: int,val_1: int,val_2: int)},bag_2: {(val_0: int,val_1: int,val_2: int)}}
{code}
I welcome any suggestions, especially on good tests...
> Make the name of the columns in schema optional
> -----------------------------------------------
>
> Key: PIG-2267
> URL: https://issues.apache.org/jira/browse/PIG-2267
> Project: Pig
> Issue Type: Bug
> Components: parser
> Affects Versions: 0.9.0, 0.10, 0.11
> Reporter: Ashutosh Chauhan
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
>
> Following two works:
> {code}
> a = load 'data';
> a = load 'data' as (f1:int);
> {code}
> Following doesn't:
> {code}
> a = load 'data' as (int);
> {code}
> It will be nice to make names of column optional.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira