You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Jonathan Coveney (JIRA)" <ji...@apache.org> on 2012/08/15 03:30:37 UTC
[jira] [Created] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Jonathan Coveney created PIG-2877:
-------------------------------------
Summary: Make SchemaTuple work in foreach (and thus, in loads)
Key: PIG-2877
URL: https://issues.apache.org/jira/browse/PIG-2877
Project: Pig
Issue Type: Improvement
Reporter: Jonathan Coveney
Assignee: Jonathan Coveney
Fix For: 0.11
This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465245#comment-13465245 ]
Dmitriy V. Ryaboy commented on PIG-2877:
----------------------------------------
how badly will it mess you up if we remove mandatory foreaches after loads?
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2877:
----------------------------------
Attachment: PIG-2877-3.patch
Julien,
I took into account your comments. Let me know what you think.
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch, PIG-2877-3.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2877:
----------------------------------
Attachment: PIG-2877-2.patch
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465258#comment-13465258 ]
Jonathan Coveney commented on PIG-2877:
---------------------------------------
What's the new pattern? When will the casting in the case of an AS statement happen? It won't screw me up, it just means for it to work on loads I'll have to move the logic to wherever the new casting post-load happens. That said, I support that change.
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2877:
----------------------------------
Status: Patch Available (was: Open)
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435425#comment-13435425 ]
Dmitriy V. Ryaboy commented on PIG-2877:
----------------------------------------
great to see this. Reviewboard?
Is a branch with this patch available on github somewhere?
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465261#comment-13465261 ]
Dmitriy V. Ryaboy commented on PIG-2877:
----------------------------------------
See PIG-2824 and my longish comment at the bottom of PIG-2661
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435435#comment-13435435 ]
Dmitriy V. Ryaboy commented on PIG-2877:
----------------------------------------
oh.. and can you post a patch with --no-prefix?
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465242#comment-13465242 ]
Jonathan Coveney commented on PIG-2877:
---------------------------------------
https://reviews.apache.org/r/7336/
and a new patch
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2877:
----------------------------------
Attachment: PIG-2877-0.patch
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Julien Le Dem (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470669#comment-13470669 ]
Julien Le Dem commented on PIG-2877:
------------------------------------
+1
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch, PIG-2877-3.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465272#comment-13465272 ]
Jonathan Coveney commented on PIG-2877:
---------------------------------------
Yeah, this would be totally compatible with that. The coercion has to happen somewhere, so whenever that is finished, I can just thread the Schema through there and do it. I think that this is actually ideal, b/c getting rid of the Foreach and directly doing the insertion should be a nice speed boost.
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2877:
----------------------------------
Resolution: Implemented
Status: Resolved (was: Patch Available)
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch, PIG-2877-2.patch, PIG-2877-3.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2877) Make SchemaTuple work in foreach (and
thus, in loads)
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2877:
----------------------------------
Attachment: PIG-2877-1.patch
> Make SchemaTuple work in foreach (and thus, in loads)
> ------------------------------------------------------
>
> Key: PIG-2877
> URL: https://issues.apache.org/jira/browse/PIG-2877
> Project: Pig
> Issue Type: Improvement
> Reporter: Jonathan Coveney
> Assignee: Jonathan Coveney
> Fix For: 0.11
>
> Attachments: PIG-2877-0.patch, PIG-2877-1.patch
>
>
> This wires foreaches to use SchemaTuple. As an aside, while SchemaTuple is not turned on by default, I made sure that it worked with ant test-commit if it were (there were some edge cases and lifecycle stuff that needed to be cleaned up).
> Further, I refactored some of the tests that I needed to work with for other reasons to be much cleaner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira