You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org> on 2009/10/02 04:58:23 UTC
[jira] Created: (PIG-990) Provide a way to pin LogicalOperator
Options
Provide a way to pin LogicalOperator Options
--------------------------------------------
Key: PIG-990
URL: https://issues.apache.org/jira/browse/PIG-990
Project: Pig
Issue Type: Bug
Components: impl
Reporter: Dmitriy V. Ryaboy
Priority: Minor
Fix For: 0.6.0
This is a proactive patch, setting up the groundwork for adding an optimizer.
Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783210#action_12783210 ]
Hadoop QA commented on PIG-990:
-------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12426301/pinned_options_6.patch
against trunk revision 884235.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/65/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/65/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/65/console
This message is automatically generated.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch, pinned_options_6.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782941#action_12782941 ]
Hadoop QA commented on PIG-990:
-------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12426178/pinned_options_4.patch
against trunk revision 884235.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/59/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/59/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/59/console
This message is automatically generated.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-990:
---------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch checked in. Thanks Dmitry.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch, pinned_options_6.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782695#action_12782695 ]
Ashutosh Chauhan commented on PIG-990:
--------------------------------------
We need to make a distinction between "user-pinned" and "system-pinned" options. I guess that needs to be added.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Status: Patch Available (was: Open)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch, pinned_options_6.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Attachment: pinned_options_5.patch
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Attachment: pinned_options_4.patch
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Status: Open (was: Patch Available)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764852#action_12764852 ]
Alan Gates commented on PIG-990:
--------------------------------
Looks good.
One comment, rather than referring to the default join as "regular" I think we should refer to it as "hash" or "symmetric hash", since these accurately describe how it works. That way users can specify the type of join they want, and if for whatever reason we switch the default they'll still get what they want.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: pinned_options.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764860#action_12764860 ]
Dmitriy V. Ryaboy commented on PIG-990:
---------------------------------------
Thanks for reviewing.
I only called it "regular" because that's what it was called in the enum that already existed inside LOJoin.
Maybe a better option would be to call the currently-default join 'hash', and also provide a 'default' key that, for now, will translate to hash, but can translate to something else (and stay pinned) if the defaults change. I'll add that to the next iteration of the patch (which will also contain whatever keyword is decided on for the map-side groups).
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: pinned_options.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Attachment: pinned_options_2.patch
- "using regular" will now translate into the default join method (hash join at the moment).
- "using hash" is an option in case we change what "regular" is
- renamed REGULAR to HASH in the Join code
- added the new group type option to CoGROUP, pinned where appropriate.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Status: Patch Available (was: Open)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Status: Open (was: Patch Available)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Status: Patch Available (was: Open)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783032#action_12783032 ]
Hadoop QA commented on PIG-990:
-------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12426246/pinned_options_5.patch
against trunk revision 884235.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
-1 release audit. The applied patch generated 363 release audit warnings (more than the trunk's current 362 warnings).
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/63/testReport/
Release audit warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/63/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/63/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/63/console
This message is automatically generated.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Attachment: pinned_options_5.patch
added a test for pinning group and join options.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782646#action_12782646 ]
Dmitriy V. Ryaboy commented on PIG-990:
---------------------------------------
I need to update it to apply to current trunk (I have the code but it's mixed in with some of my CBO work).
I'll do that today.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782693#action_12782693 ]
Hadoop QA commented on PIG-990:
-------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12426160/pinned_options_3.patch
against trunk revision 884235.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to cause Findbugs to fail.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/55/testReport/
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/55/console
This message is automatically generated.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy reassigned PIG-990:
-------------------------------------
Assignee: Dmitriy V. Ryaboy
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: pinned_options.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782717#action_12782717 ]
Ashutosh Chauhan commented on PIG-990:
--------------------------------------
Right. This jira is about user pinned so this is good to go. We do need system pinned for cbo. Lets discuss that offline.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai closed PIG-990.
--------------------------
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch, pinned_options_6.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Attachment: pinned_options.patch
This patch does a few things:
1) Introduce a HashSet of pinnedOptions to LogicalOperator.
2) Add a class constant to LOJoin to represent the "type of join" option
3) Modify the parser to pin the OPTION_JOIN when it is explicitly specified in the pig script
4) Introduce the syntax 'using "regular"' to pin the default (regular) join type. Currently that's redundant, but it's required if we start choosing join types for the user, and the user wants to tell us that this particular join must be done in the regular way.
5) minor cleanup in this area of the parser to use just one LogicalOperator variable instead of three.
No rush to commit this as it's not being used by anything yet, but posting it for review now so as to let people review the cbo-oriented changes one at a time instead of a giant code dump.
Note that if/when we add a group type as per PIG-984, that should also be an option that can be pinned by the user.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: pinned_options.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Attachment: (was: pinned_options_5.patch)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Fix Version/s: (was: 0.6.0)
0.7.0
Status: Patch Available (was: Open)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Attachment: pinned_options_6.patch
added Apache License header to the TestPinOptions.java
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch, pinned_options_5.patch, pinned_options_6.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Attachment: pinned_options_3.patch
updated to work with latest trunk (r884299)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782639#action_12782639 ]
Alan Gates commented on PIG-990:
--------------------------------
Is this ready to be run through the tests and possibly committed? It looks ready to go but it's never been marked patch available.
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782697#action_12782697 ]
Dmitriy V. Ryaboy commented on PIG-990:
---------------------------------------
This patch lost a parenthesis somewhere in my merge attempt. I am running the fixed version through test-commit and findbugs.
Ashutosh: pinOptions is userPinned. If we want to add systemPinned, we can do that as a separate patch (we don't even need that for the cbo as it stands, right?).
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-990) Provide a way to pin LogicalOperator
Options
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy updated PIG-990:
----------------------------------
Status: Open (was: Patch Available)
> Provide a way to pin LogicalOperator Options
> --------------------------------------------
>
> Key: PIG-990
> URL: https://issues.apache.org/jira/browse/PIG-990
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: pinned_options.patch, pinned_options_2.patch, pinned_options_3.patch, pinned_options_4.patch
>
>
> This is a proactive patch, setting up the groundwork for adding an optimizer.
> Some of the LogicalOperators have options. For example, LOJoin has a variety of join types (regular, fr, skewed, merge), which can be set by the user or chosen by a hypothetical optimizer. If a user selects a join type, pig philoophy guides us to always respect the user's choice and not explore alternatives. Therefore, we need a way to "pin" options.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.