You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Pradeep Kamath (JIRA)" <ji...@apache.org> on 2008/08/28 01:01:44 UTC
[jira] Created: (PIG-401) SplitIntroducer does not handle some
cases correctly
SplitIntroducer does not handle some cases correctly
----------------------------------------------------
Key: PIG-401
URL: https://issues.apache.org/jira/browse/PIG-401
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Pradeep Kamath
Fix For: types_branch
Attachments: PIG-401-partial.patch
The splitIntroducer does not handle the following cases correctly:
* As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
Current code which needs to change:
{code}
List<LogicalOperator> sucs = mPlan.getSuccessors(root);
if(sucs==null) return;
int size = sucs.size();
if(size==0 || size==1) return; --> THIS should change to recursively process the succesor if it exists(size == 1 case)
{code}
* If there are more than one operators in the plan which need implicit splits to be introduced
* If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
* The kind of changes need for cogroup maybe needed for some other operators
I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-401) SplitIntroducer does not handle some
cases correctly
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-401:
---------------------------
Attachment: PIG401.patch
This patch moves the SplitIntroducer into the optimizer (and renames it as ImplicitSplitInserter for whatever reason). This addresses two issues:
1) catching splits past the initial node in the graph
2) patching up schemas and inner plans of other operators after a split is introduced.
> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
> Key: PIG-401
> URL: https://issues.apache.org/jira/browse/PIG-401
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG-401-partial.patch, PIG401.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
> * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
> List<LogicalOperator> sucs = mPlan.getSuccessors(root);
> if(sucs==null) return;
> int size = sucs.size();
> if(size==0 || size==1) return; --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
> * If there are more than one operators in the plan which need implicit splits to be introduced
> * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
> * The kind of changes need for cogroup maybe needed for some other operators
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-401) SplitIntroducer does not handle some
cases correctly
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-401:
---------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
PIG401.patch checked in.
> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
> Key: PIG-401
> URL: https://issues.apache.org/jira/browse/PIG-401
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG-401-partial.patch, PIG401.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
> * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
> List<LogicalOperator> sucs = mPlan.getSuccessors(root);
> if(sucs==null) return;
> int size = sucs.size();
> if(size==0 || size==1) return; --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
> * If there are more than one operators in the plan which need implicit splits to be introduced
> * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
> * The kind of changes need for cogroup maybe needed for some other operators
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-401) SplitIntroducer does not handle some
cases correctly
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-401:
-------------------------------
Priority: Critical (was: Major)
> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
> Key: PIG-401
> URL: https://issues.apache.org/jira/browse/PIG-401
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG-401-partial.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
> * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
> List<LogicalOperator> sucs = mPlan.getSuccessors(root);
> if(sucs==null) return;
> int size = sucs.size();
> if(size==0 || size==1) return; --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
> * If there are more than one operators in the plan which need implicit splits to be introduced
> * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
> * The kind of changes need for cogroup maybe needed for some other operators
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-401) SplitIntroducer does not handle some
cases correctly
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-401:
-------------------------------
Attachment: PIG-401-partial.patch
> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
> Key: PIG-401
> URL: https://issues.apache.org/jira/browse/PIG-401
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Fix For: types_branch
>
> Attachments: PIG-401-partial.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
> * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
> List<LogicalOperator> sucs = mPlan.getSuccessors(root);
> if(sucs==null) return;
> int size = sucs.size();
> if(size==0 || size==1) return; --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
> * If there are more than one operators in the plan which need implicit splits to be introduced
> * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
> * The kind of changes need for cogroup maybe needed for some other operators
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-401) SplitIntroducer does not handle some
cases correctly
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-401:
---------------------------
Status: Patch Available (was: Open)
> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
> Key: PIG-401
> URL: https://issues.apache.org/jira/browse/PIG-401
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG-401-partial.patch, PIG401.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
> * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
> List<LogicalOperator> sucs = mPlan.getSuccessors(root);
> if(sucs==null) return;
> int size = sucs.size();
> if(size==0 || size==1) return; --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
> * If there are more than one operators in the plan which need implicit splits to be introduced
> * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
> * The kind of changes need for cogroup maybe needed for some other operators
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-401) SplitIntroducer does not handle some
cases correctly
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich reassigned PIG-401:
----------------------------------
Assignee: Alan Gates
> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
> Key: PIG-401
> URL: https://issues.apache.org/jira/browse/PIG-401
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Pradeep Kamath
> Assignee: Alan Gates
> Fix For: types_branch
>
> Attachments: PIG-401-partial.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
> * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
> List<LogicalOperator> sucs = mPlan.getSuccessors(root);
> if(sucs==null) return;
> int size = sucs.size();
> if(size==0 || size==1) return; --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
> * If there are more than one operators in the plan which need implicit splits to be introduced
> * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
> * The kind of changes need for cogroup maybe needed for some other operators
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.