You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Pradeep Kamath (JIRA)" <ji...@apache.org> on 2008/08/28 01:01:44 UTC

[jira] Created: (PIG-401) SplitIntroducer does not handle some cases correctly

SplitIntroducer does not handle some cases correctly
----------------------------------------------------

                 Key: PIG-401
                 URL: https://issues.apache.org/jira/browse/PIG-401
             Project: Pig
          Issue Type: Bug
    Affects Versions: types_branch
            Reporter: Pradeep Kamath
             Fix For: types_branch
         Attachments: PIG-401-partial.patch

The splitIntroducer does not handle the following cases correctly:
   * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
Current code which needs to change:
{code}
        List<LogicalOperator> sucs = mPlan.getSuccessors(root);
        if(sucs==null) return; 
        int size = sucs.size();
        if(size==0 || size==1) return;  --> THIS should change to recursively process the succesor if it exists(size == 1 case)
{code}

   * If there are more than one operators in the plan which need implicit splits to be introduced
   * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
   * The kind of changes need for cogroup maybe needed for some other operators 

I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-401) SplitIntroducer does not handle some cases correctly

Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Gates updated PIG-401:
---------------------------

    Attachment: PIG401.patch

This patch moves the SplitIntroducer into the optimizer (and renames it as ImplicitSplitInserter for whatever reason).  This addresses two issues:

1) catching splits past the initial node in the graph
2) patching up schemas and inner plans of other operators after a split is introduced.

> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
>                 Key: PIG-401
>                 URL: https://issues.apache.org/jira/browse/PIG-401
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Alan Gates
>            Priority: Critical
>             Fix For: types_branch
>
>         Attachments: PIG-401-partial.patch, PIG401.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
>    * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
>         List<LogicalOperator> sucs = mPlan.getSuccessors(root);
>         if(sucs==null) return; 
>         int size = sucs.size();
>         if(size==0 || size==1) return;  --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
>    * If there are more than one operators in the plan which need implicit splits to be introduced
>    * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
>    * The kind of changes need for cogroup maybe needed for some other operators 
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-401) SplitIntroducer does not handle some cases correctly

Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Gates updated PIG-401:
---------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

PIG401.patch checked in.

> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
>                 Key: PIG-401
>                 URL: https://issues.apache.org/jira/browse/PIG-401
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Alan Gates
>            Priority: Critical
>             Fix For: types_branch
>
>         Attachments: PIG-401-partial.patch, PIG401.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
>    * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
>         List<LogicalOperator> sucs = mPlan.getSuccessors(root);
>         if(sucs==null) return; 
>         int size = sucs.size();
>         if(size==0 || size==1) return;  --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
>    * If there are more than one operators in the plan which need implicit splits to be introduced
>    * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
>    * The kind of changes need for cogroup maybe needed for some other operators 
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-401) SplitIntroducer does not handle some cases correctly

Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Olga Natkovich updated PIG-401:
-------------------------------

    Priority: Critical  (was: Major)

> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
>                 Key: PIG-401
>                 URL: https://issues.apache.org/jira/browse/PIG-401
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Alan Gates
>            Priority: Critical
>             Fix For: types_branch
>
>         Attachments: PIG-401-partial.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
>    * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
>         List<LogicalOperator> sucs = mPlan.getSuccessors(root);
>         if(sucs==null) return; 
>         int size = sucs.size();
>         if(size==0 || size==1) return;  --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
>    * If there are more than one operators in the plan which need implicit splits to be introduced
>    * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
>    * The kind of changes need for cogroup maybe needed for some other operators 
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-401) SplitIntroducer does not handle some cases correctly

Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pradeep Kamath updated PIG-401:
-------------------------------

    Attachment: PIG-401-partial.patch

> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
>                 Key: PIG-401
>                 URL: https://issues.apache.org/jira/browse/PIG-401
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>             Fix For: types_branch
>
>         Attachments: PIG-401-partial.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
>    * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
>         List<LogicalOperator> sucs = mPlan.getSuccessors(root);
>         if(sucs==null) return; 
>         int size = sucs.size();
>         if(size==0 || size==1) return;  --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
>    * If there are more than one operators in the plan which need implicit splits to be introduced
>    * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
>    * The kind of changes need for cogroup maybe needed for some other operators 
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-401) SplitIntroducer does not handle some cases correctly

Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Gates updated PIG-401:
---------------------------

    Status: Patch Available  (was: Open)

> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
>                 Key: PIG-401
>                 URL: https://issues.apache.org/jira/browse/PIG-401
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Alan Gates
>            Priority: Critical
>             Fix For: types_branch
>
>         Attachments: PIG-401-partial.patch, PIG401.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
>    * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
>         List<LogicalOperator> sucs = mPlan.getSuccessors(root);
>         if(sucs==null) return; 
>         int size = sucs.size();
>         if(size==0 || size==1) return;  --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
>    * If there are more than one operators in the plan which need implicit splits to be introduced
>    * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
>    * The kind of changes need for cogroup maybe needed for some other operators 
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (PIG-401) SplitIntroducer does not handle some cases correctly

Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Olga Natkovich reassigned PIG-401:
----------------------------------

    Assignee: Alan Gates

> SplitIntroducer does not handle some cases correctly
> ----------------------------------------------------
>
>                 Key: PIG-401
>                 URL: https://issues.apache.org/jira/browse/PIG-401
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Alan Gates
>             Fix For: types_branch
>
>         Attachments: PIG-401-partial.patch
>
>
> The splitIntroducer does not handle the following cases correctly:
>    * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
> Current code which needs to change:
> {code}
>         List<LogicalOperator> sucs = mPlan.getSuccessors(root);
>         if(sucs==null) return; 
>         int size = sucs.size();
>         if(size==0 || size==1) return;  --> THIS should change to recursively process the succesor if it exists(size == 1 case)
> {code}
>    * If there are more than one operators in the plan which need implicit splits to be introduced
>    * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
>    * The kind of changes need for cogroup maybe needed for some other operators 
> I started looking at it some and have a patch for the first two issues - *UNTESTED* which I am attaching to this issue

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.