[jira] Commented: (PIG-158) Rework logical plan

["Pi Song (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584120#action_12584120 ] 

Pi Song commented on PIG-158:
-----------------------------

This is not good for nested operations.

I really want to see we switch to the plans in plans approach. This way those operators that do not have predecessors aren't problems because they are not root nodes on the main logical plan.

Take LOFilter as example. This operator takes a bag of tuples as input in the primary plan. For each input tuple, there is a transient internal logical plan that does consume the tuple and outputs something. Then the output tuple gets processed in the outer operator.

We can have special load/store that only work with one tuple in a bag at a time to make this more clean.

Visitors when visiting a node can visit internal nodes in internal plans first before visiting the actual node. We may need to have a distinction between operators that have plans and operators that don't. This is not too complex in my opinion.


The nested data model in Pig is already good. I just want to see nested plans to cope with it nicely.

> Rework logical plan
> -------------------
>
>                 Key: PIG-158
>                 URL: https://issues.apache.org/jira/browse/PIG-158
>             Project: Pig
>          Issue Type: Sub-task
>          Components: impl
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: logical_operators.patch, logical_operators_rev_1.patch, logical_operators_rev_2.patch, logical_operators_rev_3.patch
>
>
> Rework the logical plan in line with http://wiki.apache.org/pig/PigExecutionModel

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.