You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Antonio Magnaghi (JIRA)" <ji...@apache.org> on 2007/11/21 01:47:43 UTC
[jira] Created: (PIG-28) Independence of Logical Plan from Physical
Plan
Independence of Logical Plan from Physical Plan
-----------------------------------------------
Key: PIG-28
URL: https://issues.apache.org/jira/browse/PIG-28
Project: Pig
Issue Type: Bug
Components: impl
Reporter: Antonio Magnaghi
The implementation of a Pig Logical Plan is currently closely tied with implementation details of the Physical Plan.
For instance, the construction of the tree of Logical Operators puts in place the data pipes in conjunction with the construction of Eval Spec and Cond nodes. Data pipes are utilized later on during the execution of the query. Conceptually, this appears to obfuscate the code.
It would be nice to remove or reduce this tight coupling. This aspect may also be relevant to the proposal to provide an abstraction layer for Pig, where front-end activities are clearly separated from back-end tasks and responsibilities (see wiki at: http://wiki.apache.org/pig/PigAbstractionLayer)
Two possible alternatives initially discussed could be:
1.) to decouple logical from physical part of plan construction by having two distinct sets of Eval Specs: a Logical Eval Spec/Cond set without physical plan details and a Physical Eval Spec/Cond set that addresses the aspects of the physical plan.
2.) to merge the logical operators with the local physical operators (used by the LocalPlanCompiler) and preserve the MapReduce physical operator.
The second option may be preferable because less extensive changes are involved. Additionally, choosing the second option would integrate the Local Plan data structures into the core code packages: long term this could simplify the maintenance and support of the Local Plan construction.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-28) Independence of Logical Plan from
Physical Plan
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-28?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich resolved PIG-28.
-------------------------------
Resolution: Fixed
closing since this is being addressed by pipeline rewrite
> Independence of Logical Plan from Physical Plan
> -----------------------------------------------
>
> Key: PIG-28
> URL: https://issues.apache.org/jira/browse/PIG-28
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Antonio Magnaghi
>
> The implementation of a Pig Logical Plan is currently closely tied with implementation details of the Physical Plan.
> For instance, the construction of the tree of Logical Operators puts in place the data pipes in conjunction with the construction of Eval Spec and Cond nodes. Data pipes are utilized later on during the execution of the query. Conceptually, this appears to obfuscate the code.
> It would be nice to remove or reduce this tight coupling. This aspect may also be relevant to the proposal to provide an abstraction layer for Pig, where front-end activities are clearly separated from back-end tasks and responsibilities (see wiki at: http://wiki.apache.org/pig/PigAbstractionLayer)
> Two possible alternatives initially discussed could be:
> 1.) to decouple logical from physical part of plan construction by having two distinct sets of Eval Specs: a Logical Eval Spec/Cond set without physical plan details and a Physical Eval Spec/Cond set that addresses the aspects of the physical plan.
> 2.) to merge the logical operators with the local physical operators (used by the LocalPlanCompiler) and preserve the MapReduce physical operator.
> The second option may be preferable because less extensive changes are involved. Additionally, choosing the second option would integrate the Local Plan data structures into the core code packages: long term this could simplify the maintenance and support of the Local Plan construction.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.