You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2010/09/01 22:34:52 UTC
[jira] Created: (PIG-1590) Use POMergeJoin for Left Outer Join when
join using 'merge'
Use POMergeJoin for Left Outer Join when join using 'merge'
-----------------------------------------------------------
Key: PIG-1590
URL: https://issues.apache.org/jira/browse/PIG-1590
Project: Pig
Issue Type: Improvement
Components: impl
Affects Versions: 0.8.0
Reporter: Ashutosh Chauhan
Priority: Minor
C = join A by $0 left, B by $0 using 'merge';
will result in map-side sort merge join. Internally, it will translate to use POMergeCogroup + ForEachFlatten. POMergeCogroup places quite a few restrictions on its loaders (A and B in this case) which is cumbersome. Currently, only Zebra is known to satisfy all those requirements. It will be better to use POMergeJoin in this case, since it has far fewer requirements on its loader. Importantly, it works with PigStorage. Plus, POMergeJoin will be faster then POMergeCogroup + FE-Flatten.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1590) Use POMergeJoin for Left Outer Join
when join using 'merge'
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905207#action_12905207 ]
Ashutosh Chauhan commented on PIG-1590:
---------------------------------------
It will entail changes in POMergeJoin and LogToPhyTranslationVisitor.
> Use POMergeJoin for Left Outer Join when join using 'merge'
> -----------------------------------------------------------
>
> Key: PIG-1590
> URL: https://issues.apache.org/jira/browse/PIG-1590
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Affects Versions: 0.8.0
> Reporter: Ashutosh Chauhan
> Priority: Minor
>
> C = join A by $0 left, B by $0 using 'merge';
> will result in map-side sort merge join. Internally, it will translate to use POMergeCogroup + ForEachFlatten. POMergeCogroup places quite a few restrictions on its loaders (A and B in this case) which is cumbersome. Currently, only Zebra is known to satisfy all those requirements. It will be better to use POMergeJoin in this case, since it has far fewer requirements on its loader. Importantly, it works with PigStorage. Plus, POMergeJoin will be faster then POMergeCogroup + FE-Flatten.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1590) Use POMergeJoin for Left Outer Join
when join using 'merge'
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905995#action_12905995 ]
Ashutosh Chauhan commented on PIG-1590:
---------------------------------------
Also inner merge join on more then 2 then tables also translates into POMergeCogroup + FE + Flatten. Here also it can be translated to use POMergeJoin and enjoy the benefits which comes along with it. Though I suspect it will introduce much more complexity in POMergeJoin then the case for left outer merge join. So, may not be worth doing.
> Use POMergeJoin for Left Outer Join when join using 'merge'
> -----------------------------------------------------------
>
> Key: PIG-1590
> URL: https://issues.apache.org/jira/browse/PIG-1590
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Affects Versions: 0.8.0
> Reporter: Ashutosh Chauhan
> Priority: Minor
>
> C = join A by $0 left, B by $0 using 'merge';
> will result in map-side sort merge join. Internally, it will translate to use POMergeCogroup + ForEachFlatten. POMergeCogroup places quite a few restrictions on its loaders (A and B in this case) which is cumbersome. Currently, only Zebra is known to satisfy all those requirements. It will be better to use POMergeJoin in this case, since it has far fewer requirements on its loader. Importantly, it works with PigStorage. Plus, POMergeJoin will be faster then POMergeCogroup + FE-Flatten.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.