You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Satish Subhashrao Saley (JIRA)" <ji...@apache.org> on 2017/09/19 21:31:00 UTC

[jira] [Updated] (PIG-4120) Broadcast the index file in case of POMergeCoGroup and POMergeJoin

     [ https://issues.apache.org/jira/browse/PIG-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Satish Subhashrao Saley updated PIG-4120:
-----------------------------------------
    Attachment: PIG-4120-1.patch

> Broadcast the index file in case of POMergeCoGroup and POMergeJoin
> ------------------------------------------------------------------
>
>                 Key: PIG-4120
>                 URL: https://issues.apache.org/jira/browse/PIG-4120
>             Project: Pig
>          Issue Type: Sub-task
>          Components: tez
>            Reporter: Rohini Palaniswamy
>            Assignee: Satish Subhashrao Saley
>             Fix For: 0.18.0
>
>         Attachments: PIG-4120-1.patch
>
>
> Currently merge join and merge cogroup use two DAGs - the first DAG creates the index file in hdfs and second DAG does the merge join.  Similar to replicate join, we can broadcast the index file and cache it and use it in merge join and merge cogroup. This will give better performance and also eliminate need for the second DAG.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)