[jira] [Commented] (PIG-3585) Make union work with tez

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13836074#comment-13836074 ] 

Rohini Palaniswamy commented on PIG-3585:
-----------------------------------------

+1. The code is good if we have union after some processing. With this patch a simple union takes 3 vertices (2 load and 1 union) instead of 1 vertex. We should make it optimal to handle in a single vertex.

bq. Tez doesn't allow multiple inputs for the root vertex. We probably can't implement UNION with a single vertex.
   With MR, we just load both inputs as two different PigSplit in PigInputFormat. That should be possible here too right?

> Make union work with tez
> ------------------------
>
>                 Key: PIG-3585
>                 URL: https://issues.apache.org/jira/browse/PIG-3585
>             Project: Pig
>          Issue Type: Sub-task
>          Components: tez
>    Affects Versions: tez-branch
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>             Fix For: tez-branch
>
>         Attachments: PIG-3585-1.patch
>
>
> Unlike MR, union needs to be implemented as multiple vertices because Tez doesn't allow multiple inputs in root vertex.
> I propose that we implement union by connecting load vertices to union vertex with broadcasting edges.



--
This message was sent by Atlassian JIRA
(v6.1#6144)