You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2015/07/21 21:22:04 UTC
[jira] [Comment Edited] (TEZ-1190) Allow multiple edges between two
vertexes
[ https://issues.apache.org/jira/browse/TEZ-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635661#comment-14635661 ]
Rohini Palaniswamy edited comment on TEZ-1190 at 7/21/15 7:21 PM:
------------------------------------------------------------------
[~bikassaha],
The classes are UnionOptimizer and TezDAGBuilder before PIG-4495 went in. The code is not straightforward tez code though.
For eg:
A = LOAD 'data';
SPLIT A into B if $0 > 5, C if $0 > 10 and $0 < 12; // This is just for an example. This simple condition can be written with just FILTER instead of split and union.
D = UNION B, C;
E = GROUP D by $1;
The pig plan would be like: V1(Load) - > VG1, V1(Load) - >VG1, VG1 - >V2 (Group by)
Vertex group VG1 takes two inputs from same source vertex V1. The output vertex is only one i.e V2.
was (Author: rohini):
[~bikassaha],
The classes are UnionOptimizer and TezDAGBuilder before PIG-4495 went in. The code is not straightforward tez code though.
For eg:
A = LOAD 'data';
SPLIT A into B if $0 > 5, C if $0 > 10 and $0 < 12; // This is just for an example. This simple condition can be written with just FILTER instead of split and union.
D = UNION B, C;
E = GROUP D by $1;
The pig plan would be like - V1(Load) -> VG1, V1(Load)->VG1, VG1->V2 (Group by)
Vertex group VG1 takes two inputs from same source vertex V1. The output vertex is only one i.e V2.
> Allow multiple edges between two vertexes
> -----------------------------------------
>
> Key: TEZ-1190
> URL: https://issues.apache.org/jira/browse/TEZ-1190
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Daniel Dai
>
> This will be helpful in some scenario. In particular example, we can merge two small pipelines together in one pair of vertex. Note it is possible the edge type between the two vertexes are different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)