[jira] [Created] (PIG-5082) Tez UnionOptimizer creates vertex group with one member

["Travis Woodruff (JIRA)" <ji...@apache.org>]

Travis Woodruff created PIG-5082:
------------------------------------

             Summary: Tez UnionOptimizer creates vertex group with one member
                 Key: PIG-5082
                 URL: https://issues.apache.org/jira/browse/PIG-5082
             Project: Pig
          Issue Type: Bug
          Components: tez
    Affects Versions: 0.16.0
            Reporter: Travis Woodruff
            Priority: Minor


This script results in a vertex group with one member:

{code}
a = LOAD '/tmp/empty.txt' USING PigStorage('\t') AS (x:chararray);
b = LOAD '/tmp/empty.txt' USING PigStorage('\t') AS (x:chararray);
c = LOAD '/tmp/empty.txt' USING PigStorage('\t') AS (y:chararray);

u1 = UNION ONSCHEMA a, b;
SPLIT u1 INTO r IF x != '', s OTHERWISE;
d = JOIN r BY x LEFT, c BY y;
u2 = UNION ONSCHEMA d, s;
e = FILTER u2 BY x == '';
f = FILTER u2 BY x == 'm';
u3 = UNION ONSCHEMA e, f;
DUMP u3;
{code}

Which results in:
{code}
java.lang.IllegalArgumentException: VertexGroup must have at least 2 members
	at org.apache.tez.dag.api.VertexGroup.<init>(VertexGroup.java:77)
	at org.apache.tez.dag.api.DAG.createVertexGroup(DAG.java:202)
	at org.apache.pig.backend.hadoop.executionengine.tez.TezDagBuilder.visitTezOp(TezDagBuilder.java:396)
	at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator.visit(TezOperator.java:255)
...
{code}

This seems to be happening because {{UnionOptimizer}} is replacing a union with a vertex group and then optimizing away a predecessor union thus removing a node and resulting in a vertex group with one member.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)