You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Cheolsoo Park (JIRA)" <ji...@apache.org> on 2013/10/04 02:29:45 UTC
[jira] [Comment Edited] (PIG-3500) Initial implementation of
TezCompiler
[ https://issues.apache.org/jira/browse/PIG-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785730#comment-13785730 ]
Cheolsoo Park edited comment on PIG-3500 at 10/4/13 12:29 AM:
--------------------------------------------------------------
Attached includes an initial version of TezCompiler with unit tests. Note that query #3 is compiled into 3 Tez vertices (two input vertices and one join vertex) unlike MR plan.
The unit test can run with ant test clean -Dtestcase=TestTezCompiler.
was (Author: cheolsoo):
Attached includes an initial version of TezCompiler with unit tests. Note that it query #3 is compiled into 3 Tez vertices (two input vertices and one join vertex) unlike MR plan.
The unit test can run with ant test clean -Dtestcase=TestTezCompiler.
> Initial implementation of TezCompiler
> -------------------------------------
>
> Key: PIG-3500
> URL: https://issues.apache.org/jira/browse/PIG-3500
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: tez-branch
>
> Attachments: PIG-3500-1.patch
>
>
> Implement TezCompiler that compiles physical plan into tez plan. To begin with, we can implement the initial version that works for basic queries as follows:
> # Load-Filter-Store
> {code}
> a = load 'file:///tmp/input' as (x:int, y:int);
> b = filter a by x > 0;
> c = foreach b generate y;
> store c into 'file:///tmp/output';
> {code}
> # Load-Filter-GroupBy-Store
> {code}
> a = load 'file:///tmp/input' as (x:int, y:int);
> b = group a by x;
> c = foreach b generate group, a;
> store c into 'file:///tmp/output';
> {code}
> # Load1-Load2-Join-Store
> {code}
> a = load 'file:///tmp/input1' as (x:int, y:int);
> b = load 'file:///tmp/input2' as (x:int, z:int);
> c = join a by x, b by x;
> d = foreach c generate a::x as x, y, z;
> store d into 'file:///tmp/output';
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)