You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Ion Bara <io...@gmail.com> on 2015/10/29 23:06:07 UTC

MROperPlan structure

Hello,

I am doing work on running Pig compiled Map Reduce jobs on our own MR
framework with similar interface.

Can someone point my to a person which can help me? (more details below)

I have studied the source code however I want to be able to inspect the
structure of MROperPlan structure for various queries.

Neither printing out the object trought .toString() method and debugging
haven't yielded any results.

Thanks in advance,
Ion

Re: MROperPlan structure

Posted by Daniel Dai <da...@hortonworks.com>.
MROperPlan is a graph containing all the MR jobs wrapped in MapReduceOper.
You can write a MROpPlanVisitor visitor to traverse the MROperPlan. Inside
the visitor, you can translate every MR job into your own framework
operator. The other choice is to rewrite MRCompiler.java - the place Pig
translate physical plan into MR framework.

Thanks,
Daniel

On 10/29/15, 3:06 PM, "Ion Bara" <io...@gmail.com> wrote:

>Hello,
>
>I am doing work on running Pig compiled Map Reduce jobs on our own MR
>framework with similar interface.
>
>Can someone point my to a person which can help me? (more details below)
>
>I have studied the source code however I want to be able to inspect the
>structure of MROperPlan structure for various queries.
>
>Neither printing out the object trought .toString() method and debugging
>haven't yielded any results.
>
>Thanks in advance,
>Ion