You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@tez.apache.org by Raajay <ra...@gmail.com> on 2015/09/01 23:12:30 UTC

DAG serialization

Hi,

Currently, I am running Hive on Tez. I wish to make some changes to the
DAGs generated by Hive before running it on Tez; changes include altering
parallelism and TaskLocationHints, etc..

Which DAG should I tap into from Hive, and write to disk, so that I can
read later, alter it and then submit to Tez ?

It does not seem like DAG.java in tez-api can be serialized.


Raajay

Re: DAG serialization

Posted by "r7raul1984@163.com" <r7...@163.com>.

Try hive hook
http://stackoverflow.com/questions/17461932/hive-execution-hook   

r7raul1984@163.com

From: Raajay
Date: 2015-09-02 05:12
To: user
Subject: DAG serialization
Hi, 

Currently, I am running Hive on Tez. I wish to make some changes to the DAGs generated by Hive before running it on Tez; changes include altering parallelism and TaskLocationHints, etc..

Which DAG should I tap into from Hive, and write to disk, so that I can read later, alter it and then submit to Tez ?

It does not seem like DAG.java in tez-api can be serialized.

Raajay

Re: DAG serialization

Posted by Hitesh Shah <hi...@apache.org>.

This may be  a question better suited for the Hive dev mailing list: dev@hive.apache.org. The change would likely be somewhere in the layer which takes the Hive logical plan and converts it into the Tez physical dag. 

— Hitesh 

On Sep 1, 2015, at 2:12 PM, Raajay <ra...@gmail.com> wrote:

> Hi, 
> 
> Currently, I am running Hive on Tez. I wish to make some changes to the DAGs generated by Hive before running it on Tez; changes include altering parallelism and TaskLocationHints, etc..
> 
> Which DAG should I tap into from Hive, and write to disk, so that I can read later, alter it and then submit to Tez ?
> 
> It does not seem like DAG.java in tez-api can be serialized.
> 
> 
> Raajay
>