You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Grandl Robert <rg...@yahoo.com> on 2014/07/10 05:43:58 UTC
pass new job name to tez
Hi guys,
I am trying to identify a DAG in Tez with a different id, based on job
name(for e.g. query55.sql from hive-testbench) + input size.
So my new identifier should be for example query55_2048MB. It seems that a DAG in tez, already takes a name which comes from a jobPlan.getName()
passed through google protobuf thing. Because the DAG is generated in Hive, I think the new identifier should come from Hive right ?
Can you pinpoint me which classes, I should change in Hive in order to propagate the new identifier for the DAG ?
Thanks,
Robert
Re: pass new job name to tez
Posted by Grandl Robert <rg...@yahoo.com>.
I created another variable hive.job.name and set it from sql script and works.
Thanks,
Robert
On Monday, August 25, 2014 5:40 PM, Grandl Robert <rg...@yahoo.com> wrote:
Hi guys,
I am still struggling with finding a way to specify a job name in the hive query and propagate that name in Tez dag. For example my hive script is named query55.
Do you know a way I can pass this name to be available in Tez dag ?
Navis suggested to look at hive.query.id. I tried in my query55.sql to do a set hive.query.id=query55; but the name did not change actually.
Did someone try this before, or have more suggestions about an easy way to do it ?
Actually what I want to achieve here is to correlate a tez dag with a query name. So have a tez dag name to be specified by the user, and hence comes through the hive query.
Thanks,
Robert
On Wednesday, July 9, 2014 8:56 PM, Navis류승우 <na...@nexr.com> wrote:
In GenTezProcContext, you can find "new TezWork(conf.getVar(HiveConf.ConfVars.HIVEQUERYID))" part. And the final name is name + ":" + (++counter);
Thanks,
Navis
2014-07-10 12:43 GMT+09:00 Grandl Robert <rg...@yahoo.com>:
Hi guys,
>
>
>I am trying to identify a DAG in Tez with a different id, based on job
name(for e.g. query55.sql from hive-testbench) + input size.
>
>
>
>So my new identifier should be for example query55_2048MB. It seems that a DAG in tez, already takes a name which comes from a jobPlan.getName()
passed through google protobuf thing. Because the DAG is generated in Hive, I think the new identifier should come from Hive right ?
>
>
>Can you pinpoint me which classes, I should change in Hive in order to propagate the new identifier for the DAG ?
>
>
>Thanks,
>Robert
>
>
Re: pass new job name to tez
Posted by Grandl Robert <rg...@yahoo.com>.
Hi guys,
I am still struggling with finding a way to specify a job name in the hive query and propagate that name in Tez dag. For example my hive script is named query55.
Do you know a way I can pass this name to be available in Tez dag ?
Navis suggested to look at hive.query.id. I tried in my query55.sql to do a set hive.query.id=query55; but the name did not change actually.
Did someone try this before, or have more suggestions about an easy way to do it ?
Actually what I want to achieve here is to correlate a tez dag with a query name. So have a tez dag name to be specified by the user, and hence comes through the hive query.
Thanks,
Robert
On Wednesday, July 9, 2014 8:56 PM, Navis류승우 <na...@nexr.com> wrote:
In GenTezProcContext, you can find "new TezWork(conf.getVar(HiveConf.ConfVars.HIVEQUERYID))" part. And the final name is name + ":" + (++counter);
Thanks,
Navis
2014-07-10 12:43 GMT+09:00 Grandl Robert <rg...@yahoo.com>:
Hi guys,
>
>
>I am trying to identify a DAG in Tez with a different id, based on job
name(for e.g. query55.sql from hive-testbench) + input size.
>
>
>
>So my new identifier should be for example query55_2048MB. It seems that a DAG in tez, already takes a name which comes from a jobPlan.getName()
passed through google protobuf thing. Because the DAG is generated in Hive, I think the new identifier should come from Hive right ?
>
>
>Can you pinpoint me which classes, I should change in Hive in order to propagate the new identifier for the DAG ?
>
>
>Thanks,
>Robert
>
>
Re: pass new job name to tez
Posted by Navis류승우 <na...@nexr.com>.
In GenTezProcContext, you can find "new
TezWork(conf.getVar(HiveConf.ConfVars.HIVEQUERYID))" part. And the final
name is name + ":" + (++counter);
Thanks,
Navis
2014-07-10 12:43 GMT+09:00 Grandl Robert <rg...@yahoo.com>:
> Hi guys,
>
> I am trying to identify a DAG in Tez with a different id, based on job
> name(for e.g. query55.sql from hive-testbench) + input size.
>
> So my new identifier should be for example query55_2048MB. It seems that a
> DAG in tez, already takes a name which comes from a jobPlan.getName()
> passed through google protobuf thing. Because the DAG is generated in Hive,
> I think the new identifier should come from Hive right ?
>
> Can you pinpoint me which classes, I should change in Hive in order to
> propagate the new identifier for the DAG ?
>
> Thanks,
> Robert
>
>