You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Abhishek Gayakwad <a....@gmail.com> on 2013/10/25 13:31:13 UTC
using global params in oozie hive action
Hi All,
Referring to this link -
http://oozie.apache.org/docs/3.3.0/WorkflowFunctionalSpec.html#GlobalConfigurations
As of schema 0.4 we can define some global parameters (e.g.job-tracker,
name-node,job-xml & configuration) which will remove lots of repetitive
params in workflow xml.
for hive actions < schema 0.4 I can use only job-xml as global param, any
specific reason for this ?
http://oozie.apache.org/docs/3.3.0/DG_HiveActionExtension.html#Hive_Action_Schema_Version_0.3
We have huge oozie hive workflows and most of the hive actions have common
params, is there any option available or future plan to provide ability to
define global hive parameters ?? This will make workflow xmls really short.
Thanks
Abhishek
Re: using global params in oozie hive action
Posted by Robert Kanter <rk...@cloudera.com>.
Ah, I see, you want to put the <param> field in the global section. We
can't really do that because its specific to the hive action. If you don't
want to repeat the parameters over and over again, tou could put your hive
action into its own workflow and then use the subworkflow action to call
that workflow whenever you want to do your hive action; this way, you don't
have to keep repeating the params.
- Robert
On Mon, Oct 28, 2013 at 5:23 AM, Abhishek Gayakwad <a....@gmail.com>wrote:
> Hi Robert,
>
> Hive schema was not enabled on our cluster, we will try to get it enabled.
>
> We have lots of hive actions defined in wf xml, and in most of the actions
> <param>s are same, is it possible to provide common params for hive
> actions(or all action) which hive action can pick up and provide as
> -hiveconf while running hive queries.
>
> <global> tag accepts only 4 options (job-tracker,name-node,job-xml and
> configuration), I don't see any proper place to provide param here and if
> we provide them in configuration, will they be picked by hive action as
> parameters to hive query.
>
> Thanks
> Abhishek
>
>
> On Fri, Oct 25, 2013 at 11:19 PM, Robert Kanter <rkanter@cloudera.com
> >wrote:
>
> > Hi Abhishek,
> >
> > Hive schemas < 0.3 had already been created when we added the global
> > section, so we couldn't change them. Is there a reason you can't use
> Hive
> > schema 0.4? IIRC, its almost identical to schema 0.3 except that it
> > doesn't require a Job Tracker or Name Node, which is what you want.
> >
> >
> > - Robert
> >
> >
> > On Fri, Oct 25, 2013 at 4:31 AM, Abhishek Gayakwad <a.gayakwad@gmail.com
> > >wrote:
> >
> > > Hi All,
> > >
> > > Referring to this link -
> > >
> > >
> >
> http://oozie.apache.org/docs/3.3.0/WorkflowFunctionalSpec.html#GlobalConfigurations
> > >
> > > As of schema 0.4 we can define some global parameters (e.g.job-tracker,
> > > name-node,job-xml & configuration) which will remove lots of repetitive
> > > params in workflow xml.
> > >
> > > for hive actions < schema 0.4 I can use only job-xml as global param,
> any
> > > specific reason for this ?
> > >
> > >
> > >
> >
> http://oozie.apache.org/docs/3.3.0/DG_HiveActionExtension.html#Hive_Action_Schema_Version_0.3
> > >
> > > We have huge oozie hive workflows and most of the hive actions have
> > common
> > > params, is there any option available or future plan to provide ability
> > to
> > > define global hive parameters ?? This will make workflow xmls really
> > short.
> > >
> > >
> > > Thanks
> > > Abhishek
> > >
> >
>
Re: using global params in oozie hive action
Posted by Abhishek Gayakwad <a....@gmail.com>.
Hi Robert,
Hive schema was not enabled on our cluster, we will try to get it enabled.
We have lots of hive actions defined in wf xml, and in most of the actions
<param>s are same, is it possible to provide common params for hive
actions(or all action) which hive action can pick up and provide as
-hiveconf while running hive queries.
<global> tag accepts only 4 options (job-tracker,name-node,job-xml and
configuration), I don't see any proper place to provide param here and if
we provide them in configuration, will they be picked by hive action as
parameters to hive query.
Thanks
Abhishek
On Fri, Oct 25, 2013 at 11:19 PM, Robert Kanter <rk...@cloudera.com>wrote:
> Hi Abhishek,
>
> Hive schemas < 0.3 had already been created when we added the global
> section, so we couldn't change them. Is there a reason you can't use Hive
> schema 0.4? IIRC, its almost identical to schema 0.3 except that it
> doesn't require a Job Tracker or Name Node, which is what you want.
>
>
> - Robert
>
>
> On Fri, Oct 25, 2013 at 4:31 AM, Abhishek Gayakwad <a.gayakwad@gmail.com
> >wrote:
>
> > Hi All,
> >
> > Referring to this link -
> >
> >
> http://oozie.apache.org/docs/3.3.0/WorkflowFunctionalSpec.html#GlobalConfigurations
> >
> > As of schema 0.4 we can define some global parameters (e.g.job-tracker,
> > name-node,job-xml & configuration) which will remove lots of repetitive
> > params in workflow xml.
> >
> > for hive actions < schema 0.4 I can use only job-xml as global param, any
> > specific reason for this ?
> >
> >
> >
> http://oozie.apache.org/docs/3.3.0/DG_HiveActionExtension.html#Hive_Action_Schema_Version_0.3
> >
> > We have huge oozie hive workflows and most of the hive actions have
> common
> > params, is there any option available or future plan to provide ability
> to
> > define global hive parameters ?? This will make workflow xmls really
> short.
> >
> >
> > Thanks
> > Abhishek
> >
>
Re: using global params in oozie hive action
Posted by Robert Kanter <rk...@cloudera.com>.
Hi Abhishek,
Hive schemas < 0.3 had already been created when we added the global
section, so we couldn't change them. Is there a reason you can't use Hive
schema 0.4? IIRC, its almost identical to schema 0.3 except that it
doesn't require a Job Tracker or Name Node, which is what you want.
- Robert
On Fri, Oct 25, 2013 at 4:31 AM, Abhishek Gayakwad <a....@gmail.com>wrote:
> Hi All,
>
> Referring to this link -
>
> http://oozie.apache.org/docs/3.3.0/WorkflowFunctionalSpec.html#GlobalConfigurations
>
> As of schema 0.4 we can define some global parameters (e.g.job-tracker,
> name-node,job-xml & configuration) which will remove lots of repetitive
> params in workflow xml.
>
> for hive actions < schema 0.4 I can use only job-xml as global param, any
> specific reason for this ?
>
>
> http://oozie.apache.org/docs/3.3.0/DG_HiveActionExtension.html#Hive_Action_Schema_Version_0.3
>
> We have huge oozie hive workflows and most of the hive actions have common
> params, is there any option available or future plan to provide ability to
> define global hive parameters ?? This will make workflow xmls really short.
>
>
> Thanks
> Abhishek
>