You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Lars Francke <la...@gmail.com> on 2020/08/10 15:26:54 UTC
Passing variables from dataset to workflow in Coordinator
Hi,
I have a simple coordinator with a single dataset:
<uri-template>${hadoop_nameNode}/${coord_stagingFolder}/${YEAR}/${MONTH}/${DAY}</uri-template>
I also have a corresponding input-event:
<data-in name="foo_event" dataset="foo_ds">
<instance>${coord:current(-1)}</instance>
</data-in>
Now in my workflow I need to pass in the ${YEAR}, MONTH and DAY variables
as properties.
So far we've always used "fake" <output-events> and basically template our
variables that way (so used data-out things that don't actually correspond
to a folder anywhere).
What's the correct way of accessing the data we need?
We can also parse the output of ${coord:dataIn('foo_event')} but that seems
a bit redundant.
I hope I'm missing something.
Thank you for your help!
Cheers,
Lars
Re: Passing variables from dataset to workflow in Coordinator
Posted by Lars Francke <la...@gmail.com>.
Hi Peter,
great, thanks for the answer. I was afraid that'd be the case.
Cheers,
Lars
On Mon, Aug 17, 2020 at 4:14 PM Peter Cseh <ge...@cloudera.com.invalid>
wrote:
> Hey!
>
> Unfortunately there is no way I can recall to do this more easily.
> But yeah, it would make sense to have a function for this.
>
> gp
>
> On Mon, Aug 10, 2020 at 5:27 PM Lars Francke <la...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I have a simple coordinator with a single dataset:
> >
> >
> >
> <uri-template>${hadoop_nameNode}/${coord_stagingFolder}/${YEAR}/${MONTH}/${DAY}</uri-template>
> >
> > I also have a corresponding input-event:
> >
> > <data-in name="foo_event" dataset="foo_ds">
> > <instance>${coord:current(-1)}</instance>
> > </data-in>
> >
> > Now in my workflow I need to pass in the ${YEAR}, MONTH and DAY variables
> > as properties.
> > So far we've always used "fake" <output-events> and basically template
> our
> > variables that way (so used data-out things that don't actually
> correspond
> > to a folder anywhere).
> >
> > What's the correct way of accessing the data we need?
> >
> > We can also parse the output of ${coord:dataIn('foo_event')} but that
> seems
> > a bit redundant.
> > I hope I'm missing something.
> >
> > Thank you for your help!
> >
> > Cheers,
> > Lars
> >
>
>
> --
> *Peter Cseh* | Software Engineer, Cloudera Search
> cloudera.com <https://www.cloudera.com>
> [image: Cloudera] <https://www.cloudera.com/>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> <https://www.cloudera.com/>
> ------------------------------
>
Re: Passing variables from dataset to workflow in Coordinator
Posted by Peter Cseh <ge...@cloudera.com.INVALID>.
Hey!
Unfortunately there is no way I can recall to do this more easily.
But yeah, it would make sense to have a function for this.
gp
On Mon, Aug 10, 2020 at 5:27 PM Lars Francke <la...@gmail.com> wrote:
> Hi,
>
> I have a simple coordinator with a single dataset:
>
>
> <uri-template>${hadoop_nameNode}/${coord_stagingFolder}/${YEAR}/${MONTH}/${DAY}</uri-template>
>
> I also have a corresponding input-event:
>
> <data-in name="foo_event" dataset="foo_ds">
> <instance>${coord:current(-1)}</instance>
> </data-in>
>
> Now in my workflow I need to pass in the ${YEAR}, MONTH and DAY variables
> as properties.
> So far we've always used "fake" <output-events> and basically template our
> variables that way (so used data-out things that don't actually correspond
> to a folder anywhere).
>
> What's the correct way of accessing the data we need?
>
> We can also parse the output of ${coord:dataIn('foo_event')} but that seems
> a bit redundant.
> I hope I'm missing something.
>
> Thank you for your help!
>
> Cheers,
> Lars
>
--
*Peter Cseh* | Software Engineer, Cloudera Search
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------