You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Serega Sheypak <se...@gmail.com> on 2013/07/12 16:26:18 UTC
Create custom EL function
Hi, I need to create custom EL function.
It accepts:
String StringWithDatetime
String Pattern (to parse Date)
It returns:
time in seconds.
Please tell me:
1. Where can I find example?
2. Where do I have to put implementation of this function?
Re: Create custom EL function
Posted by Harsh J <ha...@cloudera.com>.
Just to follow-up, Robert recently posted a blog post on this topic:
http://blog.cloudera.com/blog/2013/09/how-to-write-an-el-function-in-apache-oozie/
On Mon, Jul 15, 2013 at 11:02 PM, Serega Sheypak
<se...@gmail.com> wrote:
> Ok, great. Thanks.
> We would have to implement custom installer to modify oozie-site.xml using
> Cloudera Manager API.
> Thanks!
>
>
> 2013/7/15 Robert Kanter <rk...@cloudera.com>
>
>> Sorry for the confusion. Hopefully this will make it clearer.
>>
>> The following replaces step 3 from my original instructions:
>>
>> If you installed Apache or CDH Tarballs:
>> 3a) Copy your jar file to /where/you/deployed/oozie/libext/
>> 3b) Re-run bin/oozie-setup.sh prepare-war
>>
>> If you installed CDH Packages/Parcels:
>> 3) Copy your jar file to /usr/lib/oozie/libext/ or /var/lib/oozie/
>>
>> All of the other steps from my original instructions apply to either type
>> of installation.
>>
>>
>> - Robert
>>
>>
>>
>> On Mon, Jul 15, 2013 at 10:19 AM, Serega Sheypak
>> <se...@gmail.com>wrote:
>>
>> > Now I'm confused. Do I have repackage war if I use parcels?
>> >
>> >
>> > 2013/7/15 Robert Kanter <rk...@cloudera.com>
>> >
>> > > I believe the method I described for packages should also apply for
>> > > parcels. Yes, you still have to modify oozie-site.xml. The only
>> > > difference in the process between a tarball installation and a CDH
>> > > packages/parcel installation is where to put your jar and to run or not
>> > run
>> > > the oozie-setup.sh script.
>> > >
>> > > - Robert
>> > >
>> > >
>> > > On Mon, Jul 15, 2013 at 9:43 AM, Serega Sheypak <
>> > serega.sheypak@gmail.com
>> > > >wrote:
>> > >
>> > > > We are using parcels.
>> > > > Also you said that I have to modify oozie-site.xml to declare my
>> > > function.
>> > > >
>> > > >
>> > > > 2013/7/15 Robert Kanter <rk...@cloudera.com>
>> > > >
>> > > > > If you're using Cloudera Manager, you've probably installed CDH via
>> > > > > packages? If so, you actually only have to copy your jar file to
>> > > > > /usr/lib/oozie/libext/ or /var/lib/oozie/ (they're the same
>> location,
>> > > one
>> > > > > of them is a symlink but I forget which) and restart Oozie; you
>> > should
>> > > > not
>> > > > > run the oozie-setup.sh command.
>> > > > >
>> > > > > - Robert
>> > > > >
>> > > > >
>> > > > > On Mon, Jul 15, 2013 at 6:46 AM, Serega Sheypak <
>> > > > serega.sheypak@gmail.com
>> > > > > >wrote:
>> > > > >
>> > > > > > Hm... function installation in production is rather tricky.
>> > > > > > 1. I think puppet can drop jar from nexus to ooze folder
>> > > > > > 2. it can stop oozie and run command
>> > > > > > 3. config update wouldn't be so easy in case of Cloudera Manager.
>> > > We'll
>> > > > > > need some special stuff to update config via API (if it's
>> possible)
>> > > > > >
>> > > > > > Thanks, I'm trying to make it work.
>> > > > > >
>> > > > > >
>> > > > > > 2013/7/13 Mona Chitnis <ch...@yahoo-inc.com>
>> > > > > >
>> > > > > > > Also if you feel your custom implementation can benefit others,
>> > > feel
>> > > > > free
>> > > > > > > to contribute to Oozie via creating a patch and uploading to a
>> > > JIRA -
>> > > > > > > https://issues.apache.org/jira/browse/OOZIE
>> > > > > > >
>> > > > > > >
>> > > > > > > On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com>
>> > wrote:
>> > > > > > >
>> > > > > > > >Hi Serega,
>> > > > > > > >
>> > > > > > > >I'm actually working on a tutorial blog post for the Cloudera
>> > > > website
>> > > > > on
>> > > > > > > >how to make an EL function. It's not quite done yet, but
>> here's
>> > > the
>> > > > > > basic
>> > > > > > > >steps you need to do:
>> > > > > > > >
>> > > > > > > >1) Write the function in a new Java class. There's no
>> interface
>> > > or
>> > > > > > class
>> > > > > > > >to inherit from, you just make a new class in whatever package
>> > you
>> > > > > want
>> > > > > > > >(e.g. my.package.AwesomeELFunctions). For what you want, its
>> > > > > signature
>> > > > > > > >would be:
>> > > > > > > > public static int functionName(String
>> > stringWithDatetime,
>> > > > > String
>> > > > > > > >pattern)
>> > > > > > > >If you want to create more than one function, they can all go
>> in
>> > > the
>> > > > > > same
>> > > > > > > >class as well.
>> > > > > > > >
>> > > > > > > >2) Compile your class. You only need Oozie dependencies if
>> > you're
>> > > > > > > >function
>> > > > > > > >requires something from Oozie. You can use javac, maven,
>> > Eclipse,
>> > > > etc
>> > > > > > to
>> > > > > > > >compile it. Make sure to create a jar.
>> > > > > > > >
>> > > > > > > >3) (Shut down Oozie and) copy the jar file to
>> > > > > > > >/where/you/deployed/oozie/libext/ and re-run
>> bin/oozie-setup.sh
>> > > > > > > >prepare-war
>> > > > > > > >
>> > > > > > > >4) In oozie-site.xml, set (or add if it doesn't exist) a
>> > property
>> > > > like
>> > > > > > > >this:
>> > > > > > > ><property>
>> > > > > > > >
>> <name>oozie.service.ELService.ext.functions.workflow</name>
>> > > > > > > > <value>
>> > > > > > > >
>> > functionName=my.package.AwesomeELFunctions#functionName
>> > > > > > > > </value>
>> > > > > > > ></property>
>> > > > > > > >If you have more functions, that property takes a comma
>> > separated
>> > > > list
>> > > > > > of
>> > > > > > > >them. The value on the left side of the equals is the name
>> that
>> > > > you'd
>> > > > > > use
>> > > > > > > >in a workflow and the value on the right side is the name of
>> > your
>> > > > > > function
>> > > > > > > >in your Java code. For the value on the left side, you can
>> > > > optionally
>> > > > > > put
>> > > > > > > >a prefix (you may have seen many built-in functions that start
>> > > with
>> > > > > > "wf:"
>> > > > > > > >for example).
>> > > > > > > >
>> > > > > > > >5) Start up Oozie. You should now be able to use the
>> > functionName
>> > > > > > custom
>> > > > > > > >EL Function you created!
>> > > > > > > >
>> > > > > > > >I believe this may only make the function available in
>> > workflows,
>> > > > but
>> > > > > > I'm
>> > > > > > > >not sure; if not, there should be a similar property in
>> > oozie-site
>> > > > you
>> > > > > > can
>> > > > > > > >set for coordinators if you need that.
>> > > > > > > >
>> > > > > > > >Once my proper tutorial blog post is posted, I'll add a link
>> to
>> > > this
>> > > > > > > >thread, but it may be a while.
>> > > > > > > >
>> > > > > > > >- Robert
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
>> > > > > > > ><se...@gmail.com>wrote:
>> > > > > > > >
>> > > > > > > >> Hi, I need to create custom EL function.
>> > > > > > > >> It accepts:
>> > > > > > > >> String StringWithDatetime
>> > > > > > > >> String Pattern (to parse Date)
>> > > > > > > >> It returns:
>> > > > > > > >> time in seconds.
>> > > > > > > >>
>> > > > > > > >> Please tell me:
>> > > > > > > >> 1. Where can I find example?
>> > > > > > > >> 2. Where do I have to put implementation of this function?
>> > > > > > > >>
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
--
Harsh J
Re: Create custom EL function
Posted by Serega Sheypak <se...@gmail.com>.
Ok, great. Thanks.
We would have to implement custom installer to modify oozie-site.xml using
Cloudera Manager API.
Thanks!
2013/7/15 Robert Kanter <rk...@cloudera.com>
> Sorry for the confusion. Hopefully this will make it clearer.
>
> The following replaces step 3 from my original instructions:
>
> If you installed Apache or CDH Tarballs:
> 3a) Copy your jar file to /where/you/deployed/oozie/libext/
> 3b) Re-run bin/oozie-setup.sh prepare-war
>
> If you installed CDH Packages/Parcels:
> 3) Copy your jar file to /usr/lib/oozie/libext/ or /var/lib/oozie/
>
> All of the other steps from my original instructions apply to either type
> of installation.
>
>
> - Robert
>
>
>
> On Mon, Jul 15, 2013 at 10:19 AM, Serega Sheypak
> <se...@gmail.com>wrote:
>
> > Now I'm confused. Do I have repackage war if I use parcels?
> >
> >
> > 2013/7/15 Robert Kanter <rk...@cloudera.com>
> >
> > > I believe the method I described for packages should also apply for
> > > parcels. Yes, you still have to modify oozie-site.xml. The only
> > > difference in the process between a tarball installation and a CDH
> > > packages/parcel installation is where to put your jar and to run or not
> > run
> > > the oozie-setup.sh script.
> > >
> > > - Robert
> > >
> > >
> > > On Mon, Jul 15, 2013 at 9:43 AM, Serega Sheypak <
> > serega.sheypak@gmail.com
> > > >wrote:
> > >
> > > > We are using parcels.
> > > > Also you said that I have to modify oozie-site.xml to declare my
> > > function.
> > > >
> > > >
> > > > 2013/7/15 Robert Kanter <rk...@cloudera.com>
> > > >
> > > > > If you're using Cloudera Manager, you've probably installed CDH via
> > > > > packages? If so, you actually only have to copy your jar file to
> > > > > /usr/lib/oozie/libext/ or /var/lib/oozie/ (they're the same
> location,
> > > one
> > > > > of them is a symlink but I forget which) and restart Oozie; you
> > should
> > > > not
> > > > > run the oozie-setup.sh command.
> > > > >
> > > > > - Robert
> > > > >
> > > > >
> > > > > On Mon, Jul 15, 2013 at 6:46 AM, Serega Sheypak <
> > > > serega.sheypak@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hm... function installation in production is rather tricky.
> > > > > > 1. I think puppet can drop jar from nexus to ooze folder
> > > > > > 2. it can stop oozie and run command
> > > > > > 3. config update wouldn't be so easy in case of Cloudera Manager.
> > > We'll
> > > > > > need some special stuff to update config via API (if it's
> possible)
> > > > > >
> > > > > > Thanks, I'm trying to make it work.
> > > > > >
> > > > > >
> > > > > > 2013/7/13 Mona Chitnis <ch...@yahoo-inc.com>
> > > > > >
> > > > > > > Also if you feel your custom implementation can benefit others,
> > > feel
> > > > > free
> > > > > > > to contribute to Oozie via creating a patch and uploading to a
> > > JIRA -
> > > > > > > https://issues.apache.org/jira/browse/OOZIE
> > > > > > >
> > > > > > >
> > > > > > > On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com>
> > wrote:
> > > > > > >
> > > > > > > >Hi Serega,
> > > > > > > >
> > > > > > > >I'm actually working on a tutorial blog post for the Cloudera
> > > > website
> > > > > on
> > > > > > > >how to make an EL function. It's not quite done yet, but
> here's
> > > the
> > > > > > basic
> > > > > > > >steps you need to do:
> > > > > > > >
> > > > > > > >1) Write the function in a new Java class. There's no
> interface
> > > or
> > > > > > class
> > > > > > > >to inherit from, you just make a new class in whatever package
> > you
> > > > > want
> > > > > > > >(e.g. my.package.AwesomeELFunctions). For what you want, its
> > > > > signature
> > > > > > > >would be:
> > > > > > > > public static int functionName(String
> > stringWithDatetime,
> > > > > String
> > > > > > > >pattern)
> > > > > > > >If you want to create more than one function, they can all go
> in
> > > the
> > > > > > same
> > > > > > > >class as well.
> > > > > > > >
> > > > > > > >2) Compile your class. You only need Oozie dependencies if
> > you're
> > > > > > > >function
> > > > > > > >requires something from Oozie. You can use javac, maven,
> > Eclipse,
> > > > etc
> > > > > > to
> > > > > > > >compile it. Make sure to create a jar.
> > > > > > > >
> > > > > > > >3) (Shut down Oozie and) copy the jar file to
> > > > > > > >/where/you/deployed/oozie/libext/ and re-run
> bin/oozie-setup.sh
> > > > > > > >prepare-war
> > > > > > > >
> > > > > > > >4) In oozie-site.xml, set (or add if it doesn't exist) a
> > property
> > > > like
> > > > > > > >this:
> > > > > > > ><property>
> > > > > > > >
> <name>oozie.service.ELService.ext.functions.workflow</name>
> > > > > > > > <value>
> > > > > > > >
> > functionName=my.package.AwesomeELFunctions#functionName
> > > > > > > > </value>
> > > > > > > ></property>
> > > > > > > >If you have more functions, that property takes a comma
> > separated
> > > > list
> > > > > > of
> > > > > > > >them. The value on the left side of the equals is the name
> that
> > > > you'd
> > > > > > use
> > > > > > > >in a workflow and the value on the right side is the name of
> > your
> > > > > > function
> > > > > > > >in your Java code. For the value on the left side, you can
> > > > optionally
> > > > > > put
> > > > > > > >a prefix (you may have seen many built-in functions that start
> > > with
> > > > > > "wf:"
> > > > > > > >for example).
> > > > > > > >
> > > > > > > >5) Start up Oozie. You should now be able to use the
> > functionName
> > > > > > custom
> > > > > > > >EL Function you created!
> > > > > > > >
> > > > > > > >I believe this may only make the function available in
> > workflows,
> > > > but
> > > > > > I'm
> > > > > > > >not sure; if not, there should be a similar property in
> > oozie-site
> > > > you
> > > > > > can
> > > > > > > >set for coordinators if you need that.
> > > > > > > >
> > > > > > > >Once my proper tutorial blog post is posted, I'll add a link
> to
> > > this
> > > > > > > >thread, but it may be a while.
> > > > > > > >
> > > > > > > >- Robert
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
> > > > > > > ><se...@gmail.com>wrote:
> > > > > > > >
> > > > > > > >> Hi, I need to create custom EL function.
> > > > > > > >> It accepts:
> > > > > > > >> String StringWithDatetime
> > > > > > > >> String Pattern (to parse Date)
> > > > > > > >> It returns:
> > > > > > > >> time in seconds.
> > > > > > > >>
> > > > > > > >> Please tell me:
> > > > > > > >> 1. Where can I find example?
> > > > > > > >> 2. Where do I have to put implementation of this function?
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Create custom EL function
Posted by Robert Kanter <rk...@cloudera.com>.
Sorry for the confusion. Hopefully this will make it clearer.
The following replaces step 3 from my original instructions:
If you installed Apache or CDH Tarballs:
3a) Copy your jar file to /where/you/deployed/oozie/libext/
3b) Re-run bin/oozie-setup.sh prepare-war
If you installed CDH Packages/Parcels:
3) Copy your jar file to /usr/lib/oozie/libext/ or /var/lib/oozie/
All of the other steps from my original instructions apply to either type
of installation.
- Robert
On Mon, Jul 15, 2013 at 10:19 AM, Serega Sheypak
<se...@gmail.com>wrote:
> Now I'm confused. Do I have repackage war if I use parcels?
>
>
> 2013/7/15 Robert Kanter <rk...@cloudera.com>
>
> > I believe the method I described for packages should also apply for
> > parcels. Yes, you still have to modify oozie-site.xml. The only
> > difference in the process between a tarball installation and a CDH
> > packages/parcel installation is where to put your jar and to run or not
> run
> > the oozie-setup.sh script.
> >
> > - Robert
> >
> >
> > On Mon, Jul 15, 2013 at 9:43 AM, Serega Sheypak <
> serega.sheypak@gmail.com
> > >wrote:
> >
> > > We are using parcels.
> > > Also you said that I have to modify oozie-site.xml to declare my
> > function.
> > >
> > >
> > > 2013/7/15 Robert Kanter <rk...@cloudera.com>
> > >
> > > > If you're using Cloudera Manager, you've probably installed CDH via
> > > > packages? If so, you actually only have to copy your jar file to
> > > > /usr/lib/oozie/libext/ or /var/lib/oozie/ (they're the same location,
> > one
> > > > of them is a symlink but I forget which) and restart Oozie; you
> should
> > > not
> > > > run the oozie-setup.sh command.
> > > >
> > > > - Robert
> > > >
> > > >
> > > > On Mon, Jul 15, 2013 at 6:46 AM, Serega Sheypak <
> > > serega.sheypak@gmail.com
> > > > >wrote:
> > > >
> > > > > Hm... function installation in production is rather tricky.
> > > > > 1. I think puppet can drop jar from nexus to ooze folder
> > > > > 2. it can stop oozie and run command
> > > > > 3. config update wouldn't be so easy in case of Cloudera Manager.
> > We'll
> > > > > need some special stuff to update config via API (if it's possible)
> > > > >
> > > > > Thanks, I'm trying to make it work.
> > > > >
> > > > >
> > > > > 2013/7/13 Mona Chitnis <ch...@yahoo-inc.com>
> > > > >
> > > > > > Also if you feel your custom implementation can benefit others,
> > feel
> > > > free
> > > > > > to contribute to Oozie via creating a patch and uploading to a
> > JIRA -
> > > > > > https://issues.apache.org/jira/browse/OOZIE
> > > > > >
> > > > > >
> > > > > > On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com>
> wrote:
> > > > > >
> > > > > > >Hi Serega,
> > > > > > >
> > > > > > >I'm actually working on a tutorial blog post for the Cloudera
> > > website
> > > > on
> > > > > > >how to make an EL function. It's not quite done yet, but here's
> > the
> > > > > basic
> > > > > > >steps you need to do:
> > > > > > >
> > > > > > >1) Write the function in a new Java class. There's no interface
> > or
> > > > > class
> > > > > > >to inherit from, you just make a new class in whatever package
> you
> > > > want
> > > > > > >(e.g. my.package.AwesomeELFunctions). For what you want, its
> > > > signature
> > > > > > >would be:
> > > > > > > public static int functionName(String
> stringWithDatetime,
> > > > String
> > > > > > >pattern)
> > > > > > >If you want to create more than one function, they can all go in
> > the
> > > > > same
> > > > > > >class as well.
> > > > > > >
> > > > > > >2) Compile your class. You only need Oozie dependencies if
> you're
> > > > > > >function
> > > > > > >requires something from Oozie. You can use javac, maven,
> Eclipse,
> > > etc
> > > > > to
> > > > > > >compile it. Make sure to create a jar.
> > > > > > >
> > > > > > >3) (Shut down Oozie and) copy the jar file to
> > > > > > >/where/you/deployed/oozie/libext/ and re-run bin/oozie-setup.sh
> > > > > > >prepare-war
> > > > > > >
> > > > > > >4) In oozie-site.xml, set (or add if it doesn't exist) a
> property
> > > like
> > > > > > >this:
> > > > > > ><property>
> > > > > > > <name>oozie.service.ELService.ext.functions.workflow</name>
> > > > > > > <value>
> > > > > > >
> functionName=my.package.AwesomeELFunctions#functionName
> > > > > > > </value>
> > > > > > ></property>
> > > > > > >If you have more functions, that property takes a comma
> separated
> > > list
> > > > > of
> > > > > > >them. The value on the left side of the equals is the name that
> > > you'd
> > > > > use
> > > > > > >in a workflow and the value on the right side is the name of
> your
> > > > > function
> > > > > > >in your Java code. For the value on the left side, you can
> > > optionally
> > > > > put
> > > > > > >a prefix (you may have seen many built-in functions that start
> > with
> > > > > "wf:"
> > > > > > >for example).
> > > > > > >
> > > > > > >5) Start up Oozie. You should now be able to use the
> functionName
> > > > > custom
> > > > > > >EL Function you created!
> > > > > > >
> > > > > > >I believe this may only make the function available in
> workflows,
> > > but
> > > > > I'm
> > > > > > >not sure; if not, there should be a similar property in
> oozie-site
> > > you
> > > > > can
> > > > > > >set for coordinators if you need that.
> > > > > > >
> > > > > > >Once my proper tutorial blog post is posted, I'll add a link to
> > this
> > > > > > >thread, but it may be a while.
> > > > > > >
> > > > > > >- Robert
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
> > > > > > ><se...@gmail.com>wrote:
> > > > > > >
> > > > > > >> Hi, I need to create custom EL function.
> > > > > > >> It accepts:
> > > > > > >> String StringWithDatetime
> > > > > > >> String Pattern (to parse Date)
> > > > > > >> It returns:
> > > > > > >> time in seconds.
> > > > > > >>
> > > > > > >> Please tell me:
> > > > > > >> 1. Where can I find example?
> > > > > > >> 2. Where do I have to put implementation of this function?
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Create custom EL function
Posted by Serega Sheypak <se...@gmail.com>.
Now I'm confused. Do I have repackage war if I use parcels?
2013/7/15 Robert Kanter <rk...@cloudera.com>
> I believe the method I described for packages should also apply for
> parcels. Yes, you still have to modify oozie-site.xml. The only
> difference in the process between a tarball installation and a CDH
> packages/parcel installation is where to put your jar and to run or not run
> the oozie-setup.sh script.
>
> - Robert
>
>
> On Mon, Jul 15, 2013 at 9:43 AM, Serega Sheypak <serega.sheypak@gmail.com
> >wrote:
>
> > We are using parcels.
> > Also you said that I have to modify oozie-site.xml to declare my
> function.
> >
> >
> > 2013/7/15 Robert Kanter <rk...@cloudera.com>
> >
> > > If you're using Cloudera Manager, you've probably installed CDH via
> > > packages? If so, you actually only have to copy your jar file to
> > > /usr/lib/oozie/libext/ or /var/lib/oozie/ (they're the same location,
> one
> > > of them is a symlink but I forget which) and restart Oozie; you should
> > not
> > > run the oozie-setup.sh command.
> > >
> > > - Robert
> > >
> > >
> > > On Mon, Jul 15, 2013 at 6:46 AM, Serega Sheypak <
> > serega.sheypak@gmail.com
> > > >wrote:
> > >
> > > > Hm... function installation in production is rather tricky.
> > > > 1. I think puppet can drop jar from nexus to ooze folder
> > > > 2. it can stop oozie and run command
> > > > 3. config update wouldn't be so easy in case of Cloudera Manager.
> We'll
> > > > need some special stuff to update config via API (if it's possible)
> > > >
> > > > Thanks, I'm trying to make it work.
> > > >
> > > >
> > > > 2013/7/13 Mona Chitnis <ch...@yahoo-inc.com>
> > > >
> > > > > Also if you feel your custom implementation can benefit others,
> feel
> > > free
> > > > > to contribute to Oozie via creating a patch and uploading to a
> JIRA -
> > > > > https://issues.apache.org/jira/browse/OOZIE
> > > > >
> > > > >
> > > > > On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com> wrote:
> > > > >
> > > > > >Hi Serega,
> > > > > >
> > > > > >I'm actually working on a tutorial blog post for the Cloudera
> > website
> > > on
> > > > > >how to make an EL function. It's not quite done yet, but here's
> the
> > > > basic
> > > > > >steps you need to do:
> > > > > >
> > > > > >1) Write the function in a new Java class. There's no interface
> or
> > > > class
> > > > > >to inherit from, you just make a new class in whatever package you
> > > want
> > > > > >(e.g. my.package.AwesomeELFunctions). For what you want, its
> > > signature
> > > > > >would be:
> > > > > > public static int functionName(String stringWithDatetime,
> > > String
> > > > > >pattern)
> > > > > >If you want to create more than one function, they can all go in
> the
> > > > same
> > > > > >class as well.
> > > > > >
> > > > > >2) Compile your class. You only need Oozie dependencies if you're
> > > > > >function
> > > > > >requires something from Oozie. You can use javac, maven, Eclipse,
> > etc
> > > > to
> > > > > >compile it. Make sure to create a jar.
> > > > > >
> > > > > >3) (Shut down Oozie and) copy the jar file to
> > > > > >/where/you/deployed/oozie/libext/ and re-run bin/oozie-setup.sh
> > > > > >prepare-war
> > > > > >
> > > > > >4) In oozie-site.xml, set (or add if it doesn't exist) a property
> > like
> > > > > >this:
> > > > > ><property>
> > > > > > <name>oozie.service.ELService.ext.functions.workflow</name>
> > > > > > <value>
> > > > > > functionName=my.package.AwesomeELFunctions#functionName
> > > > > > </value>
> > > > > ></property>
> > > > > >If you have more functions, that property takes a comma separated
> > list
> > > > of
> > > > > >them. The value on the left side of the equals is the name that
> > you'd
> > > > use
> > > > > >in a workflow and the value on the right side is the name of your
> > > > function
> > > > > >in your Java code. For the value on the left side, you can
> > optionally
> > > > put
> > > > > >a prefix (you may have seen many built-in functions that start
> with
> > > > "wf:"
> > > > > >for example).
> > > > > >
> > > > > >5) Start up Oozie. You should now be able to use the functionName
> > > > custom
> > > > > >EL Function you created!
> > > > > >
> > > > > >I believe this may only make the function available in workflows,
> > but
> > > > I'm
> > > > > >not sure; if not, there should be a similar property in oozie-site
> > you
> > > > can
> > > > > >set for coordinators if you need that.
> > > > > >
> > > > > >Once my proper tutorial blog post is posted, I'll add a link to
> this
> > > > > >thread, but it may be a while.
> > > > > >
> > > > > >- Robert
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
> > > > > ><se...@gmail.com>wrote:
> > > > > >
> > > > > >> Hi, I need to create custom EL function.
> > > > > >> It accepts:
> > > > > >> String StringWithDatetime
> > > > > >> String Pattern (to parse Date)
> > > > > >> It returns:
> > > > > >> time in seconds.
> > > > > >>
> > > > > >> Please tell me:
> > > > > >> 1. Where can I find example?
> > > > > >> 2. Where do I have to put implementation of this function?
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Create custom EL function
Posted by Robert Kanter <rk...@cloudera.com>.
I believe the method I described for packages should also apply for
parcels. Yes, you still have to modify oozie-site.xml. The only
difference in the process between a tarball installation and a CDH
packages/parcel installation is where to put your jar and to run or not run
the oozie-setup.sh script.
- Robert
On Mon, Jul 15, 2013 at 9:43 AM, Serega Sheypak <se...@gmail.com>wrote:
> We are using parcels.
> Also you said that I have to modify oozie-site.xml to declare my function.
>
>
> 2013/7/15 Robert Kanter <rk...@cloudera.com>
>
> > If you're using Cloudera Manager, you've probably installed CDH via
> > packages? If so, you actually only have to copy your jar file to
> > /usr/lib/oozie/libext/ or /var/lib/oozie/ (they're the same location, one
> > of them is a symlink but I forget which) and restart Oozie; you should
> not
> > run the oozie-setup.sh command.
> >
> > - Robert
> >
> >
> > On Mon, Jul 15, 2013 at 6:46 AM, Serega Sheypak <
> serega.sheypak@gmail.com
> > >wrote:
> >
> > > Hm... function installation in production is rather tricky.
> > > 1. I think puppet can drop jar from nexus to ooze folder
> > > 2. it can stop oozie and run command
> > > 3. config update wouldn't be so easy in case of Cloudera Manager. We'll
> > > need some special stuff to update config via API (if it's possible)
> > >
> > > Thanks, I'm trying to make it work.
> > >
> > >
> > > 2013/7/13 Mona Chitnis <ch...@yahoo-inc.com>
> > >
> > > > Also if you feel your custom implementation can benefit others, feel
> > free
> > > > to contribute to Oozie via creating a patch and uploading to a JIRA -
> > > > https://issues.apache.org/jira/browse/OOZIE
> > > >
> > > >
> > > > On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com> wrote:
> > > >
> > > > >Hi Serega,
> > > > >
> > > > >I'm actually working on a tutorial blog post for the Cloudera
> website
> > on
> > > > >how to make an EL function. It's not quite done yet, but here's the
> > > basic
> > > > >steps you need to do:
> > > > >
> > > > >1) Write the function in a new Java class. There's no interface or
> > > class
> > > > >to inherit from, you just make a new class in whatever package you
> > want
> > > > >(e.g. my.package.AwesomeELFunctions). For what you want, its
> > signature
> > > > >would be:
> > > > > public static int functionName(String stringWithDatetime,
> > String
> > > > >pattern)
> > > > >If you want to create more than one function, they can all go in the
> > > same
> > > > >class as well.
> > > > >
> > > > >2) Compile your class. You only need Oozie dependencies if you're
> > > > >function
> > > > >requires something from Oozie. You can use javac, maven, Eclipse,
> etc
> > > to
> > > > >compile it. Make sure to create a jar.
> > > > >
> > > > >3) (Shut down Oozie and) copy the jar file to
> > > > >/where/you/deployed/oozie/libext/ and re-run bin/oozie-setup.sh
> > > > >prepare-war
> > > > >
> > > > >4) In oozie-site.xml, set (or add if it doesn't exist) a property
> like
> > > > >this:
> > > > ><property>
> > > > > <name>oozie.service.ELService.ext.functions.workflow</name>
> > > > > <value>
> > > > > functionName=my.package.AwesomeELFunctions#functionName
> > > > > </value>
> > > > ></property>
> > > > >If you have more functions, that property takes a comma separated
> list
> > > of
> > > > >them. The value on the left side of the equals is the name that
> you'd
> > > use
> > > > >in a workflow and the value on the right side is the name of your
> > > function
> > > > >in your Java code. For the value on the left side, you can
> optionally
> > > put
> > > > >a prefix (you may have seen many built-in functions that start with
> > > "wf:"
> > > > >for example).
> > > > >
> > > > >5) Start up Oozie. You should now be able to use the functionName
> > > custom
> > > > >EL Function you created!
> > > > >
> > > > >I believe this may only make the function available in workflows,
> but
> > > I'm
> > > > >not sure; if not, there should be a similar property in oozie-site
> you
> > > can
> > > > >set for coordinators if you need that.
> > > > >
> > > > >Once my proper tutorial blog post is posted, I'll add a link to this
> > > > >thread, but it may be a while.
> > > > >
> > > > >- Robert
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
> > > > ><se...@gmail.com>wrote:
> > > > >
> > > > >> Hi, I need to create custom EL function.
> > > > >> It accepts:
> > > > >> String StringWithDatetime
> > > > >> String Pattern (to parse Date)
> > > > >> It returns:
> > > > >> time in seconds.
> > > > >>
> > > > >> Please tell me:
> > > > >> 1. Where can I find example?
> > > > >> 2. Where do I have to put implementation of this function?
> > > > >>
> > > >
> > > >
> > >
> >
>
Re: Create custom EL function
Posted by Serega Sheypak <se...@gmail.com>.
We are using parcels.
Also you said that I have to modify oozie-site.xml to declare my function.
2013/7/15 Robert Kanter <rk...@cloudera.com>
> If you're using Cloudera Manager, you've probably installed CDH via
> packages? If so, you actually only have to copy your jar file to
> /usr/lib/oozie/libext/ or /var/lib/oozie/ (they're the same location, one
> of them is a symlink but I forget which) and restart Oozie; you should not
> run the oozie-setup.sh command.
>
> - Robert
>
>
> On Mon, Jul 15, 2013 at 6:46 AM, Serega Sheypak <serega.sheypak@gmail.com
> >wrote:
>
> > Hm... function installation in production is rather tricky.
> > 1. I think puppet can drop jar from nexus to ooze folder
> > 2. it can stop oozie and run command
> > 3. config update wouldn't be so easy in case of Cloudera Manager. We'll
> > need some special stuff to update config via API (if it's possible)
> >
> > Thanks, I'm trying to make it work.
> >
> >
> > 2013/7/13 Mona Chitnis <ch...@yahoo-inc.com>
> >
> > > Also if you feel your custom implementation can benefit others, feel
> free
> > > to contribute to Oozie via creating a patch and uploading to a JIRA -
> > > https://issues.apache.org/jira/browse/OOZIE
> > >
> > >
> > > On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com> wrote:
> > >
> > > >Hi Serega,
> > > >
> > > >I'm actually working on a tutorial blog post for the Cloudera website
> on
> > > >how to make an EL function. It's not quite done yet, but here's the
> > basic
> > > >steps you need to do:
> > > >
> > > >1) Write the function in a new Java class. There's no interface or
> > class
> > > >to inherit from, you just make a new class in whatever package you
> want
> > > >(e.g. my.package.AwesomeELFunctions). For what you want, its
> signature
> > > >would be:
> > > > public static int functionName(String stringWithDatetime,
> String
> > > >pattern)
> > > >If you want to create more than one function, they can all go in the
> > same
> > > >class as well.
> > > >
> > > >2) Compile your class. You only need Oozie dependencies if you're
> > > >function
> > > >requires something from Oozie. You can use javac, maven, Eclipse, etc
> > to
> > > >compile it. Make sure to create a jar.
> > > >
> > > >3) (Shut down Oozie and) copy the jar file to
> > > >/where/you/deployed/oozie/libext/ and re-run bin/oozie-setup.sh
> > > >prepare-war
> > > >
> > > >4) In oozie-site.xml, set (or add if it doesn't exist) a property like
> > > >this:
> > > ><property>
> > > > <name>oozie.service.ELService.ext.functions.workflow</name>
> > > > <value>
> > > > functionName=my.package.AwesomeELFunctions#functionName
> > > > </value>
> > > ></property>
> > > >If you have more functions, that property takes a comma separated list
> > of
> > > >them. The value on the left side of the equals is the name that you'd
> > use
> > > >in a workflow and the value on the right side is the name of your
> > function
> > > >in your Java code. For the value on the left side, you can optionally
> > put
> > > >a prefix (you may have seen many built-in functions that start with
> > "wf:"
> > > >for example).
> > > >
> > > >5) Start up Oozie. You should now be able to use the functionName
> > custom
> > > >EL Function you created!
> > > >
> > > >I believe this may only make the function available in workflows, but
> > I'm
> > > >not sure; if not, there should be a similar property in oozie-site you
> > can
> > > >set for coordinators if you need that.
> > > >
> > > >Once my proper tutorial blog post is posted, I'll add a link to this
> > > >thread, but it may be a while.
> > > >
> > > >- Robert
> > > >
> > > >
> > > >
> > > >
> > > >On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
> > > ><se...@gmail.com>wrote:
> > > >
> > > >> Hi, I need to create custom EL function.
> > > >> It accepts:
> > > >> String StringWithDatetime
> > > >> String Pattern (to parse Date)
> > > >> It returns:
> > > >> time in seconds.
> > > >>
> > > >> Please tell me:
> > > >> 1. Where can I find example?
> > > >> 2. Where do I have to put implementation of this function?
> > > >>
> > >
> > >
> >
>
Re: Create custom EL function
Posted by Robert Kanter <rk...@cloudera.com>.
If you're using Cloudera Manager, you've probably installed CDH via
packages? If so, you actually only have to copy your jar file to
/usr/lib/oozie/libext/ or /var/lib/oozie/ (they're the same location, one
of them is a symlink but I forget which) and restart Oozie; you should not
run the oozie-setup.sh command.
- Robert
On Mon, Jul 15, 2013 at 6:46 AM, Serega Sheypak <se...@gmail.com>wrote:
> Hm... function installation in production is rather tricky.
> 1. I think puppet can drop jar from nexus to ooze folder
> 2. it can stop oozie and run command
> 3. config update wouldn't be so easy in case of Cloudera Manager. We'll
> need some special stuff to update config via API (if it's possible)
>
> Thanks, I'm trying to make it work.
>
>
> 2013/7/13 Mona Chitnis <ch...@yahoo-inc.com>
>
> > Also if you feel your custom implementation can benefit others, feel free
> > to contribute to Oozie via creating a patch and uploading to a JIRA -
> > https://issues.apache.org/jira/browse/OOZIE
> >
> >
> > On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com> wrote:
> >
> > >Hi Serega,
> > >
> > >I'm actually working on a tutorial blog post for the Cloudera website on
> > >how to make an EL function. It's not quite done yet, but here's the
> basic
> > >steps you need to do:
> > >
> > >1) Write the function in a new Java class. There's no interface or
> class
> > >to inherit from, you just make a new class in whatever package you want
> > >(e.g. my.package.AwesomeELFunctions). For what you want, its signature
> > >would be:
> > > public static int functionName(String stringWithDatetime, String
> > >pattern)
> > >If you want to create more than one function, they can all go in the
> same
> > >class as well.
> > >
> > >2) Compile your class. You only need Oozie dependencies if you're
> > >function
> > >requires something from Oozie. You can use javac, maven, Eclipse, etc
> to
> > >compile it. Make sure to create a jar.
> > >
> > >3) (Shut down Oozie and) copy the jar file to
> > >/where/you/deployed/oozie/libext/ and re-run bin/oozie-setup.sh
> > >prepare-war
> > >
> > >4) In oozie-site.xml, set (or add if it doesn't exist) a property like
> > >this:
> > ><property>
> > > <name>oozie.service.ELService.ext.functions.workflow</name>
> > > <value>
> > > functionName=my.package.AwesomeELFunctions#functionName
> > > </value>
> > ></property>
> > >If you have more functions, that property takes a comma separated list
> of
> > >them. The value on the left side of the equals is the name that you'd
> use
> > >in a workflow and the value on the right side is the name of your
> function
> > >in your Java code. For the value on the left side, you can optionally
> put
> > >a prefix (you may have seen many built-in functions that start with
> "wf:"
> > >for example).
> > >
> > >5) Start up Oozie. You should now be able to use the functionName
> custom
> > >EL Function you created!
> > >
> > >I believe this may only make the function available in workflows, but
> I'm
> > >not sure; if not, there should be a similar property in oozie-site you
> can
> > >set for coordinators if you need that.
> > >
> > >Once my proper tutorial blog post is posted, I'll add a link to this
> > >thread, but it may be a while.
> > >
> > >- Robert
> > >
> > >
> > >
> > >
> > >On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
> > ><se...@gmail.com>wrote:
> > >
> > >> Hi, I need to create custom EL function.
> > >> It accepts:
> > >> String StringWithDatetime
> > >> String Pattern (to parse Date)
> > >> It returns:
> > >> time in seconds.
> > >>
> > >> Please tell me:
> > >> 1. Where can I find example?
> > >> 2. Where do I have to put implementation of this function?
> > >>
> >
> >
>
Re: Create custom EL function
Posted by Serega Sheypak <se...@gmail.com>.
Hm... function installation in production is rather tricky.
1. I think puppet can drop jar from nexus to ooze folder
2. it can stop oozie and run command
3. config update wouldn't be so easy in case of Cloudera Manager. We'll
need some special stuff to update config via API (if it's possible)
Thanks, I'm trying to make it work.
2013/7/13 Mona Chitnis <ch...@yahoo-inc.com>
> Also if you feel your custom implementation can benefit others, feel free
> to contribute to Oozie via creating a patch and uploading to a JIRA -
> https://issues.apache.org/jira/browse/OOZIE
>
>
> On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com> wrote:
>
> >Hi Serega,
> >
> >I'm actually working on a tutorial blog post for the Cloudera website on
> >how to make an EL function. It's not quite done yet, but here's the basic
> >steps you need to do:
> >
> >1) Write the function in a new Java class. There's no interface or class
> >to inherit from, you just make a new class in whatever package you want
> >(e.g. my.package.AwesomeELFunctions). For what you want, its signature
> >would be:
> > public static int functionName(String stringWithDatetime, String
> >pattern)
> >If you want to create more than one function, they can all go in the same
> >class as well.
> >
> >2) Compile your class. You only need Oozie dependencies if you're
> >function
> >requires something from Oozie. You can use javac, maven, Eclipse, etc to
> >compile it. Make sure to create a jar.
> >
> >3) (Shut down Oozie and) copy the jar file to
> >/where/you/deployed/oozie/libext/ and re-run bin/oozie-setup.sh
> >prepare-war
> >
> >4) In oozie-site.xml, set (or add if it doesn't exist) a property like
> >this:
> ><property>
> > <name>oozie.service.ELService.ext.functions.workflow</name>
> > <value>
> > functionName=my.package.AwesomeELFunctions#functionName
> > </value>
> ></property>
> >If you have more functions, that property takes a comma separated list of
> >them. The value on the left side of the equals is the name that you'd use
> >in a workflow and the value on the right side is the name of your function
> >in your Java code. For the value on the left side, you can optionally put
> >a prefix (you may have seen many built-in functions that start with "wf:"
> >for example).
> >
> >5) Start up Oozie. You should now be able to use the functionName custom
> >EL Function you created!
> >
> >I believe this may only make the function available in workflows, but I'm
> >not sure; if not, there should be a similar property in oozie-site you can
> >set for coordinators if you need that.
> >
> >Once my proper tutorial blog post is posted, I'll add a link to this
> >thread, but it may be a while.
> >
> >- Robert
> >
> >
> >
> >
> >On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
> ><se...@gmail.com>wrote:
> >
> >> Hi, I need to create custom EL function.
> >> It accepts:
> >> String StringWithDatetime
> >> String Pattern (to parse Date)
> >> It returns:
> >> time in seconds.
> >>
> >> Please tell me:
> >> 1. Where can I find example?
> >> 2. Where do I have to put implementation of this function?
> >>
>
>
Re: Create custom EL function
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
Also if you feel your custom implementation can benefit others, feel free
to contribute to Oozie via creating a patch and uploading to a JIRA -
https://issues.apache.org/jira/browse/OOZIE
On 7/12/13 11:12 AM, "Robert Kanter" <rk...@cloudera.com> wrote:
>Hi Serega,
>
>I'm actually working on a tutorial blog post for the Cloudera website on
>how to make an EL function. It's not quite done yet, but here's the basic
>steps you need to do:
>
>1) Write the function in a new Java class. There's no interface or class
>to inherit from, you just make a new class in whatever package you want
>(e.g. my.package.AwesomeELFunctions). For what you want, its signature
>would be:
> public static int functionName(String stringWithDatetime, String
>pattern)
>If you want to create more than one function, they can all go in the same
>class as well.
>
>2) Compile your class. You only need Oozie dependencies if you're
>function
>requires something from Oozie. You can use javac, maven, Eclipse, etc to
>compile it. Make sure to create a jar.
>
>3) (Shut down Oozie and) copy the jar file to
>/where/you/deployed/oozie/libext/ and re-run bin/oozie-setup.sh
>prepare-war
>
>4) In oozie-site.xml, set (or add if it doesn't exist) a property like
>this:
><property>
> <name>oozie.service.ELService.ext.functions.workflow</name>
> <value>
> functionName=my.package.AwesomeELFunctions#functionName
> </value>
></property>
>If you have more functions, that property takes a comma separated list of
>them. The value on the left side of the equals is the name that you'd use
>in a workflow and the value on the right side is the name of your function
>in your Java code. For the value on the left side, you can optionally put
>a prefix (you may have seen many built-in functions that start with "wf:"
>for example).
>
>5) Start up Oozie. You should now be able to use the functionName custom
>EL Function you created!
>
>I believe this may only make the function available in workflows, but I'm
>not sure; if not, there should be a similar property in oozie-site you can
>set for coordinators if you need that.
>
>Once my proper tutorial blog post is posted, I'll add a link to this
>thread, but it may be a while.
>
>- Robert
>
>
>
>
>On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak
><se...@gmail.com>wrote:
>
>> Hi, I need to create custom EL function.
>> It accepts:
>> String StringWithDatetime
>> String Pattern (to parse Date)
>> It returns:
>> time in seconds.
>>
>> Please tell me:
>> 1. Where can I find example?
>> 2. Where do I have to put implementation of this function?
>>
Re: Create custom EL function
Posted by Robert Kanter <rk...@cloudera.com>.
Hi Serega,
I'm actually working on a tutorial blog post for the Cloudera website on
how to make an EL function. It's not quite done yet, but here's the basic
steps you need to do:
1) Write the function in a new Java class. There's no interface or class
to inherit from, you just make a new class in whatever package you want
(e.g. my.package.AwesomeELFunctions). For what you want, its signature
would be:
public static int functionName(String stringWithDatetime, String
pattern)
If you want to create more than one function, they can all go in the same
class as well.
2) Compile your class. You only need Oozie dependencies if you're function
requires something from Oozie. You can use javac, maven, Eclipse, etc to
compile it. Make sure to create a jar.
3) (Shut down Oozie and) copy the jar file to
/where/you/deployed/oozie/libext/ and re-run bin/oozie-setup.sh prepare-war
4) In oozie-site.xml, set (or add if it doesn't exist) a property like this:
<property>
<name>oozie.service.ELService.ext.functions.workflow</name>
<value>
functionName=my.package.AwesomeELFunctions#functionName
</value>
</property>
If you have more functions, that property takes a comma separated list of
them. The value on the left side of the equals is the name that you'd use
in a workflow and the value on the right side is the name of your function
in your Java code. For the value on the left side, you can optionally put
a prefix (you may have seen many built-in functions that start with "wf:"
for example).
5) Start up Oozie. You should now be able to use the functionName custom
EL Function you created!
I believe this may only make the function available in workflows, but I'm
not sure; if not, there should be a similar property in oozie-site you can
set for coordinators if you need that.
Once my proper tutorial blog post is posted, I'll add a link to this
thread, but it may be a while.
- Robert
On Fri, Jul 12, 2013 at 7:26 AM, Serega Sheypak <se...@gmail.com>wrote:
> Hi, I need to create custom EL function.
> It accepts:
> String StringWithDatetime
> String Pattern (to parse Date)
> It returns:
> time in seconds.
>
> Please tell me:
> 1. Where can I find example?
> 2. Where do I have to put implementation of this function?
>