Re: Oozie workflow to use "hadoop jar" to run a main() method in an MR class

[Robert Kanter <rk...@cloudera.com>]

Hi,

(Moving to user@oozie.apache.org list)

You can use the Java action (
http://oozie.apache.org/docs/4.0.0/WorkflowFunctionalSpec.html#a3.2.7_Java_Action)
to run a main-class; it sounds exactly like what you’re looking for.  In
addition to the documentation, you may also find this blog post I wrote to
be helpful:
http://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/


- Robert





On Fri, Nov 1, 2013 at 6:44 PM, Corey Nolet <cj...@gmail.com> wrote:

> Hello Oozie Devs,
>
> I'm trying to configure a map-reduce workflow that will allow me to fire
> off a static main-class method just like I would with the "hadoop jar"
> command. I'm noticing that the workflow spec provides me with the ability
> to specify the job conf xml but I've already wired up a configuration based
> on command-line arguments.
>
> The reason I'd prefer to wire up the configuration in my main-class is
> because I'm using input formats like the AccumuloInputFormat which is
> storing serialized objects which have been base64 encoded.
>
> I was browsing through the JIRA and didn't see any tickets relating to
> being able to wire up a static main-class and have it called similar to
> "hadoop jar". Am I missing something? I'd be happy to work on this feature
> if it doesn't exist already.
>
> Thanks!
>

Re: Oozie workflow to use "hadoop jar" to run a main() method in an MR class

[Corey Nolet <cj...@gmail.com>]

Hi Robert,

Thank you for your reply! I saw the Java action in the documentation and I
went down that route during my first pass. I wasn't sure if it was expected
to have Oozie fire up a 1-mapper, 0-reducer job just to configure and fire
up another job- it just seemed like there may have been a better way
(possibly even if it didn't exist yet).

Now that I've looked at your blog and thought about it, it does make more
sense to me.

Thanks again!




On Mon, Nov 4, 2013 at 12:48 PM, Robert Kanter <rk...@cloudera.com> wrote:

> Hi,
>
> (Moving to user@oozie.apache.org list)
>
> You can use the Java action (
>
> http://oozie.apache.org/docs/4.0.0/WorkflowFunctionalSpec.html#a3.2.7_Java_Action
> )
> to run a main-class; it sounds exactly like what you’re looking for.  In
> addition to the documentation, you may also find this blog post I wrote to
> be helpful:
>
> http://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/
>
>
> - Robert
>
>
>
>
>
> On Fri, Nov 1, 2013 at 6:44 PM, Corey Nolet <cj...@gmail.com> wrote:
>
> > Hello Oozie Devs,
> >
> > I'm trying to configure a map-reduce workflow that will allow me to fire
> > off a static main-class method just like I would with the "hadoop jar"
> > command. I'm noticing that the workflow spec provides me with the ability
> > to specify the job conf xml but I've already wired up a configuration
> based
> > on command-line arguments.
> >
> > The reason I'd prefer to wire up the configuration in my main-class is
> > because I'm using input formats like the AccumuloInputFormat which is
> > storing serialized objects which have been base64 encoded.
> >
> > I was browsing through the JIRA and didn't see any tickets relating to
> > being able to wire up a static main-class and have it called similar to
> > "hadoop jar". Am I missing something? I'd be happy to work on this
> feature
> > if it doesn't exist already.
> >
> > Thanks!
> >
>