You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Peter Cseh <ge...@cloudera.com> on 2017/05/26 10:40:15 UTC
OYA! - no more LauncherMappers
Hi everyone,
OOZIE-1770 - Create Oozie Application Master for YARN is committed to
master!
I would like to thank the effort of everybody who was involved in the
design, the development or provided feedback in Jira or on ReviewBoard.
This is a big change in how Oozie works and there is still a lot to do: you
can check out OOZIE-2889 for details.
Thanks again everyone!
gp
--
Peter Cseh
Software Engineer
<http://www.cloudera.com>
Re: OYA! - no more LauncherMappers
Posted by Dongying Jiao <pi...@gmail.com>.
Hi Andras:
I create JIRA OOZIE-2941: https://issues.apache.org/jira/browse/OOZIE-2941
Thanks
2017-06-12 15:05 GMT+08:00 Andras Piros <an...@cloudera.com>:
> Hi Dongying,
>
> while implementing Oozie On YARN, we tried to avoid the most backward
> incompatibility issues possible. However, there are some of these still
> remaining as part as *OOZIE-2889
> <https://issues.apache.org/jira/browse/OOZIE-2889>*. It is mostly targeted
> for Oozie 5.0.0.
>
> Can you please file a JIRA under this umbrella issue w/ the exact steps to
> reproduce?
>
> Thanks,
>
> Andras
>
> On Mon, Jun 12, 2017 at 8:44 AM, Dongying Jiao <pi...@gmail.com>
> wrote:
>
> > Hi:
> > After the merge of OOZIE-1770, it seems the previous workflow can't run
> > successfully. I tried map-reduce apps in oozie examples, it failed. So
> does
> > it mean we need to modify existing workflows in next oozie release?
> >
> > Thanks
> >
> >
> >
> > 2017-05-31 3:00 GMT+08:00 Robert Kanter <rk...@cloudera.com>:
> >
> > > Thanks for getting this in. I think this is going to greatly improve
> > > things for Oozie going forward, and allow us to address common issues
> we
> > > couldn't before.
> > >
> > > On Fri, May 26, 2017 at 4:23 AM, Peter Bacsko <pb...@cloudera.com>
> > > wrote:
> > >
> > > > Finally, it's a milestone :)
> > > >
> > > > By the way, we should summarize what changed and how it affects
> current
> > > > Oozie workflows (perhaps adding something to the Oozie website makes
> > > sense
> > > > too). Right now OYA lacks good backward compatibility, which means
> that
> > > MR
> > > > properties don't have any effect. This will change though.
> > > >
> > > > Another thing that we have to agree on is patching: lots of stuff
> > changed
> > > > inside JavaActionExecutor, LauncherMapper no longer exists, etc.
> > > Therefore
> > > > if someone discovers a problem in these classes (or just wants to add
> > an
> > > > improvement), it will not be possible to patch them. I suggest
> > branching
> > > > off from the commit before OYA and then we can start maintaining two
> > > > branches. But this is just an idea.
> > > >
> > > > Peter
> > > >
> > > > On Fri, May 26, 2017 at 12:40 PM, Peter Cseh <ge...@cloudera.com>
> > > > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > OOZIE-1770 - Create Oozie Application Master for YARN is committed
> to
> > > > > master!
> > > > > I would like to thank the effort of everybody who was involved in
> the
> > > > > design, the development or provided feedback in Jira or on
> > ReviewBoard.
> > > > >
> > > > > This is a big change in how Oozie works and there is still a lot to
> > do:
> > > > you
> > > > > can check out OOZIE-2889 for details.
> > > > >
> > > > > Thanks again everyone!
> > > > > gp
> > > > >
> > > > >
> > > > > --
> > > > > Peter Cseh
> > > > > Software Engineer
> > > > > <http://www.cloudera.com>
> > > > >
> > > >
> > >
> >
>
Re: OYA! - no more LauncherMappers
Posted by Andras Piros <an...@cloudera.com>.
Hi Dongying,
while implementing Oozie On YARN, we tried to avoid the most backward
incompatibility issues possible. However, there are some of these still
remaining as part as *OOZIE-2889
<https://issues.apache.org/jira/browse/OOZIE-2889>*. It is mostly targeted
for Oozie 5.0.0.
Can you please file a JIRA under this umbrella issue w/ the exact steps to
reproduce?
Thanks,
Andras
On Mon, Jun 12, 2017 at 8:44 AM, Dongying Jiao <pi...@gmail.com>
wrote:
> Hi:
> After the merge of OOZIE-1770, it seems the previous workflow can't run
> successfully. I tried map-reduce apps in oozie examples, it failed. So does
> it mean we need to modify existing workflows in next oozie release?
>
> Thanks
>
>
>
> 2017-05-31 3:00 GMT+08:00 Robert Kanter <rk...@cloudera.com>:
>
> > Thanks for getting this in. I think this is going to greatly improve
> > things for Oozie going forward, and allow us to address common issues we
> > couldn't before.
> >
> > On Fri, May 26, 2017 at 4:23 AM, Peter Bacsko <pb...@cloudera.com>
> > wrote:
> >
> > > Finally, it's a milestone :)
> > >
> > > By the way, we should summarize what changed and how it affects current
> > > Oozie workflows (perhaps adding something to the Oozie website makes
> > sense
> > > too). Right now OYA lacks good backward compatibility, which means that
> > MR
> > > properties don't have any effect. This will change though.
> > >
> > > Another thing that we have to agree on is patching: lots of stuff
> changed
> > > inside JavaActionExecutor, LauncherMapper no longer exists, etc.
> > Therefore
> > > if someone discovers a problem in these classes (or just wants to add
> an
> > > improvement), it will not be possible to patch them. I suggest
> branching
> > > off from the commit before OYA and then we can start maintaining two
> > > branches. But this is just an idea.
> > >
> > > Peter
> > >
> > > On Fri, May 26, 2017 at 12:40 PM, Peter Cseh <ge...@cloudera.com>
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > OOZIE-1770 - Create Oozie Application Master for YARN is committed to
> > > > master!
> > > > I would like to thank the effort of everybody who was involved in the
> > > > design, the development or provided feedback in Jira or on
> ReviewBoard.
> > > >
> > > > This is a big change in how Oozie works and there is still a lot to
> do:
> > > you
> > > > can check out OOZIE-2889 for details.
> > > >
> > > > Thanks again everyone!
> > > > gp
> > > >
> > > >
> > > > --
> > > > Peter Cseh
> > > > Software Engineer
> > > > <http://www.cloudera.com>
> > > >
> > >
> >
>
Re: OYA! - no more LauncherMappers
Posted by Dongying Jiao <pi...@gmail.com>.
Hi:
After the merge of OOZIE-1770, it seems the previous workflow can't run
successfully. I tried map-reduce apps in oozie examples, it failed. So does
it mean we need to modify existing workflows in next oozie release?
Thanks
2017-05-31 3:00 GMT+08:00 Robert Kanter <rk...@cloudera.com>:
> Thanks for getting this in. I think this is going to greatly improve
> things for Oozie going forward, and allow us to address common issues we
> couldn't before.
>
> On Fri, May 26, 2017 at 4:23 AM, Peter Bacsko <pb...@cloudera.com>
> wrote:
>
> > Finally, it's a milestone :)
> >
> > By the way, we should summarize what changed and how it affects current
> > Oozie workflows (perhaps adding something to the Oozie website makes
> sense
> > too). Right now OYA lacks good backward compatibility, which means that
> MR
> > properties don't have any effect. This will change though.
> >
> > Another thing that we have to agree on is patching: lots of stuff changed
> > inside JavaActionExecutor, LauncherMapper no longer exists, etc.
> Therefore
> > if someone discovers a problem in these classes (or just wants to add an
> > improvement), it will not be possible to patch them. I suggest branching
> > off from the commit before OYA and then we can start maintaining two
> > branches. But this is just an idea.
> >
> > Peter
> >
> > On Fri, May 26, 2017 at 12:40 PM, Peter Cseh <ge...@cloudera.com>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > OOZIE-1770 - Create Oozie Application Master for YARN is committed to
> > > master!
> > > I would like to thank the effort of everybody who was involved in the
> > > design, the development or provided feedback in Jira or on ReviewBoard.
> > >
> > > This is a big change in how Oozie works and there is still a lot to do:
> > you
> > > can check out OOZIE-2889 for details.
> > >
> > > Thanks again everyone!
> > > gp
> > >
> > >
> > > --
> > > Peter Cseh
> > > Software Engineer
> > > <http://www.cloudera.com>
> > >
> >
>
Re: OYA! - no more LauncherMappers
Posted by Robert Kanter <rk...@cloudera.com>.
Thanks for getting this in. I think this is going to greatly improve
things for Oozie going forward, and allow us to address common issues we
couldn't before.
On Fri, May 26, 2017 at 4:23 AM, Peter Bacsko <pb...@cloudera.com> wrote:
> Finally, it's a milestone :)
>
> By the way, we should summarize what changed and how it affects current
> Oozie workflows (perhaps adding something to the Oozie website makes sense
> too). Right now OYA lacks good backward compatibility, which means that MR
> properties don't have any effect. This will change though.
>
> Another thing that we have to agree on is patching: lots of stuff changed
> inside JavaActionExecutor, LauncherMapper no longer exists, etc. Therefore
> if someone discovers a problem in these classes (or just wants to add an
> improvement), it will not be possible to patch them. I suggest branching
> off from the commit before OYA and then we can start maintaining two
> branches. But this is just an idea.
>
> Peter
>
> On Fri, May 26, 2017 at 12:40 PM, Peter Cseh <ge...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > OOZIE-1770 - Create Oozie Application Master for YARN is committed to
> > master!
> > I would like to thank the effort of everybody who was involved in the
> > design, the development or provided feedback in Jira or on ReviewBoard.
> >
> > This is a big change in how Oozie works and there is still a lot to do:
> you
> > can check out OOZIE-2889 for details.
> >
> > Thanks again everyone!
> > gp
> >
> >
> > --
> > Peter Cseh
> > Software Engineer
> > <http://www.cloudera.com>
> >
>
Re: OYA! - no more LauncherMappers
Posted by Robert Kanter <rk...@cloudera.com>.
Thanks for getting this in. I think this is going to greatly improve
things for Oozie going forward, and allow us to address common issues we
couldn't before.
On Fri, May 26, 2017 at 4:23 AM, Peter Bacsko <pb...@cloudera.com> wrote:
> Finally, it's a milestone :)
>
> By the way, we should summarize what changed and how it affects current
> Oozie workflows (perhaps adding something to the Oozie website makes sense
> too). Right now OYA lacks good backward compatibility, which means that MR
> properties don't have any effect. This will change though.
>
> Another thing that we have to agree on is patching: lots of stuff changed
> inside JavaActionExecutor, LauncherMapper no longer exists, etc. Therefore
> if someone discovers a problem in these classes (or just wants to add an
> improvement), it will not be possible to patch them. I suggest branching
> off from the commit before OYA and then we can start maintaining two
> branches. But this is just an idea.
>
> Peter
>
> On Fri, May 26, 2017 at 12:40 PM, Peter Cseh <ge...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > OOZIE-1770 - Create Oozie Application Master for YARN is committed to
> > master!
> > I would like to thank the effort of everybody who was involved in the
> > design, the development or provided feedback in Jira or on ReviewBoard.
> >
> > This is a big change in how Oozie works and there is still a lot to do:
> you
> > can check out OOZIE-2889 for details.
> >
> > Thanks again everyone!
> > gp
> >
> >
> > --
> > Peter Cseh
> > Software Engineer
> > <http://www.cloudera.com>
> >
>
Re: OYA! - no more LauncherMappers
Posted by Peter Bacsko <pb...@cloudera.com>.
Finally, it's a milestone :)
By the way, we should summarize what changed and how it affects current
Oozie workflows (perhaps adding something to the Oozie website makes sense
too). Right now OYA lacks good backward compatibility, which means that MR
properties don't have any effect. This will change though.
Another thing that we have to agree on is patching: lots of stuff changed
inside JavaActionExecutor, LauncherMapper no longer exists, etc. Therefore
if someone discovers a problem in these classes (or just wants to add an
improvement), it will not be possible to patch them. I suggest branching
off from the commit before OYA and then we can start maintaining two
branches. But this is just an idea.
Peter
On Fri, May 26, 2017 at 12:40 PM, Peter Cseh <ge...@cloudera.com> wrote:
> Hi everyone,
>
> OOZIE-1770 - Create Oozie Application Master for YARN is committed to
> master!
> I would like to thank the effort of everybody who was involved in the
> design, the development or provided feedback in Jira or on ReviewBoard.
>
> This is a big change in how Oozie works and there is still a lot to do: you
> can check out OOZIE-2889 for details.
>
> Thanks again everyone!
> gp
>
>
> --
> Peter Cseh
> Software Engineer
> <http://www.cloudera.com>
>
Re: OYA! - no more LauncherMappers
Posted by Peter Bacsko <pb...@cloudera.com>.
Finally, it's a milestone :)
By the way, we should summarize what changed and how it affects current
Oozie workflows (perhaps adding something to the Oozie website makes sense
too). Right now OYA lacks good backward compatibility, which means that MR
properties don't have any effect. This will change though.
Another thing that we have to agree on is patching: lots of stuff changed
inside JavaActionExecutor, LauncherMapper no longer exists, etc. Therefore
if someone discovers a problem in these classes (or just wants to add an
improvement), it will not be possible to patch them. I suggest branching
off from the commit before OYA and then we can start maintaining two
branches. But this is just an idea.
Peter
On Fri, May 26, 2017 at 12:40 PM, Peter Cseh <ge...@cloudera.com> wrote:
> Hi everyone,
>
> OOZIE-1770 - Create Oozie Application Master for YARN is committed to
> master!
> I would like to thank the effort of everybody who was involved in the
> design, the development or provided feedback in Jira or on ReviewBoard.
>
> This is a big change in how Oozie works and there is still a lot to do: you
> can check out OOZIE-2889 for details.
>
> Thanks again everyone!
> gp
>
>
> --
> Peter Cseh
> Software Engineer
> <http://www.cloudera.com>
>