Re: MaaS + Apache Twill ?

[Nick Allen <ni...@nickallen.org>]

In the long run it is going to make more sense to get on a platform that
someone else supports (aka Apache Twill).  That is, assuming that the
Apache Twill project is sustainable and of high quality.  That lets someone
else do the work to ensure that MaaS continues to work as YARN evolves,
provides new features, and ensures broader testing and usage of core
components.

On Tue, Apr 18, 2017 at 3:50 PM, James Sirota <js...@apache.org> wrote:

> I would stick to MaaS.  We already have service discovery and scaling
> built into MaaS so we really don't have a need for features that Twill
> offers.  I think had we known that Twill existed a few years ago we would
> have considered it, but given where we are today I don't see a reason to
> switch.
>
> Thanks,
> James
>
> 18.04.2017, 12:30, "Nick Allen" <ni...@nickallen.org>:
> >>   Can Twill handle long-running applications?
> >
> > I did see this mentioned in the slide deck for "Big Data North America
> > 2016" around slide 33 or so. The deck is referenced from their home page.
> > If we blindly trust the slide deck, then yes, it does support this. :)
> >
> > On Tue, Apr 18, 2017 at 3:12 PM, Casey Stella <ce...@gmail.com>
> wrote:
> >
> >>  Some things to look for that address some of the complexities within
> MaaS:
> >>
> >>     - Can Twill handle long-running applications?
> >>     - Does Twill provide any sort of name service abstraction or
> somesuch
> >>     that lets us communicate with the app master after app deployment?
> >>     - Does Twill provide any mechanism for impersonation (i.e. we can
> run
> >>     the MaaS twill app as metron, but submit and run models as $user)?
> >>
> >>  On Tue, Apr 18, 2017 at 1:50 PM, Nick Allen <ni...@nickallen.org>
> wrote:
> >>
> >>  > I ran across the Apache Twill [1] project recently whose goal is to
> >>  reduce
> >>  > the complexity of developing distributed applications that run on
> YARN.
> >>  My
> >>  > first thought is that it might offer additional capabilities and/or
> >>  > simplify our current MaaS implementation.
> >>  >
> >>  > Here are a list of features provided by Twill that I think might be
> >>  useful
> >>  > for MaaS.
> >>  >
> >>  > - Service discovery
> >>  > - Elastic scaling
> >>  > - High Availability
> >>  > - Placement policies - Which rack/host should the model run on?
> >>  > - Security - Kerberos ticket refresh?
> >>  >
> >>  > Just wanted to float the thought in the community and see if anyone
> has
> >>  > experience with Twill. I need to do some more research myself.
> >>  >
> >>  > [1] http://twill.apache.org/
> >>  >
>
> -------------------
> Thank you,
>
> James Sirota
> PPMC- Apache Metron (Incubating)
> jsirota AT apache DOT org
>