Change to DriverRestart event

[Andrew Chung <af...@gmail.com>]

Hi,

Would you think it is useful to include the number of times the driver has
started (application attempt ID in YARN) in the Driver restart EventHandler
instead of just the start time? Does Mesos provide a similar HA function
and a similar count on restarts? This may break backward compatibility, but
I don't think there are any actual users of the Restart function as of now.

Thanks,
Andrew

Re: Change to DriverRestart event

[John Yang <jo...@gmail.com>]

Hi Andrew,


I'm not sure Mesos has built-in functionality for automatically re-starting
a "Framework", which REEF uses to host the Driver. Maybe this is because
unlike YARN AM which runs as a container in the YARN cluster, Mesos
Framework by default runs outside of the Mesos cluster. Running REEF Driver
inside the Mesos cluster <https://issues.apache.org/jira/browse/REEF-98>
and supporting Driver HA in reef-runtime-mesos
<https://issues.apache.org/jira/browse/REEF-99> are open issues.

Nevertheless I'm also +1 on improving Start/Restart as you proposed. They
will be also useful in implementing related features for the Mesos runtime.


Thanks,
John


On Fri, Aug 7, 2015 at 12:59 AM, Markus Weimer <ma...@weimo.de> wrote:

> +1 on introducing a new type for Start and Restart. The time is almost
> meaningless, and providing some more information is helpful, e.g. the
> restart counter. On the C# side, we've already made the switch to that.
>
> Markus
>
>
> On 2015-08-06 09:45, Andrew Chung wrote:
> > Hi,
> >
> > Would you think it is useful to include the number of times the driver
> has
> > started (application attempt ID in YARN) in the Driver restart
> EventHandler
> > instead of just the start time? Does Mesos provide a similar HA function
> > and a similar count on restarts? This may break backward compatibility,
> but
> > I don't think there are any actual users of the Restart function as of
> now.
> >
> > Thanks,
> > Andrew
> >
>

Re: Change to DriverRestart event

[Markus Weimer <ma...@weimo.de>]

+1 on introducing a new type for Start and Restart. The time is almost
meaningless, and providing some more information is helpful, e.g. the
restart counter. On the C# side, we've already made the switch to that.

Markus


On 2015-08-06 09:45, Andrew Chung wrote:
> Hi,
> 
> Would you think it is useful to include the number of times the driver has
> started (application attempt ID in YARN) in the Driver restart EventHandler
> instead of just the start time? Does Mesos provide a similar HA function
> and a similar count on restarts? This may break backward compatibility, but
> I don't think there are any actual users of the Restart function as of now.
> 
> Thanks,
> Andrew
>