[jira] [Commented] (MESOS-930) Provide slave<->executor protocol

["Benjamin Mahler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883520#comment-13883520 ] 

Benjamin Mahler commented on MESOS-930:
---------------------------------------

I think we should step back a bit and think more generally about our goals around language bindings, and then think about ways we can achieve them. Pure bindings that contain the driver logic may have disadvantages as well, mostly around having to manage the driver related logic across all of the bindings.

This discussion started due to the issues with our current implementation, that is, the Python bindings may be incompatible with some Python event / threading models.

Let's step back and think about goals first, a design doc or proposal with alternatives would be great here!

Goals (anything I'm missing?):
# Make it easy to add bindings for a particular language.
# Allow for bindings to follow an idiomatic programming model for the particular language.
# Ensure that the bindings can be maintainable: can they be updated easily? Can they be bundled as supported and tested with Mesos?

I think with the suggestion in this ticket, we're neglecting (1) and (3). (1) because the bindings will need to re-implement the driver functionality (this will become increasingly difficult if/when the scheduler driver becomes stateful). (3) because whenever we want to add driver-side implementation, we would need to ensure all of the bindings are updated.

With our current implementation, (1) and (2) are neglected. (1) because some languages do not have a bridge to C\+\+. (2) because the API is synchronous and we use libev which might interfere with other languages that use an event model.

With these things in mind, we may want to consider an alternative route, where we continue to have C\+\+ drivers, but rather than requiring a bridge through C\+\+, we also offer the option of inter-process communication through pipes or sockets. This way a language binding could opt to run the driver as a subprocess and communicate to it without needing the C\+\+ bridge. This would look identical to running an embedded driver as done today, but with a different flag passed when constructing the driver.

What do you think? Do you want to drive a design doc / proposal with alternatives for this?

> Provide slave<->executor protocol
> ---------------------------------
>
>                 Key: MESOS-930
>                 URL: https://issues.apache.org/jira/browse/MESOS-930
>             Project: Mesos
>          Issue Type: Improvement
>          Components: general
>            Reporter: Nikita Vetoshkin
>            Priority: Minor
>
> This ticket is the result of the discussion started in mailing list (http://www.mail-archive.com/dev@mesos.apache.org/msg05477.html).
> It would be great if Mesos provided protocol for slave<->executor communication additionally to currently provided c++ based language bindings.
> ocumenting
> wire protocol could open ways to implement Executors in pure python or
> golang or any other language. It could provide some benefits:
>   * in Python one could use gevent which is pretty popular
>   * golang has it's own eventloop builtin
>   * pure language binding could save a lot of trouble bridging with
> unfriendly C++
>   * building and using pure language client could be performed using native
> tools like `pip install` or `go get` without need to establish c++ dev
> environment.
> Before moving any further we need to decide, if this is actually a good thing to do. According to discussion in mail list - looks like it's worth doing.
> So next logical thing is to decide:
>   * should protocol be something utterly new
>   * should we document current protocol used by libprocess
>   * should libprocess protocol be brushed a little before documenting it for external implementation
> While waiting for discussion I'd like to start documenting current protocol. Where can one do it in a way suitable for comments? RB?



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)