[Discuss] HTTP for the control flow?

[Markus Weimer <ma...@weimo.de>]

Hi,

one thing that keeps coming up in discussions in the office is the idea
of using HTTP for the control flow messages of REEF. That is, the
heartbeats from Driver to Evaluator as well as messages from the Driver
to the Evaluator such as Task launch and shutdown commands.

The main reason to consider this approach is to free-ride on all the
work people have done in the HTTP space already: There is HTTPS for
security, (reverse) proxies for scalability and protocol intercepting
debugging tools to name just a few.

The main reason not to do it would be potential losses in efficiency, as
HTTP isn't really a low latency protocol.

What do you think? Is this a worthwhile idea to entertain? If so, can
the people more familiar with wake speak to the effort involved?

Markus

Re: [Discuss] HTTP for the control flow?

[Markus Weimer <ma...@weimo.de>]

Hi,

> If one runs REEF jobs in a small scale, using HTTP should be okay
> (workable) although it's not efficient.

Help me understand this. Why wouldn't it be efficient? Doesn't HTTP 2
address many of the issues that traditionally made HTTP inefficient for
use cases like this?

> But if there's a business value, it's worthwhile thinking about HTTP.

Not so much business as laziness value. Consider REEF-247, which would
probably be easier to resolve if we were HTTP based. Also, we will
eventually have to add authentication and encryption to our control
flow. And that is something we would get (almost) for free with HTTP.

> Wake's designed such that it can plug in multiple transports. The current
> one is MessagingTransport that uses raw TCP. One can implement
> HTTPMessagingTransport and ask Wake to use it. Currently, this is not
> Tang-ified, but we should be able to. The default implementation should be
> the current MessagingTransport if we go with this model.

That sounds like a decent plan to make this happen. Thanks!

Markus

Re: [Discuss] HTTP for the control flow?

[Byung-Gon Chun <bg...@gmail.com>]

Hi Markus,

If one runs REEF jobs in a small scale, using HTTP should be okay
(workable) although it's not efficient.
Personally I won't use it for communication between components within a
system.
But if there's a business value, it's worthwhile thinking about HTTP.

Wake's designed such that it can plug in multiple transports. The current
one is MessagingTransport that uses raw TCP. One can implement
HTTPMessagingTransport and ask Wake to use it. Currently, this is not
Tang-ified, but we should be able to. The default implementation should be
the current MessagingTransport if we go with this model.

-Gon

On Mon, Mar 30, 2015 at 3:21 AM, Markus Weimer <ma...@weimo.de> wrote:

> Hi,
>
> one thing that keeps coming up in discussions in the office is the idea
> of using HTTP for the control flow messages of REEF. That is, the
> heartbeats from Driver to Evaluator as well as messages from the Driver
> to the Evaluator such as Task launch and shutdown commands.
>
> The main reason to consider this approach is to free-ride on all the
> work people have done in the HTTP space already: There is HTTPS for
> security, (reverse) proxies for scalability and protocol intercepting
> debugging tools to name just a few.
>
> The main reason not to do it would be potential losses in efficiency, as
> HTTP isn't really a low latency protocol.
>
> What do you think? Is this a worthwhile idea to entertain? If so, can
> the people more familiar with wake speak to the effort involved?
>
> Markus
>



-- 
Byung-Gon Chun