Launching Accumulo Servers Programatically

[Jim Klucar <kl...@gmail.com>]

Devs,

I've been working on the mesos-accumulo framework. For those not familiar
with Mesos, it is a cluster resource manager that allows you to launch
tasks across the cluster. To do that, you write a mesos framework.

We have proven this is possible with accumulo by writing a quick Scala
framework implementation that uses the accumulo script to launch servers
from a command line. This worked fine but we would like more insight into
the servers once they're launched, outside of does the pid still exist on
the machine. What I would like to do is something like:

org.apache.accumulo.server.Master master = new o.a.a.server.Master();

from inside the mesos executor code. Then setup some config, or pass it in
the constructor and call master.run() or whatever. The point is at the end
of some procedure the server is running, and I can call methods like
master.shutdown() or master.ping() or whatever else is exposed.  A nice
standard server interface across server types would be great.

I realize this may be quite a refactoring of all the server classes so I
thought I'd gather some opinions on the idea here instead of just writing
JIRA tickets.

Jim

Re: Launching Accumulo Servers Programatically

[Josh Elser <jo...@gmail.com>]

Very similar to the problems I ran into on ACCUMULO-2739 (trying to run 
MAC in-process).

One thing I really like from HBase is how their RPC servers are 
implemented. They build the server using protocol-buffers and then they 
have code which receives RPCs and dispatches them to that server "stub". 
The cool part is that internally, the server just looks like a normal 
object and can make "RPCs" locally.

If you look at the public static void main(String[]) method on each 
"server" process, we have the same implementation in nearly all of them. 
It would be nice to consolidate this all (and would likely scratch your 
itch too).

Maybe collecting a list as sub-tasks on a JIRA issue would be good if 
you want to work on this. Otherwise, brainstorming more here is also 
swell :)

Jim Klucar wrote:
> Devs,
>
> I've been working on the mesos-accumulo framework. For those not familiar
> with Mesos, it is a cluster resource manager that allows you to launch
> tasks across the cluster. To do that, you write a mesos framework.
>
> We have proven this is possible with accumulo by writing a quick Scala
> framework implementation that uses the accumulo script to launch servers
> from a command line. This worked fine but we would like more insight into
> the servers once they're launched, outside of does the pid still exist on
> the machine. What I would like to do is something like:
>
> org.apache.accumulo.server.Master master = new o.a.a.server.Master();
>
> from inside the mesos executor code. Then setup some config, or pass it in
> the constructor and call master.run() or whatever. The point is at the end
> of some procedure the server is running, and I can call methods like
> master.shutdown() or master.ping() or whatever else is exposed.  A nice
> standard server interface across server types would be great.
>
> I realize this may be quite a refactoring of all the server classes so I
> thought I'd gather some opinions on the idea here instead of just writing
> JIRA tickets.
>
> Jim
>