API for containers?

[Rick Mangi <ri...@chartbeat.com>]

Hello,

I’m wondering if there’s currently any API (or other way) to find out which container would handle a theoretical key and what node that container is running on.

The use-case I’m thinking of is if I wanted to run a rest service on each node which could handle queries against the current state of a job by talking to a shared kv store. I saw that there is an elastic search kv implementation in the codebase but didn’t see any way to direct a query to a specific instance of elastic search running on the cluster. To do this you would want to look at a key and determine which container would handle that key in the stream, then hit a service on that node (potentially there could be more than 1 container on the node of course…).

Thanks,

Rick

Re: API for containers?

[Boris Shkolnik <bo...@gmail.com>]

Here is couple of ideas (not sure if that exactly what you want).
You can read JobModel of a job (form JobCoordinator server) and this job
model will contain container to tasks (and to partitions) mapping.

In order to find containers and the nodes - you can use yarn web api.
http://RM/cluster, then go to ApplicationMaster and list all the containers.

But remember this mapping can change, if a container fails, it will be
restarted on the same or different node.

On Wed, Nov 4, 2015 at 11:42 AM, Rick Mangi <ri...@chartbeat.com> wrote:

> Hello,
>
> I’m wondering if there’s currently any API (or other way) to find out
> which container would handle a theoretical key and what node that container
> is running on.
>
> The use-case I’m thinking of is if I wanted to run a rest service on each
> node which could handle queries against the current state of a job by
> talking to a shared kv store. I saw that there is an elastic search kv
> implementation in the codebase but didn’t see any way to direct a query to
> a specific instance of elastic search running on the cluster. To do this
> you would want to look at a key and determine which container would handle
> that key in the stream, then hit a service on that node (potentially there
> could be more than 1 container on the node of course…).
>
> Thanks,
>
> Rick
>
>
>