metascheduling jobs across cluster nodes and nodes behind firewall

[Pankaj Saha <ps...@binghamton.edu>]

Hello Devs,

It is interesting to see the work being done using Mesos. I have also been
working on the metascheduling problem for some time.  My focus is on
getting Mesos to work across clusters/clouds including agent nodes that do
not have public IP addresses. It is typical with Jetstream, Chameleon, and
other clouds, to get nodes that are only internally accessible. I have been
able to get Mesos to work in our campus cluster with both internally
accessible and public-IP nodes. I have attached a diagram to show what I am
working on.


​

I will be presenting this work at the Gateways workshop next month. Once I
get a chance to clean up the code, and automate the installation, I will
share it. It will be great to work with you all so that once your have the
thrift/Java client for Airavata, the same can perhaps be used with my
metascheduling setup.

Thanks
Pankaj

Re: metascheduling jobs across cluster nodes and nodes behind firewall

[Suresh Marru <sm...@apache.org>]

Hi Pankaj,

Thank you for summarizing your efforts to the list. Can you please submit a PR and work along with the community? Perfection is the enemy in open source, so it will be great to see you commit early and commit often on meta scheduling related activities.

Suresh
 
> On Oct 20, 2016, at 9:51 AM, Pankaj Saha <ps...@binghamton.edu> wrote:
> 
> Hello Devs,
> 
> It is interesting to see the work being done using Mesos. I have also been working on the metascheduling problem for some time.  My focus is on getting Mesos to work across clusters/clouds including agent nodes that do not have public IP addresses. It is typical with Jetstream, Chameleon, and other clouds, to get nodes that are only internally accessible. I have been able to get Mesos to work in our campus cluster with both internally accessible and public-IP nodes. I have attached a diagram to show what I am working on. 
> 
> <Untitled drawing.jpg>
> ​
> 
> I will be presenting this work at the Gateways workshop next month. Once I get a chance to clean up the code, and automate the installation, I will share it. It will be great to work with you all so that once your have the thrift/Java client for Airavata, the same can perhaps be used with my metascheduling setup.
> 
> Thanks
> Pankaj
> <Multi-Cloud Resource Management using Apache Mesos with Apache Airavata.pdf>