You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Gourav Rattihalli <gr...@binghamton.edu> on 2016/08/11 21:24:35 UTC

Integrating Apache Aurora and Marathon with Airavata

Hi Dev's,

I'm working on integrating Apache Aurora and Marathon. I have completed the
implementation of a Java based library that can connect to a running
instances of Aurora and Marathon to submit and monitor jobs. Now, I would
like to integrate this Java library with GFAC and Orchestrator so that the
job launch can be done by Airavata. Could anyone please guide me on this
integration? An example would be great on how to add a new resource
management framework to Airavata.

-- 
Regards,
Gourav Rattihalli

Re: Integrating Apache Aurora and Marathon with Airavata

Posted by Gourav Rattihalli <gr...@binghamton.edu>.
Hi Sameera,

No,  I haven't had a chance to review the design changes that Abhishek has
been adding as part of his GSOC project. I have so far been focused on
getting my client side library to work with the older and current version
of Marathon, Aurora, and Mesos. I will review his design early next week
and use it for my future commits.

On Fri, Aug 12, 2016 at 3:18 PM, Shameera Rathnayaka <shameerainfo@gmail.com
> wrote:

> Hi Gourav,
>
> Have you followed any well know design patterns in you code?
>
> Thanks,
> Shameera.
>
> On Fri, Aug 12, 2016 at 12:11 PM Suresh Marru <sm...@apache.org> wrote:
>
>> On Aug 11, 2016, at 5:24 PM, Gourav Rattihalli <gr...@binghamton.edu>
>> wrote:
>>
>> Hi Dev's,
>>
>> I'm working on integrating Apache Aurora and Marathon. I have completed
>> the implementation of a Java based library that can connect to a running
>> instances of Aurora and Marathon to submit and monitor jobs. Now, I would
>> like to integrate this Java library with GFAC and Orchestrator so that the
>> job launch can be done by Airavata. Could anyone please guide me on this
>> integration? An example would be great on how to add a new resource
>> management framework to Airavata.
>>
>>
>> Hi Gourav,
>>
>> A good example will be to see how the SSH based job submissions work and
>> write your own implementations.
>>
>> For example:
>>
>> This is the task interface - https://github.com/apache/
>> airavata/blob/master/modules/gfac/gfac-core/src/main/java/
>> org/apache/airavata/gfac/core/task/Task.java
>>
>> And here are some task implementations - https://github.com/apache/
>> airavata/tree/master/modules/gfac/gfac-impl/src/main/java/
>> org/apache/airavata/gfac/impl/task
>>
>> Walking through one concrete implementations like Local submissions or
>> SSH or BES will get you an understanding on how to do a similar one for
>> Aurora and Marathon.
>>
>> Suresh
>>
>>
>> --
>> Regards,
>> Gourav Rattihalli
>>
>> --
> Shameera Rathnayaka
>



-- 
Regards,
Gourav Rattihalli

Re: Integrating Apache Aurora and Marathon with Airavata

Posted by Shameera Rathnayaka <sh...@gmail.com>.
Hi Gourav,

Have you followed any well know design patterns in you code?

Thanks,
Shameera.

On Fri, Aug 12, 2016 at 12:11 PM Suresh Marru <sm...@apache.org> wrote:

> On Aug 11, 2016, at 5:24 PM, Gourav Rattihalli <gr...@binghamton.edu>
> wrote:
>
> Hi Dev's,
>
> I'm working on integrating Apache Aurora and Marathon. I have completed
> the implementation of a Java based library that can connect to a running
> instances of Aurora and Marathon to submit and monitor jobs. Now, I would
> like to integrate this Java library with GFAC and Orchestrator so that the
> job launch can be done by Airavata. Could anyone please guide me on this
> integration? An example would be great on how to add a new resource
> management framework to Airavata.
>
>
> Hi Gourav,
>
> A good example will be to see how the SSH based job submissions work and
> write your own implementations.
>
> For example:
>
> This is the task interface -
> https://github.com/apache/airavata/blob/master/modules/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/task/Task.java
>
> And here are some task implementations -
> https://github.com/apache/airavata/tree/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/impl/task
>
> Walking through one concrete implementations like Local submissions or SSH
> or BES will get you an understanding on how to do a similar one for Aurora
> and Marathon.
>
> Suresh
>
>
> --
> Regards,
> Gourav Rattihalli
>
> --
Shameera Rathnayaka

Re: Integrating Apache Aurora and Marathon with Airavata

Posted by Suresh Marru <sm...@apache.org>.
> On Aug 11, 2016, at 5:24 PM, Gourav Rattihalli <gr...@binghamton.edu> wrote:
> 
> Hi Dev's,
> 
> I'm working on integrating Apache Aurora and Marathon. I have completed the implementation of a Java based library that can connect to a running instances of Aurora and Marathon to submit and monitor jobs. Now, I would like to integrate this Java library with GFAC and Orchestrator so that the job launch can be done by Airavata. Could anyone please guide me on this integration? An example would be great on how to add a new resource management framework to Airavata.

Hi Gourav,

A good example will be to see how the SSH based job submissions work and write your own implementations. 

For example:

This is the task interface - https://github.com/apache/airavata/blob/master/modules/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/task/Task.java <https://github.com/apache/airavata/blob/master/modules/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/task/Task.java>

And here are some task implementations - https://github.com/apache/airavata/tree/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/impl/task <https://github.com/apache/airavata/tree/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/impl/task>

Walking through one concrete implementations like Local submissions or SSH or BES will get you an understanding on how to do a similar one for Aurora and Marathon. 

Suresh

> 
> -- 
> Regards,
> Gourav Rattihalli