You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Marlon Pierce <ma...@iu.edu> on 2013/04/15 16:34:42 UTC
Airavata GSOC cloud and grid service provider plugins
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi all--
There has been some great discussion on GSOC projects involving
JS/HTML5/REST/etc user interface environments. These will be great
for exercising the Airavata API and evaluating the best ways to wrap
it as a service.
There's also been a lot of work put into redesigning the GFAC
component to make it easier to provide plugins. It would be great if
we could identify projects that exercise this as well and make a wide
range of new plugins in the process. Some possibilities:
* Azure and other cloud provider plugins to complement the EC2 plugin.
* Open Science Grid-based plugins. These may be interesting for
anyone who has or wants to set up a campus grid
(https://twiki.grid.iu.edu/bin/view/CampusGrids/WebHome).
* JSDL/BES-based service providers. JSDL and BES are Open Grid Forum
standards for running jobs on high performance computers. UNICORE and
SAGA are a couple of implementations.
More creative suggestions for plugins are also welcome.
Marlon
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJRbBACAAoJEOEgD2XReDo5/GcH/RLPW1JeNOI9s0YWWCDh1yWR
jHLOrDg+HzvHlinDfH56earbL0z28Sc+VcN5lkEdgBa35JXMjcZvzmynT3dDEBUH
BIsFM+iqYH4B1fxg8NZ6BsqschosTdVw2JZu8sGapML9m/MOFAIkxB0XLJ6FGyst
Dp3QIVsZL7H5MXl8K9K2efx12qj5lCdKzwzRt1LH+tbSCcpnoeXs8BcnG6jB9kQ4
Q0pLvgQxsTIP2RlapS8SgcZggzdpdKNWimfM1VxBHpcwwIJTzK3LXt17XVeZ/Zan
4K7lJamUkYWCmptFWkyoyzoOrb4h5zsFoNFWsFd8moKefyDjHMcEG9rbhLHyGfk=
=9SId
-----END PGP SIGNATURE-----
Re: Airavata GSOC cloud and grid service provider plugins
Posted by Danushka Menikkumbura <da...@gmail.com>.
Yes. Exactly.
In essence, we would have a unified plugin framework across the system, not
limiting to the GFac core.
Please find the attached PluginManager.png that depicts the big picture.
Furthermore, in the long run we can have the plugin manager as a separate
module so that even XBaya could make use of it to extend itself and with
that in place we would end up writing UI plugins to extend XBaya so that
adding a new UI feature is just a matter of writing a plugin and dropping
it in the plugins directory.
Thanks,
Danushka
On Tue, Apr 16, 2013 at 6:50 AM, Lahiru Gunathilake <gl...@gmail.com>wrote:
> Hi Suresh,
>
> I think he meant, airavat can be configured to have any transport for its
> messaging, either AMQP or just WS-eventing or anything else, we just
> program to the interface and any implementation configured will be executed
> in a given time (like axis2 transport layer). We have similar messaging
> implementation inside gfac-core but its only internal to gfac-core.
>
> Danushka correct me if I am wrong.
>
> Lahiru
>
>
> On Mon, Apr 15, 2013 at 8:47 PM, Suresh Marru <sm...@apache.org> wrote:
>
> > Hi Danushka,
> >
> > Do you think this will be a GSoC project by itself? I did not get what
> did
> > you mean by pluggable transport messaging layer? Within GFac?
> >
> > The web based workflow monitor will also need some light messaging layer,
> > thinking out loud.
> >
> > Suresh
> >
> > On Apr 15, 2013, at 2:01 PM, Danushka Menikkumbura <
> > danushka.menikkumbura@gmail.com> wrote:
> >
> > > Just FYI ; I had something similar in my mind and we had a discussion
> [1]
> > > on dev@ few weeks back. I still believe that it would be a must-have
> > > feature in Airavata.
> > >
> > > [1] -
> > >
> >
> http://apache.markmail.org/message/fnujm4j4kk2h2kxu?q=Decoupling+GFac+Providers
> > >
> > > Thanks,
> > > Danushka
> > >
> > >
> > > On Mon, Apr 15, 2013 at 9:27 PM, Suresh Marru <sm...@apache.org>
> wrote:
> > >
> > >> + 1 this is a great topic and a stand alone one by itself.
> > >>
> > >> A good example to watch here is what Milinda did last year [1] , he
> not
> > >> only added a cloud provider but fundamentally fixed the GFac
> > architecture
> > >> [2]. A similar goal of a concrete task of adding the provider Marlon
> > >> suggested while critically reviewing the provider architecture will be
> > >> great.
> > >>
> > >> Suresh
> > >>
> > >> [1] - https://issues.apache.org/jira/browse/AIRAVATA-357
> > >> [2] - https://issues.apache.org/jira/browse/AIRAVATA-477
> > >>
> > >> On Apr 15, 2013, at 10:34 AM, Marlon Pierce <ma...@iu.edu> wrote:
> > >>
> > >>> -----BEGIN PGP SIGNED MESSAGE-----
> > >>> Hash: SHA1
> > >>>
> > >>> Hi all--
> > >>>
> > >>> There has been some great discussion on GSOC projects involving
> > >>> JS/HTML5/REST/etc user interface environments. These will be great
> > >>> for exercising the Airavata API and evaluating the best ways to wrap
> > >>> it as a service.
> > >>>
> > >>> There's also been a lot of work put into redesigning the GFAC
> > >>> component to make it easier to provide plugins. It would be great if
> > >>> we could identify projects that exercise this as well and make a wide
> > >>> range of new plugins in the process. Some possibilities:
> > >>>
> > >>> * Azure and other cloud provider plugins to complement the EC2
> plugin.
> > >>>
> > >>> * Open Science Grid-based plugins. These may be interesting for
> > >>> anyone who has or wants to set up a campus grid
> > >>> (https://twiki.grid.iu.edu/bin/view/CampusGrids/WebHome).
> > >>>
> > >>> * JSDL/BES-based service providers. JSDL and BES are Open Grid Forum
> > >>> standards for running jobs on high performance computers. UNICORE
> and
> > >>> SAGA are a couple of implementations.
> > >>>
> > >>> More creative suggestions for plugins are also welcome.
> > >>>
> > >>>
> > >>> Marlon
> > >>> -----BEGIN PGP SIGNATURE-----
> > >>> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> > >>> Comment: GPGTools - http://gpgtools.org
> > >>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
> > >>>
> > >>> iQEcBAEBAgAGBQJRbBACAAoJEOEgD2XReDo5/GcH/RLPW1JeNOI9s0YWWCDh1yWR
> > >>> jHLOrDg+HzvHlinDfH56earbL0z28Sc+VcN5lkEdgBa35JXMjcZvzmynT3dDEBUH
> > >>> BIsFM+iqYH4B1fxg8NZ6BsqschosTdVw2JZu8sGapML9m/MOFAIkxB0XLJ6FGyst
> > >>> Dp3QIVsZL7H5MXl8K9K2efx12qj5lCdKzwzRt1LH+tbSCcpnoeXs8BcnG6jB9kQ4
> > >>> Q0pLvgQxsTIP2RlapS8SgcZggzdpdKNWimfM1VxBHpcwwIJTzK3LXt17XVeZ/Zan
> > >>> 4K7lJamUkYWCmptFWkyoyzoOrb4h5zsFoNFWsFd8moKefyDjHMcEG9rbhLHyGfk=
> > >>> =9SId
> > >>> -----END PGP SIGNATURE-----
> > >>
> > >>
> >
> >
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>
Re: Airavata GSOC cloud and grid service provider plugins
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Hi Suresh,
I think he meant, airavat can be configured to have any transport for its
messaging, either AMQP or just WS-eventing or anything else, we just
program to the interface and any implementation configured will be executed
in a given time (like axis2 transport layer). We have similar messaging
implementation inside gfac-core but its only internal to gfac-core.
Danushka correct me if I am wrong.
Lahiru
On Mon, Apr 15, 2013 at 8:47 PM, Suresh Marru <sm...@apache.org> wrote:
> Hi Danushka,
>
> Do you think this will be a GSoC project by itself? I did not get what did
> you mean by pluggable transport messaging layer? Within GFac?
>
> The web based workflow monitor will also need some light messaging layer,
> thinking out loud.
>
> Suresh
>
> On Apr 15, 2013, at 2:01 PM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
> > Just FYI ; I had something similar in my mind and we had a discussion [1]
> > on dev@ few weeks back. I still believe that it would be a must-have
> > feature in Airavata.
> >
> > [1] -
> >
> http://apache.markmail.org/message/fnujm4j4kk2h2kxu?q=Decoupling+GFac+Providers
> >
> > Thanks,
> > Danushka
> >
> >
> > On Mon, Apr 15, 2013 at 9:27 PM, Suresh Marru <sm...@apache.org> wrote:
> >
> >> + 1 this is a great topic and a stand alone one by itself.
> >>
> >> A good example to watch here is what Milinda did last year [1] , he not
> >> only added a cloud provider but fundamentally fixed the GFac
> architecture
> >> [2]. A similar goal of a concrete task of adding the provider Marlon
> >> suggested while critically reviewing the provider architecture will be
> >> great.
> >>
> >> Suresh
> >>
> >> [1] - https://issues.apache.org/jira/browse/AIRAVATA-357
> >> [2] - https://issues.apache.org/jira/browse/AIRAVATA-477
> >>
> >> On Apr 15, 2013, at 10:34 AM, Marlon Pierce <ma...@iu.edu> wrote:
> >>
> >>> -----BEGIN PGP SIGNED MESSAGE-----
> >>> Hash: SHA1
> >>>
> >>> Hi all--
> >>>
> >>> There has been some great discussion on GSOC projects involving
> >>> JS/HTML5/REST/etc user interface environments. These will be great
> >>> for exercising the Airavata API and evaluating the best ways to wrap
> >>> it as a service.
> >>>
> >>> There's also been a lot of work put into redesigning the GFAC
> >>> component to make it easier to provide plugins. It would be great if
> >>> we could identify projects that exercise this as well and make a wide
> >>> range of new plugins in the process. Some possibilities:
> >>>
> >>> * Azure and other cloud provider plugins to complement the EC2 plugin.
> >>>
> >>> * Open Science Grid-based plugins. These may be interesting for
> >>> anyone who has or wants to set up a campus grid
> >>> (https://twiki.grid.iu.edu/bin/view/CampusGrids/WebHome).
> >>>
> >>> * JSDL/BES-based service providers. JSDL and BES are Open Grid Forum
> >>> standards for running jobs on high performance computers. UNICORE and
> >>> SAGA are a couple of implementations.
> >>>
> >>> More creative suggestions for plugins are also welcome.
> >>>
> >>>
> >>> Marlon
> >>> -----BEGIN PGP SIGNATURE-----
> >>> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> >>> Comment: GPGTools - http://gpgtools.org
> >>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
> >>>
> >>> iQEcBAEBAgAGBQJRbBACAAoJEOEgD2XReDo5/GcH/RLPW1JeNOI9s0YWWCDh1yWR
> >>> jHLOrDg+HzvHlinDfH56earbL0z28Sc+VcN5lkEdgBa35JXMjcZvzmynT3dDEBUH
> >>> BIsFM+iqYH4B1fxg8NZ6BsqschosTdVw2JZu8sGapML9m/MOFAIkxB0XLJ6FGyst
> >>> Dp3QIVsZL7H5MXl8K9K2efx12qj5lCdKzwzRt1LH+tbSCcpnoeXs8BcnG6jB9kQ4
> >>> Q0pLvgQxsTIP2RlapS8SgcZggzdpdKNWimfM1VxBHpcwwIJTzK3LXt17XVeZ/Zan
> >>> 4K7lJamUkYWCmptFWkyoyzoOrb4h5zsFoNFWsFd8moKefyDjHMcEG9rbhLHyGfk=
> >>> =9SId
> >>> -----END PGP SIGNATURE-----
> >>
> >>
>
>
--
System Analyst Programmer
PTI Lab
Indiana University
Re: Airavata GSOC cloud and grid service provider plugins
Posted by Suresh Marru <sm...@apache.org>.
Hi Danushka,
Do you think this will be a GSoC project by itself? I did not get what did you mean by pluggable transport messaging layer? Within GFac?
The web based workflow monitor will also need some light messaging layer, thinking out loud.
Suresh
On Apr 15, 2013, at 2:01 PM, Danushka Menikkumbura <da...@gmail.com> wrote:
> Just FYI ; I had something similar in my mind and we had a discussion [1]
> on dev@ few weeks back. I still believe that it would be a must-have
> feature in Airavata.
>
> [1] -
> http://apache.markmail.org/message/fnujm4j4kk2h2kxu?q=Decoupling+GFac+Providers
>
> Thanks,
> Danushka
>
>
> On Mon, Apr 15, 2013 at 9:27 PM, Suresh Marru <sm...@apache.org> wrote:
>
>> + 1 this is a great topic and a stand alone one by itself.
>>
>> A good example to watch here is what Milinda did last year [1] , he not
>> only added a cloud provider but fundamentally fixed the GFac architecture
>> [2]. A similar goal of a concrete task of adding the provider Marlon
>> suggested while critically reviewing the provider architecture will be
>> great.
>>
>> Suresh
>>
>> [1] - https://issues.apache.org/jira/browse/AIRAVATA-357
>> [2] - https://issues.apache.org/jira/browse/AIRAVATA-477
>>
>> On Apr 15, 2013, at 10:34 AM, Marlon Pierce <ma...@iu.edu> wrote:
>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Hi all--
>>>
>>> There has been some great discussion on GSOC projects involving
>>> JS/HTML5/REST/etc user interface environments. These will be great
>>> for exercising the Airavata API and evaluating the best ways to wrap
>>> it as a service.
>>>
>>> There's also been a lot of work put into redesigning the GFAC
>>> component to make it easier to provide plugins. It would be great if
>>> we could identify projects that exercise this as well and make a wide
>>> range of new plugins in the process. Some possibilities:
>>>
>>> * Azure and other cloud provider plugins to complement the EC2 plugin.
>>>
>>> * Open Science Grid-based plugins. These may be interesting for
>>> anyone who has or wants to set up a campus grid
>>> (https://twiki.grid.iu.edu/bin/view/CampusGrids/WebHome).
>>>
>>> * JSDL/BES-based service providers. JSDL and BES are Open Grid Forum
>>> standards for running jobs on high performance computers. UNICORE and
>>> SAGA are a couple of implementations.
>>>
>>> More creative suggestions for plugins are also welcome.
>>>
>>>
>>> Marlon
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
>>> Comment: GPGTools - http://gpgtools.org
>>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>>
>>> iQEcBAEBAgAGBQJRbBACAAoJEOEgD2XReDo5/GcH/RLPW1JeNOI9s0YWWCDh1yWR
>>> jHLOrDg+HzvHlinDfH56earbL0z28Sc+VcN5lkEdgBa35JXMjcZvzmynT3dDEBUH
>>> BIsFM+iqYH4B1fxg8NZ6BsqschosTdVw2JZu8sGapML9m/MOFAIkxB0XLJ6FGyst
>>> Dp3QIVsZL7H5MXl8K9K2efx12qj5lCdKzwzRt1LH+tbSCcpnoeXs8BcnG6jB9kQ4
>>> Q0pLvgQxsTIP2RlapS8SgcZggzdpdKNWimfM1VxBHpcwwIJTzK3LXt17XVeZ/Zan
>>> 4K7lJamUkYWCmptFWkyoyzoOrb4h5zsFoNFWsFd8moKefyDjHMcEG9rbhLHyGfk=
>>> =9SId
>>> -----END PGP SIGNATURE-----
>>
>>
Re: Airavata GSOC cloud and grid service provider plugins
Posted by Danushka Menikkumbura <da...@gmail.com>.
Just FYI ; I had something similar in my mind and we had a discussion [1]
on dev@ few weeks back. I still believe that it would be a must-have
feature in Airavata.
[1] -
http://apache.markmail.org/message/fnujm4j4kk2h2kxu?q=Decoupling+GFac+Providers
Thanks,
Danushka
On Mon, Apr 15, 2013 at 9:27 PM, Suresh Marru <sm...@apache.org> wrote:
> + 1 this is a great topic and a stand alone one by itself.
>
> A good example to watch here is what Milinda did last year [1] , he not
> only added a cloud provider but fundamentally fixed the GFac architecture
> [2]. A similar goal of a concrete task of adding the provider Marlon
> suggested while critically reviewing the provider architecture will be
> great.
>
> Suresh
>
> [1] - https://issues.apache.org/jira/browse/AIRAVATA-357
> [2] - https://issues.apache.org/jira/browse/AIRAVATA-477
>
> On Apr 15, 2013, at 10:34 AM, Marlon Pierce <ma...@iu.edu> wrote:
>
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Hi all--
> >
> > There has been some great discussion on GSOC projects involving
> > JS/HTML5/REST/etc user interface environments. These will be great
> > for exercising the Airavata API and evaluating the best ways to wrap
> > it as a service.
> >
> > There's also been a lot of work put into redesigning the GFAC
> > component to make it easier to provide plugins. It would be great if
> > we could identify projects that exercise this as well and make a wide
> > range of new plugins in the process. Some possibilities:
> >
> > * Azure and other cloud provider plugins to complement the EC2 plugin.
> >
> > * Open Science Grid-based plugins. These may be interesting for
> > anyone who has or wants to set up a campus grid
> > (https://twiki.grid.iu.edu/bin/view/CampusGrids/WebHome).
> >
> > * JSDL/BES-based service providers. JSDL and BES are Open Grid Forum
> > standards for running jobs on high performance computers. UNICORE and
> > SAGA are a couple of implementations.
> >
> > More creative suggestions for plugins are also welcome.
> >
> >
> > Marlon
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> > Comment: GPGTools - http://gpgtools.org
> > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
> >
> > iQEcBAEBAgAGBQJRbBACAAoJEOEgD2XReDo5/GcH/RLPW1JeNOI9s0YWWCDh1yWR
> > jHLOrDg+HzvHlinDfH56earbL0z28Sc+VcN5lkEdgBa35JXMjcZvzmynT3dDEBUH
> > BIsFM+iqYH4B1fxg8NZ6BsqschosTdVw2JZu8sGapML9m/MOFAIkxB0XLJ6FGyst
> > Dp3QIVsZL7H5MXl8K9K2efx12qj5lCdKzwzRt1LH+tbSCcpnoeXs8BcnG6jB9kQ4
> > Q0pLvgQxsTIP2RlapS8SgcZggzdpdKNWimfM1VxBHpcwwIJTzK3LXt17XVeZ/Zan
> > 4K7lJamUkYWCmptFWkyoyzoOrb4h5zsFoNFWsFd8moKefyDjHMcEG9rbhLHyGfk=
> > =9SId
> > -----END PGP SIGNATURE-----
>
>
Re: Airavata GSOC cloud and grid service provider plugins
Posted by Suresh Marru <sm...@apache.org>.
+ 1 this is a great topic and a stand alone one by itself.
A good example to watch here is what Milinda did last year [1] , he not only added a cloud provider but fundamentally fixed the GFac architecture [2]. A similar goal of a concrete task of adding the provider Marlon suggested while critically reviewing the provider architecture will be great.
Suresh
[1] - https://issues.apache.org/jira/browse/AIRAVATA-357
[2] - https://issues.apache.org/jira/browse/AIRAVATA-477
On Apr 15, 2013, at 10:34 AM, Marlon Pierce <ma...@iu.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi all--
>
> There has been some great discussion on GSOC projects involving
> JS/HTML5/REST/etc user interface environments. These will be great
> for exercising the Airavata API and evaluating the best ways to wrap
> it as a service.
>
> There's also been a lot of work put into redesigning the GFAC
> component to make it easier to provide plugins. It would be great if
> we could identify projects that exercise this as well and make a wide
> range of new plugins in the process. Some possibilities:
>
> * Azure and other cloud provider plugins to complement the EC2 plugin.
>
> * Open Science Grid-based plugins. These may be interesting for
> anyone who has or wants to set up a campus grid
> (https://twiki.grid.iu.edu/bin/view/CampusGrids/WebHome).
>
> * JSDL/BES-based service providers. JSDL and BES are Open Grid Forum
> standards for running jobs on high performance computers. UNICORE and
> SAGA are a couple of implementations.
>
> More creative suggestions for plugins are also welcome.
>
>
> Marlon
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJRbBACAAoJEOEgD2XReDo5/GcH/RLPW1JeNOI9s0YWWCDh1yWR
> jHLOrDg+HzvHlinDfH56earbL0z28Sc+VcN5lkEdgBa35JXMjcZvzmynT3dDEBUH
> BIsFM+iqYH4B1fxg8NZ6BsqschosTdVw2JZu8sGapML9m/MOFAIkxB0XLJ6FGyst
> Dp3QIVsZL7H5MXl8K9K2efx12qj5lCdKzwzRt1LH+tbSCcpnoeXs8BcnG6jB9kQ4
> Q0pLvgQxsTIP2RlapS8SgcZggzdpdKNWimfM1VxBHpcwwIJTzK3LXt17XVeZ/Zan
> 4K7lJamUkYWCmptFWkyoyzoOrb4h5zsFoNFWsFd8moKefyDjHMcEG9rbhLHyGfk=
> =9SId
> -----END PGP SIGNATURE-----
Re: Airavata GSOC cloud and grid service provider plugins
Posted by Danushka Menikkumbura <da...@gmail.com>.
> More creative suggestions for plugins are also welcome.
>
A plug-able transport layer for WS-Messenger [1] would also be quite
appealing as I believe.
[1] -
http://apache.markmail.org/message/rpz5uom2opeewrg3?q=AMQP+Support+for+WS-Messenger
Thanks,
Danushka