You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by "Suresh Marru (Created) (JIRA)" <ji...@apache.org> on 2012/04/06 05:57:19 UTC

[jira] [Created] (AIRAVATA-357) Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
----------------------------------------------------------------------------------------------------------

                 Key: AIRAVATA-357
                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
             Project: Airavata
          Issue Type: New Feature
          Components: GFac, Workflow Interpreter, XBaya
            Reporter: Suresh Marru
             Fix For: WISHLIST


Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.

This idea has two implementation paths:

Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 

Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 

User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Issue Comment Edited] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256052#comment-13256052 ] 

Milinda Lakmal Pathirage edited comment on AIRAVATA-357 at 4/17/12 11:23 PM:
-----------------------------------------------------------------------------

I'm planning to focus more on the scenario where we execute map tasks on the cloud and reduce tasks on the local cluster like scenario assuming that the computing resource requirements are low for reduce task when compared with the map task. But this involves data movement overhead and additional setup cost for local Hadoop cluster. And also we may need to tweak Hadoop to support this. This has additional infrastructure setup complexity when compared with cloud bursting setup where we do whole computation on a Hadoop cluster running on cloud.

Initial plan is to do simple resource selection based on the location of data and availability of computing resources. Plan is to improve this based on the experience gained during the implementation of simple version of the algorithm. 
                
      was (Author: milinda):
    I'm planning to focus more on the scenario where we execute map tasks on the cloud and reduce tasks on the local cluster like scenario assuming that the computing resource requirements are low for reduce task when compared with the map task. But this involves data movement overhead and additional setup cost for local Hadoop cluster. And also we may need to tweak Hadoop to support this. This has additional infrastructure setup complexity when compared with cloud bursting setup where we do whole computation on a Hadoop cluster running on cloud.
Initial plan is to do simple resource selection based on the location of data and availability of computing resources. Plan is to improve this based on the experience gained during the implementation of simple version of the algorithm. 
                  
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Milinda Lakmal Pathirage updated AIRAVATA-357:
----------------------------------------------

    Attachment: hadoop-provider-partial-30-06-2012.patch
    
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>            Assignee: Milinda Lakmal Pathirage
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>         Attachments: hadoop-provider-partial-30-06-2012.patch
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Issue Comment Edited] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273361#comment-13273361 ] 

Milinda Lakmal Pathirage edited comment on AIRAVATA-357 at 5/11/12 5:17 PM:
----------------------------------------------------------------------------

I started working on Hadoop integration and found the previous implementation of Hadoop provider in OGCE. Now I'm working on migrating that to new GFac API. After that I am planning to integrate Apache Whirr to that so that we can support cloud deployments.
                
      was (Author: milinda):
    I started to working on Hadoop integration and found the previous implementation of Hadoop provider in OGCE. Now I'm working on migrating that to new GFac API. After that I am planning to integrate Apache Whirr to that so that we can support cloud deployments.
                  
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273361#comment-13273361 ] 

Milinda Lakmal Pathirage commented on AIRAVATA-357:
---------------------------------------------------

I started to working on Hadoop integration and found the previous implementation of Hadoop provider in OGCE. Now I'm working on migrating that to new GFac API. After that I am planning to integrate Apache Whirr to that so that we can support cloud deployments.
                
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404670#comment-13404670 ] 

Milinda Lakmal Pathirage commented on AIRAVATA-357:
---------------------------------------------------

During the last couple of weeks I was able to finish integrating Apache Whirr to support cloud deployment of Hadoop and Hadoop provider is alsmost complete. But I hit a road block while implementing input data staging and out put data staging. We(Me and my mentor) had a offline discussion to figure out a solution to this and we came up with following to possible solutions:

 * Implement the input/output data staging outside the GFac and provide them as XBaya workflow activities. So that the users who are using Hadoop support in XBaya can handle data staging on their own.

 * Implement data staging support inside GFac. Based on the input data location and output data location specified by the user GFac will automatically habdle the data movement before/after Hadoop job execution.

We presented these two solutions to Airavata team in a offline discussion and they preffered the second approach where GFac handles the data staging. I figured out that there are things need to be fixed in GFac Core to support this and we have created several issues[1] to track the progress of this. GFac Core changes are happening in a seperate
branch[2] created specifically for this.

Partially implemented Hadoop Provider source is attached to this issue as a patch.

[1] https://issues.apache.org/jira/browse/AIRAVATA-477
[2] https://svn.apache.org/repos/asf/incubator/airavata/sandbox/gfac-refactoring
                
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>            Assignee: Milinda Lakmal Pathirage
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Milinda Lakmal Pathirage reassigned AIRAVATA-357:
-------------------------------------------------

    Assignee: Milinda Lakmal Pathirage
    
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>            Assignee: Milinda Lakmal Pathirage
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254868#comment-13254868 ] 

Milinda Lakmal Pathirage commented on AIRAVATA-357:
---------------------------------------------------

Thank you Suresh for your comments. I think developing this as a separate component will help us in the long run. I'll try GFac and see whether that approach is feasible for us. My plan is to implement a separate library abstracting the cloud bursting functionality and use it for implementing workflow activities and other modules as required.
                
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AIRAVATA-357) Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248040#comment-13248040 ] 

Milinda Lakmal Pathirage commented on AIRAVATA-357:
---------------------------------------------------

Exploration of Apache Whirr based Hadoop deployment on clouds in the first phase will provide us with the required knowledge about the complexity of the task and problem associated with moving from local clusters to cloud platforms. I think as the next phase we should consider about moving part of the job to cloud based on the increasing computing resource requirements. This will allow us to understand overhead introduced by this approach and complexities involved like resuming partially completed jobs. Finally we could explorer the area of bursting the computation into computational grids.

I think that there will be a requirement for some heuristics based algorithm for selecting which computing resource to choose and when to burst into cloud. But I think we should implement the simplest case first and then go for the complex scenarios.

I would like to know where this component will reside in Airavata architecture? Will this be a activity in a workflow or workflow activity + some daemon which handles provisioning of cloud resources and job? I'm bias towards the later approach, but I would like to know what is the most natural way when compared with the other Airavata components?
                
> Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Thilina Gunarathne (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258642#comment-13258642 ] 

Thilina Gunarathne commented on AIRAVATA-357:
---------------------------------------------

I don't think it's a good idea to execute map tasks on cloud and reduce tasks on a local cluster. In almost all the conventional MapReduce applications, Map ->Reduce communication involves significant amount of data, often times even larger than the input data. Transferring intermediate data from cloud -> local cluster or the other way round would cause significant latencies, unreliability and  data transfer charges in the case of commercial clouds. In case we find time to look in to partial migration of jobs, one of the goals of the algorithm should be to minimize the amount of cloud <-> local cluster data transfers. 
                
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AIRAVATA-357) Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Suresh Marru (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248051#comment-13248051 ] 

Suresh Marru commented on AIRAVATA-357:
---------------------------------------

Thank you for quickly grabbing attention to this task. Yes, what you suggest on moving part of the job is very appealing and will have large community uptake. This is so useful for check pointable scientific applications. Airavata framework should consider providing a capability to plug in application logic which evaluates the long running execution, and trigger job migration requesting increased resources or also request a change in data partitioning or user inputs. Any study on understanding the associated over heads will be a big plus. 

As for the logic to when to move to cloud, its a complex beast, as you suggest, you can do heuristics or CBR or other adaptive learning techniques, but really the challenge comes in effectively mapping the current problem at hand with previous data. The challenge being there may be semantic differences among these executions which needs elaborative metadata and provenance description. But glad to see you thinking on these aspects. 

 As for level of integration, all these suggestions are arguable and there are pros and cons of each. I personally favor lose coupling and not having any dependency on critical path of basic airavata functionality, so I second your later approach as well. You can also make the cloud provider API's themselves can be integrated into GFac. GFac chain of responsibility based architecture make this integration seamless. If you are planning a GSoc project based on this task, we can help elaborate in more detail. But in short, these capabilities will require changes to XBaya, GFac, and possibly WS-Messenger (adding publishers). 
                
> Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Milinda Lakmal Pathirage (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256052#comment-13256052 ] 

Milinda Lakmal Pathirage commented on AIRAVATA-357:
---------------------------------------------------

I'm planning to focus more on the scenario where we execute map tasks on the cloud and reduce tasks on the local cluster like scenario assuming that the computing resource requirements are low for reduce task when compared with the map task. But this involves data movement overhead and additional setup cost for local Hadoop cluster. And also we may need to tweak Hadoop to support this. This has additional infrastructure setup complexity when compared with cloud bursting setup where we do whole computation on a Hadoop cluster running on cloud.
Initial plan is to do simple resource selection based on the location of data and availability of computing resources. Plan is to improve this based on the experience gained during the implementation of simple version of the algorithm. 
                
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Thilina Gunarathne (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254900#comment-13254900 ] 

Thilina Gunarathne commented on AIRAVATA-357:
---------------------------------------------

That sounds good and I'm impressed about the GSoc proposal you submitted..  Following are couple of comments regarding the steps you proposed in the proposal, that you would have to think through..

1. What's the complexity of partial migration of Hadoop jobs and in what level are you planning on doing it (eg: adding cloud worker instance to the existing Hadoop cluster vs starting a new master in the cloud to migrate part of the data as a separate job vs ....)? 

 2. What kind of complexity/sophistication you are aiming with  of the heuristics based resource selection algorithms? and how this needs to be different to the existing grid oriented solutions to cater for the clouds? What ideas/algorithms can we borrow from those existing systems?

I'm bit optimistic about the step 3 you propose with regards to the limited time frame we have, but I'm fine with it given the research nature of that component. IMHO any contributions for step 3 would be very valuable, even if we weren't able to finish implementing that step completely during the  GSOC.
                
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (AIRAVATA-357) [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr

Posted by "Suresh Marru (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Suresh Marru updated AIRAVATA-357:
----------------------------------

    Summary: [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr  (was: Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr)
    
> [GSoC] Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRAVATA-357
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-357
>             Project: Airavata
>          Issue Type: New Feature
>          Components: GFac, Workflow Interpreter, XBaya
>            Reporter: Suresh Marru
>              Labels: gsoc2012, mentor
>             Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services. 
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow. 
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira