You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by "Milinda Lakmal Pathirage (Commented) (JIRA)" <ji...@apache.org> on 2012/04/06 06:31:29 UTC
[jira] [Commented] (AIRAVATA-357) Provide cloud bursting like
capabilities to Airavata computational workflows integrating with Apache
Whirr
[ https://issues.apache.org/jira/browse/AIRAVATA-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248040#comment-13248040 ]
Milinda Lakmal Pathirage commented on AIRAVATA-357:
---------------------------------------------------
Exploration of Apache Whirr based Hadoop deployment on clouds in the first phase will provide us with the required knowledge about the complexity of the task and problem associated with moving from local clusters to cloud platforms. I think as the next phase we should consider about moving part of the job to cloud based on the increasing computing resource requirements. This will allow us to understand overhead introduced by this approach and complexities involved like resuming partially completed jobs. Finally we could explorer the area of bursting the computation into computational grids.
I think that there will be a requirement for some heuristics based algorithm for selecting which computing resource to choose and when to burst into cloud. But I think we should implement the simplest case first and then go for the complex scenarios.
I would like to know where this component will reside in Airavata architecture? Will this be a activity in a workflow or workflow activity + some daemon which handles provisioning of cloud resources and job? I'm bias towards the later approach, but I would like to know what is the most natural way when compared with the other Airavata components?
> Provide cloud bursting like capabilities to Airavata computational workflows integrating with Apache Whirr
> ----------------------------------------------------------------------------------------------------------
>
> Key: AIRAVATA-357
> URL: https://issues.apache.org/jira/browse/AIRAVATA-357
> Project: Airavata
> Issue Type: New Feature
> Components: GFac, Workflow Interpreter, XBaya
> Reporter: Suresh Marru
> Labels: gsoc2012, mentor
> Fix For: WISHLIST
>
>
> Apache Airavata provides capabilities to construct execute and monitor computational workflows with built in providers to execute applications on compute intensive resources. The users of Airavata constantly have the need to execute workflows which have tasks with a hybrid combination of computational grids and computational clouds. More over, some applications can be executed on multiple types of resources. The selection will depend on the distribution of data to be processed, the tolerable latency on shared batch queued grid clusters, and on the nature and size of the problem to be solved and data analysis tasks. Especially for applications which reduce the data as they proceed in the graph, they better fit for local map-reduce executions residing on hadoop based file systems.
> This idea has two implementation paths:
> Firstly extend airavata workflow providers to integrate with cloud based run times. Integrating with higher level API's like Apache Whirr provides a great value addition to this project to encompass multiple cloud services.
> Secondly, extending Airavata enactment called Workflow Interpreter so a component in the workflow is capable of initially running on local clusters deploying hadoop file systems and if the execution is saturating the local resources, the executions can scale out to commercial on-demand clouds or leverage high speed low latency interconnects on computational grids. This scaling technique is often referred to as Cloud Bursting. This project will not only enable this capability to Airavata workflow.
> User community & Impact of the software: Airavata is primarily targeted to build science gateways using computational resources from various disciplines. The initial targeted set of gateways include projects supporting research and education in chemistry, life sciences, biophysics, environmental sciences, geosciences astronomy and nuclear physics. The goal of airavata is to enhance productivity of these gateways to utilize cyberinfrastructure of resources (e.g., local lab resources, the Extreme Science and Engineering Discovery Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using open community based software components and services like Airavata, gateways will be able to focus on providing additional scientific capabilities and to expanding the number of supported users. The capabilities of these gateways will offer clear benefits to society.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira