Re: [PROPOSAL] Airavata for the incubator

[Lahiru Gunathilake <gl...@gmail.com>]

Hi Ross,

+1 for this proposal and it looks very interesting to me.. I would like to
contribute to this project.

Regards
Lahiru

On Fri, Apr 29, 2011 at 10:09 AM, Ross Gardler <rg...@apache.org> wrote:

> On 29 Apr 2011, at 16:56, Waruna Ranasinghe <wa...@gmail.com> wrote:
>
> > Hi all,
> >
> > +1 for the proposal Airavata. I would like to contribute to this project.
>
> Yay! Another person who sees personal value. Look forward to working with
> you too :-)
>
> Ross
>
>
> >
> >
> > Thanks,
> > Waruna
> >
> >>>>> On 4/23/11 5:53 PM, Ross Gardler wrote:
> >>>>>>
> >>>>>> I would like to propose Airavata for entry into the Apache
> Incubator.
> >>>>>>
> >>>>>> The full proposal can be found at [1] and is copied at the end of
> this
> >>>>>> mail. For those in a hurry here's a quick summary:
> >>>>>>
> >>>>>> Airavata is a software toolkit currently used to build science
> >> gateways
> >>>>>> but that has a much wider potential use. It provides features to
> >>>>>> compose, manage, execute, and monitor large scale applications and
> >>>>>> workflows on computational resources ranging from local clusters to
> >>>>>> national grids and computing clouds. Users can use Airavata back end
> >>>>>> services and build gadgets to deploy in open social containers such
> as
> >>>>>> Apache Rave and modify them to suite their needs. Airavata builds on
> >>>>>> general concepts of service oriented computing, distributed
> messaging,
> >>>>>> and workflow composition and orchestration.
> >>>>>>
> >>>>>> Airavata will provide web interfaces and scalable Service Oriented
> >>>>>> Architecture based backend services to build or enhance Science
> >> Gateway
> >>>>>> (see https://www.teragrid.org/web/science-gateways/) and similar
> >>>>>> environments. Airavata will specifically focus on:
> >>>>>>
> >>>>>> 1. sophisticated server-side tools for registering and managing
> large
> >>>>>> scale applications on computational resources.
> >>>>>>
> >>>>>> 2. graphical user interfaces to construct, execute, control, manage
> >> and
> >>>>>> reuse of scientific workflows.
> >>>>>>
> >>>>>> 3. interfacing and interoperability with with various external
> (third
> >>>>>> party) data and provenance management tools
> >>>>>>
> >>>>>> The project team consists of a number of existing Apache Committers
> >> and
> >>>>>> the code comes from the same stable as some of the code donated to
> >>>>>> Apache Rave (Incubating).
> >>>>>>
> >>>>>> We welcome your questions, suggestions, observations and support.
> >>>>>>
> >>>>>> Ross
> >>>>>>
> >>>>>> [1] http://wiki.apache.org/incubator/AiravataProposal
> >>>>>>
> >>>>>> FULL PROPOSAL TEXT
> >>>>>> ==================
> >>>>>>
> >>>>>> = Airavata Proposal for Apache Incubator =
> >>>>>>
> >>>>>> == Abstract ==
> >>>>>> Airavata is a software toolkit currently used to build science
> >>>>>> gateways but that has a much wider potential use. It provides
> features
> >>>>>> to compose, manage, execute, and monitor large scale applications
> and
> >>>>>> workflows on computational resources ranging from local clusters to
> >>>>>> national grids and computing clouds. Users can use Airavata back end
> >>>>>> services and build gadgets to deploy in open social containers such
> as
> >>>>>> Apache Rave and modify them to suite their needs. Airavata builds on
> >>>>>> general concepts of service oriented computing, distributed
> messaging,
> >>>>>> and workflow composition and orchestration.
> >>>>>>
> >>>>>>
> >>>>>> == Proposal ==
> >>>>>>
> >>>>>> Airavata will provide web interfaces and scalable Service Oriented
> >>>>>> Architecture based backend services to build or enhance Science
> >>>>>> Gateway (see https://www.teragrid.org/web/science-gateways/)
> >>>>>> and similar environments. Airavata will specifically focus on:
> >>>>>>
> >>>>>> 1. sophisticated server-side tools for registering and managing
> large
> >>>>>> scale applications on computational resources.
> >>>>>> 2. graphical user interfaces to construct, execute, control, manage
> >> and
> >>>>>> reuse of scientific workflows.
> >>>>>> 3. interfacing and interoperability with with various external
> (third
> >>>>>> party) data and provenance management tools.
> >>>>>>
> >>>>>> == Background ==
> >>>>>> Working in close quarters with Apache Axis2 committers and inspired
> by
> >>>>>> the true open source community driven software development of ASF,
> >>>>>> Suresh Marru and Marlon Pierce have been pioneering the idea of a
> >>>>>> Science Gateways software-based Apache project since late 2008. Many
> >>>>>> Apache members have fostered these ideas and guided them to arrive
> at
> >>>>>> this proposal.
> >>>>>>
> >>>>>> Currently the software is a actively used in various science
> >>>>>> gateways. But the tools are general purpose and build upon widely
> used
> >>>>>> Apache tools like Axis2, ODE engine. The core team is motivated to
> >>>>>> expand the community and build a community welcoming both
> synergistic
> >>>>>> software components and also new usage scenarios.
> >>>>>>
> >>>>>> It is perhaps worth noting that one of the three seed projects that
> >>>>>> make up the Apache Rave (Incubating) project is also the product of
> >>>>>> this same team and is derived from the same Science Gateways
> >>>>>> community.
> >>>>>>
> >>>>>> == Rationale ==
> >>>>>>
> >>>>>> The nature of computational problems has evolved from simple desktop
> >>>>>> calculations to complex, multidisciplinary activities that require
> the
> >>>>>> monitoring and analysis of remote data streams, database and web
> >>>>>> search and large ensembles of simulations. In the academic domain
> >>>>>> Science Gateways have emerged to address these needs and have built
> >>>>>> software platforms that provide a community of users with the
> ability
> >>>>>> to easily solve computational problems within a specific domain. The
> >>>>>> tools developed to support these gateways are potentially of value
> to
> >>>>>> any organisation needing to perform complex computations. Gateways
> >>>>>> provide a convenient interface to the underlying infrastrucure
> without
> >>>>>> the need for a deep understanding of the intricacies that
> >>>>>> infrastructure.
> >>>>>>
> >>>>>> We summarize the rationale for choosing The Apache Software
> Foundation
> >>>>>> (ASF) below. This is what we hope to gain from participating in the
> >>>>>> ASF.
> >>>>>>
> >>>>>> 1. '''Broader impact''': our science gateway tool set is based on
> >>>>>> Service Oriented Architecture principles, and it has always been our
> >>>>>> goal to align our software with broader trends in the development of
> >>>>>> software for distributed systems. Participating in the ASF provides
> a
> >>>>>> concrete way to implement this idea. In particular, we have done
> >>>>>> extensive work on the workflow systems, messaging, and application
> >>>>>> management as Web services from the perspective of computational
> >> science
> >>>>>> use cases (i.e., high failure rates, very long running jobs, dynamic
> >>>>>> service creation, workflows not expressible as directed acyclic
> >> graphs,
> >>>>>> etc). These requirements and our work to implement them have already
> >> had
> >>>>>> direct impact on the Apache Axis 2 and Apache ODE projects. As an
> >> Apache
> >>>>>> project, it is hoped that our community will have an enhanced
> >>>>>> opportunity for collaboration and complementary development with
> >> Apache
> >>>>>> Hadoop (for scientific application management), Apache QPID (for
> >>>>>> messaging), Apache Rave (incubator - Open Social Container) and
> >> others.
> >>>>>> It is our goal to expand our software’s usage beyond just science
> >>>>>> gateways to the broader enterprise community.
> >>>>>> 2. '''Sustainability''': Science gateway software development (and
> >>>>>> cyberinfrastructure software generally) is primarily funded in the
> US
> >> by
> >>>>>> the National Science Foundation (NSF), so the long term
> sustainability
> >>>>>> of software across funding cycles is a longstanding problem. The NSF
> >> is
> >>>>>> attempting to solve this problem, and its vision for sustainable
> >>>>>> software is described here:
> >>>>>> http://www.nsf.gov/pubs/2010/nsf10015/nsf10015.jsp. Participating
> in
> >> the
> >>>>>> ASF is our project’s vision for reaching software sustainability
> >> that
> >>>>>> underpins the NSF CF21 vision. As a successful ASF project (after
> >>>>>> incubation), we will have created a community led, rather than
> funding
> >>>>>> led, environment for the development of our sotware. This community,
> >>>>>> through our community engagement work and adoption of meritocratic
> >>>>>> principles, will expand beyond our current core team and existing
> >>>>>> project collaborations. This will greatly increase the chances that
> >> our
> >>>>>> software will continue to grow and improve beyond the participation
> of
> >>>>>> any individuals.
> >>>>>> 3. '''Maturity''': much of the software included in this proposal
> was
> >>>>>> developed initially by graduate students as part of their Ph. D.
> work.
> >>>>>> The Open Grid Computing Environment has devoted significant effort
> >>>>>> (through salaried staff and volunteers from collaborating
> >> institutions)
> >>>>>> to convert these research projects into mature, reliable,
> >> well-written,
> >>>>>> packaged components. The code is currently hosted at SourceForge,
> but
> >> we
> >>>>>> recognize the need to go beyond just the SourceForge support tools
> to
> >>>>>> participate in a real community of software engineering experts. It
> is
> >>>>>> our desire, through the Apache Incubator, to take our software
> >>>>>> engineering efforts to a higher level by learning from the
> substantial
> >>>>>> experience of appropraite Apache Committers. Apache mentors will
> >> provide
> >>>>>> initial guidance, as will the attraction of additional committers
> from
> >>>>>> the relevant Apache projects.
> >>>>>>
> >>>>>> == Initial Goals ==
> >>>>>>
> >>>>>> * Implement a standalone version of the code base with a simple
> hello
> >>>>>> world service, workflow and gadget(s) to access the examples.
> >>>>>> * Migration of documentation and design knowledge from existing SF
> >> project
> >>>>>> * Re-architect Grid based security (GSI) dependencies and adopt more
> >>>>>> general purpose security implementations.
> >>>>>> * Make sure Cloud (including hadoop) support is more first class.
> >>>>>> * Aim to have the first Apache release within the first 6 months
> >>>>>> * Verify with Apache Legal that some of the more esoteric licences
> in
> >>>>>> our dependencies are acceptable, or replace them as appropriate
> >>>>>>
> >>>>>> == Current Status ==
> >>>>>>
> >>>>>> The proposed tools are currently hosted on SourceForge at
> >>>>>> http://sourceforge.net/projects/ogce/ (source at
> >>>>>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/) and
> >> are
> >>>>>> described at http://www.collab-ogce.org.
> >>>>>>
> >>>>>> == Meritocracy ==
> >>>>>>
> >>>>>> A significant portion of initial committers are already ASF
> >>>>>> Committers/Members,
> >>>>>> and the entire team is well experienced with open source software
> >>>>>> development. The existing code base has resulted from
> >>>>>> multi-institutional collaborative projects. The developers are well
> >>>>>> aware of the Apache way and will honor the meritocracy policy of ASF
> >>>>>> foundation.
> >>>>>>
> >>>>>> == Community ==
> >>>>>>
> >>>>>> To date our focus has been serving our immediate partners needs
> rather
> >>>>>> than looking outwards in order to build a broader community with
> >>>>>> diverse needs. Whilst the core team area likely to remain focussed
> on
> >>>>>> the Science Gateways communities we are keen to welcome community
> >>>>>> members from other disciplines.
> >>>>>>
> >>>>>> == Core Developers ==
> >>>>>> Our core developers consist of participants from academic,
> >>>>>> not-for-profit and for-profit organisations. Many are already well
> >>>>>> versed in The Apache Way.
> >>>>>>
> >>>>>> Amongst our initial team we have one or more committers on the
> >>>>>> following Apache top level projects; axis, geronimo, synapse, ws,
> >>>>>> ws-pmc, ws-woden as well as Apache Rave (Incubating).
> >>>>>>
> >>>>>> == Alignment ==
> >>>>>> Airavata software is built upon Apache Projects like Axis2, ODE,
> >>>>>> Rampart, Tomcat and Maven. We will try to closely align the project
> >>>>>> with ODE to ensure BPEL workflow compatibility. We will align with
> >>>>>> metadata management projects like Apache OODT. Web interfaces within
> >>>>>> the Airavata software will be synergistically developed with Apache
> >>>>>> Rave.
> >>>>>>
> >>>>>> == Known Risks ==
> >>>>>> === Orphaned products ===
> >>>>>> We acknowledge the need to seek project contributions outside the
> >> current
> >>>>>> developers. The core team actively travels and conducts workshops
> and
> >>>>>> tutorials at relevant academic conferences like Supercomputing,
> >> TeraGrid,
> >>>>>> Collaborative Technologies Systems and SciDAC. Previous experiences
> >>>>>> have showed that these tutorials and outreach efforts will bring in
> >>>>>> community participation. The general strategy will be to encourage
> >>>>>> users to be active in the community and develop patches and
> >>>>>> contribute. Also, the core developers use the Airavata software in
> >>>>>> multiple projects with a life span ranging from 2 to 10 years, so
> the
> >>>>>> risk of orphaned products is very minimal.
> >>>>>>
> >>>>>> Furthermore, by opening our doors to non-academic organisations
> >>>>>> already adopting large scale computation related projects in the ASF
> >>>>>> we hope to be able to build community beyond the proposing teams
> >>>>>> Science Gateway interests.
> >>>>>>
> >>>>>> === Inexperience with Open Source ===
> >>>>>> The core team is very familiar with open source practices. The
> >>>>>> developers include existing Apache members who have long term
> >> experience
> >>>>>> with
> >>>>>> the Apache Way. The OGCE project has been an active
> >>>>>> open source project in SourceForge since November 2006. We welcome
> the
> >>>>>> new directions and are well prepared to follow the Apache way.
> >>>>>>
> >>>>>> === Homogenous Developers ===
> >>>>>> We have a semi-distributed development environment distributed among
> >>>>>> Indiana University and Lanka Software Foundation. We fully expect
> >>>>>> contributions from the partnering science gateways adding to the
> >>>>>> heterogeneous development.
> >>>>>>
> >>>>>> === Reliance on Salaried Developers ===
> >>>>>> The core developers are self motivated on the project and also are
> >>>>>> funded through various federal, state and endowment research
> >>>>>> grants. Participation in these research efforts based on Airavata
> >>>>>> software is mostly voluntary and above and beyond the requirements
> of
> >>>>>> the salaried jobs.
> >>>>>>
> >>>>>> The Open Gateway Computing project, from which the initial code
> >>>>>> donation is sourced, is funded for the next 3 years and is mandated
> by
> >>>>>> the funding guidelines to open source software development -
> >>>>>> http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=1032742. We
> >>>>>> believe in the Airavata software capabilities and its vital role in
> >>>>>> providing sustainable middleware for Science Gateways. Nevertheless,
> >>>>>> the core team will actively build upon Airavata software and foster
> >>>>>> developer community outside the current core.
> >>>>>>
> >>>>>> === Relationships with Other Apache Products ===
> >>>>>> See “Alignment† above. Airavata is based on the concepts of
> >> Service
> >>>>>> Oriented Architecture and all services run within Tomcat
> >>>>>> container. The web services are based on Axis2. The orchestration of
> >>>>>> the scientific workflows uses Orchestration Director Engine. The
> >>>>>> software is built using Apache Maven.
> >>>>>>
> >>>>>> === An Excessive Fascination with the Apache Brand ===
> >>>>>> The Apache brand would certainly help promote the software suite,
> but
> >>>>>> gaining the brand is not the motivation for this project. Airavata
> is
> >>>>>> being proposed to Apache because of the belief in Apache’s
> >> meritocracy
> >>>>>> model for mentored, community-driven, open source software is the
> best
> >>>>>> way to develop sustainable software. See “Rational† above. Most
> >>>>>> importantly, The Apache Software Foundation will help us create an
> >>>>>> institution-neutral contribution venue and will help us build a
> >>>>>> long-standing community around Airavata to sustain and improve it
> >>>>>> beyond the span of specific, targeted research grants.
> >>>>>>
> >>>>>> == Documentation ==
> >>>>>> Existing documentation is available from the OGCE wiki,
> >>>>>> http://www.collab-ogce.org/ogce/index.php/Main_Page. In addition,
> >>>>>> there is abundance of presentation and self guided video tutorial
> >>>>>> material. Effort will be put in to collect all this information into
> >>>>>> meaningful documentation on the Apache websites.
> >>>>>>
> >>>>>> == Initial Source ==
> >>>>>> The initial source of the project is in SourceForge. The source is
> >>>>>> available for anonymous check out from svn at
> >>>>>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/
> >>>>>>
> >>>>>> == Source and Intellectual Property Submission Plan ==
> >>>>>> Indiana University is the current holder of Intellectual Property
> >>>>>> rights for the software. The university has approved the code
> donation
> >>>>>> and signed trustees approval, Corporate Contributor Licence
> Agreement
> >>>>>> and Software Grant Agreement have been emailed to ASF secretary and
> >>>>>> received acknowledgement.
> >>>>>>
> >>>>>> Specifically Indiana University will donate 4 components into
> Airavata
> >>>>>> project.
> >>>>>>
> >>>>>> 1. XBaya Scientific Workflow Suite - includes a GUI for workflow
> >>>>>> composition and monitoring. The composed workflow can be exported to
> >>>>>> various workflow languages like BPEL, SCUFL, Condor DAG, Jython and
> >>>>>> Java. The defacto workflow enacting engine used is Apache ODE.
> >>>>>> 2. GFac - an application wrapper service that can be used to wrap
> >>>>>> command line-driven science applications and make them into robust,
> >>>>>> network- accessible services. This component is build on Axis2 web
> >>>>>> service stack.
> >>>>>> 3. XRegistry - a registry service for storing deployment information
> >>>>>> about wrapped application services and constructed workflows.
> >>>>>> 4. WS-Messenger - a “publish-subscribe† based message broker
> >>>>>> implemented on top of Apache Axis2 web services stack. It implements
> >> the
> >>>>>> WS-Eventing and WS-Notifications specifications and incorporates a
> >>>>>> message box component that facilities communications with clients
> >> behind
> >>>>>> firewalls and overcomes network glitches.
> >>>>>>
> >>>>>> == External Dependencies ==
> >>>>>>
> >>>>>> Following the guideline -http://www.apache.org/legal/resolved.html,
> >> the
> >>>>>> following are the dependent software and all of them are in binary
> >>>>>> format in java archive (jar files).
> >>>>>>
> >>>>>> * CDDL license - Javax activation, JSR311, Portlet-API, Servlet-API
> >>>>>> * Apache V2: cog-jglobus, globus, caster, gridsphere, Woodstox,
> xmpp,
> >>>>>> xsul, sigiri, atomixmiser, weps-beans.
> >>>>>> * BSD: puretls,
> >>>>>> * MIT: bcporv, hsqldb, dom4j, slf4j
> >>>>>> * PSFL: Jython
> >>>>>> * GPL 2.0: mysql-connector-java
> >>>>>> * Other:
> >>>>>> * cryptix32, cryptix-asn1 (http://www.cryptix.org/LICENSE.TXT)
> >>>>>> * backport (public domain)
> >>>>>> * jaxen (http://jaxen.codehaus.org/license.html)
> >>>>>>
> >>>>>> Licence incompatibilities (GPL) will be resolved during incubation.
> >>>>>>
> >>>>>> == Cryptography ==
> >>>>>> The software does not implement any cryptographic algorithms.
> However,
> >>>>>> to perform secured messaging and data movement and SSL
> communications,
> >>>>>> the software depends upon third party security libraries. These
> >>>>>> external libraries depend in turn on Java Security, Puretls, Cryptix
> >>>>>> and Bounce Castle libraries. Apache Cryptographic steps will be
> >>>>>> followed to register the use of these libraries.
> >>>>>>
> >>>>>> == Required Resources ==
> >>>>>>
> >>>>>> === Mailing lists ===
> >>>>>> 1. airavata-dev
> >>>>>> 2. airavata-commits
> >>>>>> 3. airavata-private
> >>>>>>
> >>>>>> === Subversion Directory ===
> >>>>>> https://svn.apache.org/repos/asf/incubator/airavata
> >>>>>>
> >>>>>> === Issue Tracking ===
> >>>>>> We intend to make use of Jira for issue tracking. Proposed key:
> >> AIRAVATA
> >>>>>>
> >>>>>> === Other Resources ===
> >>>>>>
> >>>>>> We intend to manage our website using the Apache CMS.
> >>>>>>
> >>>>>> == Initial Committers ==
> >>>>>> Names of initial committers with affiliation and current ASF status:
> >>>>>>
> >>>>>> || '''Name''' || '''Email''' || '''Affiliation''' || '''ICLA''' ||
> >>>>>> '''ASF Status''' || '''Apache Id''' ||
> >>>>>> || Suresh Marru || smarru@cs.indiana.edu || Indiana University ||
> On
> >>>>>> File || Apache Commiter || smarru ||
> >>>>>> || Marlon Pierce || mpierce@cs.indiana.edu || Indiana University ||
> >> On
> >>>>>> File || Apache Commiter || mpierce ||
> >>>>>> || Srinath Perera || hemapani@apache.org || Lanka Software
> Foundation
> >> ||
> >>>>>> On File || Apache Member || hemapani ||
> >>>>>> || Aleksander Slominski || aslom at us.ibm.com || IBM || On File ||
> >>>>>> Apache Member || aslom ||
> >>>>>> || Raminderjeet Singh || ramifnu@indiana.edu || Indiana University
> ||
> >> On
> >>>>>> File || Apache Commiter || raminder ||
> >>>>>> || Archit Kulshrestha || akulshre@indiana.edu || Indiana University
> >> ||
> >>>>>> On File || N/A || N/A ||
> >>>>>> || Chathura Herath || chathura@apache.org || Indiana University ||
> On
> >>>>>> File || Apache Commiter || chathura ||
> >>>>>> || Eran Chinthaka || chinthaka@apache.org || Indiana University ||
> On
> >>>>>> File || Apache Member || chinthaka ||
> >>>>>> || Thilina Gunaratne || thilina@apache.org || Indiana University ||
> >> On
> >>>>>> File || Apache Commiter || thilina ||
> >>>>>> || Wathsala Vithanage || wathsala@opensource.lk || Lanka Software
> >>>>>> Foundation || On File || N/A || N/A ||
> >>>>>>
> >>>>>> All the parties are affiliated with companies and organizations that
> >>>>>> are familiar with the development of open source. We expect that the
> >>>>>> amount of volunteer work will increase, and more developers will
> come
> >>>>>> on board.
> >>>>>>
> >>>>>> == Champion ==
> >>>>>> Ross Gardler, Apache Software Foundation
> >>>>>>
> >>>>>> == Nominated Mentors ==
> >>>>>> * Ross Gardler, Member, Apache Software Foundation
> >>>>>> * Alek Slominski, Member, Apache Software Foundation
> >>>>>> * Ate Douma, Member, Apache Software Foundation
> >>>>>> * Sanjiva Weerawarna, Member, Apache Software Foundation
> >>>>>> * Paul Fremantle, Member, Apache Software Foundation
> >>>>>>
> >>>>>> == Sponsoring Entity ==
> >>>>>> Apache Incubator Project.
> >>>>>>
> >>>>>>
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> >>>>>> For additional commands, e-mail: general-help@incubator.apache.org
> >>>>>>
> >>
> >
> > --
> > -----------------------------------------------------
> > Regards,
> > Waruna Ranasinghe
> >
> > blog: http://warunapw.blogspot.com
> > twitter: http://twitter.com/warunapww
> > http://lk.linkedin.com/in/waruna
> > www.facebook.com/waruna.ranasinghe
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>