Re: Questions about Airavata and OODT

["Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>]

Here’s my replies, sorry for the delay, Andrea:



>>
>>>Dear Chris,
>>> 
>>>I'm an Altec S.p.A. (Thales-Alenia Group) Software Engineer, we are
>>>currently evaluating available batch processing frameworks and data
>>>products curation frameworks, we plan to build our system on open
>>>source technologies but we are facing difficulties in selecting the
>>>technologies stack because all evaluated projects seem to be immature
>>>or lacking documentation.
>>> 
>>>As a company we early adopters of SMAP data products and a quick
>>>search on google about oodt airavata and smap yielded your name.
>>> 
>>>We are currently evaluating Apache OODT and Apache Airavata and you
>>>seem to be PMC member of both projects (that is why i wrote this email
>>>to you ) so we would like to ask you some questions to guide our
>>>evaluation (feel free to ignore this mail if answering our questions
>>>takes you too much time).
>>> 
>>>How is OODT employed in SMAP?

OODT is used to implement the instrument ground data processing system. It
orchestrates the level 1-4 processors; archives and catalogs the data;
allows
an operator to manage the data processing on the ground, and delivers data
to the NASA DAACs.

>>> 
>>>Is it possible  request support or access to documentation related to
>>>oodt as early adopters of the SMAP data products?

Yes, the point here would be Cynthia Wong, who I have CC’ed on this.

>>> 
>>>How is Airavata related to OODT ( they seem to be close in
>>>capabilities)?

OODT and Airavata have been sort of hand in hand. Airavata came out of the
NSF
TeraGrid and XSEDE projects and is a newer Apache project - OODT has
existed at
the ASF since 2010, and before that, for nearly a decade. They provide
similar
capabilities and we have talked about integrating them at times.

>>> 
>>>Which project has the widest user base?

Hard to tell - OODT’s is pretty clearly NASA, the NIH, elements of the
Square
Kilometre Array project, bioinformatics efforts at CHLA and elsewhere,
DARPA,
some company oriented use in private industry and wide use in Academia.
Airavata
has a natural user base out of the XSEDE national project and is used by
many in
NSF funded academia efforts.

>>> 
>>>Both seem to be lacking a consistent documentation source (about usage
>>>and development) is there an end user focused documentation source?

The closest would be on the OODT wiki. I would check that out, it has a
ton of
docs on there. The other bet is to include your questions to the
dev@oodt.apache.org
list (or similarly Airavata), and there are folks active in both places to
discuss.

>>>
>>>(the only documentation in the confluence apache wiki seems to be
>>>related to internal development and seems really outdated)

Hmm, not the case. It’s pretty actively developed. Have you checked out
OODT RADIX? Or the user guides? Maybe starting from the File Manager one
would
be a good start.

>>> 
>>>Are you aware of public services built on airavata or oodt?

Check out the DARPA Open Catalog; several of them in there.

Also here is a public project:

http://github.com/chrismattmann/drat/

Also check out ImageCat, built from DARPA Memex:

http://github.com/chrismattmann/imagecat/

>>> 
>>>Are these framework production ready?

Yep, they are currently used in production.

>>> 
>>>Are there any management tools?

Yep, OODT has the OPSUI, check the wiki for that. I think Airavata has
some tools used for management as well.

>>> 
>>>Considering our previous questions, are you aware of a production
>>>ready framework similar to airavata or oodt (We are scared by the lack
>>>of documentation for the latter)?

Not really - which is why they fit a niche. They are the glue code for
data systems and really good at what they do.

>>> 
>>>Thanks for your time and your experience.

Anytime! Keep the questions coming.

Cheers,
Chris

>>> 
>>>Best regards
>>> 
>>>Andrea Fonti
>>>Technical Operations
>>>S/W Applications Design & Development
>>>Altec S.p.A.
>>>
>>>---- ALTEC S.p.A. ----
>>>COMUNICAZIONE IMPORTANTE
>>>La presente email è destinata solo al/i destinatario/i sopra indicato/i.
>>>Può riguardare un messaggio confidenziale e contenere informazioni e
>>>dati di natura RISERVATA O CLASSIFICATA. Non è consentita la sua
>>>lettura indebita, la sua duplicazione o la sua divulgazione.
>>> Le opinioni dell'autore di questa email non sono necessariamente le
>>>opinioni di ALTEC S.p.A. Nulla di quanto contenuto in questa email può
>>>sostituire un contratto ovvero una obbligazione vincolante per ALTEC
>>>S.p.A. Nel caso doveste ricevere la presente email  per errore, siete
>>>cortesemente pregati di distruggere la Vostra copia e di contattarci
>>>telefonicamente o a mezzo email o facsimile. Grazie.
>>>
>>>IMPORTANT NOTICE
>>>This email is intended only for the use of the above mentioned
>>>addressee(s). It is confidential and may contain legally PRIVILEGED OR
>>>CLASSIFIED information and data. Unauthorized reading, copying,
>>>disclosure or use is strictly prohibited. The views of the  author
>>>hereof may not necessarily constitute the views of ALTEC S.p.A.
>>>Nothing in this email shall bind ALTEC S.p.A. in any contract or
>>>obligation. If you received this email/facsimile through an error,
>>>please destroy your copy and telephone or email/fax us  immediately.
>>>Thank you.
>>
>>
>>---- ALTEC S.p.A. ----
>>
>>COMUNICAZIONE IMPORTANTE
>>La presente email   destinata solo al/i destinatario/i sopra indicato/i.
>>Pu  riguardare un messaggio confidenziale e contenere informazioni e
>>dati di natura RISERVATA O CLASSIFICATA. Non   consentita la sua
>>lettura indebita, la sua duplicazione o la sua divulgazione. Le
>>opinioni dell'autore di questa email non sono  necessariamente le
>>opinioni di ALTEC S.p.A.
>>Nulla di quanto contenuto in questa email pu  sostituire un contratto
>>ovvero una obbligazione vincolante per ALTEC S.p.A.
>>Nel caso doveste ricevere la presente email per errore, siete
>>cortesemente pregati di distruggere la Vostra copia e di contattarci
>>telefonicamente o a mezzo email o facsimile. Grazie.
>>
>>IMPORTANT NOTICE
>>This email is intended only for the use of the above mentioned
>>addressee(s).
>>It is confidential and may contain legally PRIVILEGED OR CLASSIFIED
>>information and data. Unauthorized reading, copying, disclosure or use
>>is strictly prohibited. The views of the author hereof may not
>>necessarily constitute the views of ALTEC S.p.A.
>>Nothing in this email shall bind ALTEC S.p.A. in any contract or
>>obligation.
>>If you received this email/facsimile through an error, please destroy
>>your copy and telephone or email/fax us immediately. Thank you.
>
>




++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++