You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by Roger Carter <ro...@gmail.com> on 2014/08/06 23:01:54 UTC
Advice for new user
Hi Everyone,
I'm new to the apache scene; I have experience with Matlab and minimal
experience with Python. This seems like a powerful tool and I'd like to
learn more. If anyone is willing to provide reccomendations for resources
or detail their experiences in learning Apache OODT, I would be most
grateful.
Thanks,
Roger
Re: Advice for new user
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi Roger,
You've come at exactly the right time. We are actually having a ConCall
this coming Thursday to discuss the direction/roadmap for the project. It
would be great if you could join us and we can possibly provide some more
insight there.
The call in details are as follows
Our roadmap telecon is scheduled for Thursday, 2014-08-07, at 17:00 UTC.
Time conversions (and iCal/vCal links): http://s.apache.org/00S
To dial in:
International: +1-650-479-3208
US Toll-Free: +1-877-668-4493
Other Toll-Free: http://s.apache.org/globalcallin
Access Code: 2329 5771
I hope as many of us as possible join in the discussion!
In addition we are also revamping the OODT website (staging can be found at
http://oodt.staging.apache.org) we are working to simplify the
documentation as well as make it clearer as to what OODT 'actually does'.
You may wish to take the vagrant radix image for a spin. This is the most
simple and direct manner to get up and running with trying OODT out.
https://cwiki.apache.org/confluence/display/OODT/Vagrant+Powered+OODT
Also the wiki is where most of our documentation actually resides. It is
actively being added to.
hth
Lewis
On Wed, Aug 6, 2014 at 2:01 PM, Roger Carter <ro...@gmail.com>
wrote:
> Hi Everyone,
>
> I'm new to the apache scene; I have experience with Matlab and minimal
> experience with Python. This seems like a powerful tool and I'd like to
> learn more. If anyone is willing to provide reccomendations for resources
> or detail their experiences in learning Apache OODT, I would be most
> grateful.
>
> Thanks,
> Roger
>
--
*Lewis*
Re: Advice for new user
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Cracking answer Ressh.
If we can make this more like the de-facto response to these queries aI
think we will be doing ourselves justice.
Lewis
On Wed, Aug 6, 2014 at 3:25 PM, Verma, Rishi (398J) <
Rishi.Verma@jpl.nasa.gov> wrote:
> Hi Roger,
>
> Great to hear from you, and thanks for considering Apache OODT!
>
> I would say, at a high-level, Apache OODT is a project centered around
> three themes:
> 1. Data management and archival
> 2. Data processing
> 3. Data sharing
>
> Depending on your use case, it would make sense to first identify which of
> those you are interested in, and then investigate the relevant modules
> in-depth. OODT is a component-based architecture, so one can just use
> modules independently of one another if so desired (or use a packaged
> bundle, like mentioned in the Quick Start section below). Much of our
> documentation is currently on our wiki [1], so that is a good place to
> start.
>
> Here are some resources:
>
> Quick-start with OODT:
> * Vagrant Virtual Machine with latest OODT (all components) pre-installed:
> https://cwiki.apache.org/confluence/display/OODT/Vagrant+Powered+OODT
> * RADiX (i.e. OODT, all components, packaged together through a single
> Maven build):
> https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OODT
>
> Data Management and Archival (i.e. taking raw products, extracting
> metadata, archiving metadata and products)
> * File Manager Developer Guide:
> http://oodt.apache.org/components/maven/filemgr/development/developer.html
> * File Manager Policy (i.e. describing the nature of your products for
> archival):
> https://cwiki.apache.org/confluence/display/OODT/Everything+you+want+to+know+about+File+Manager+Policy
> * Crawler (i.e. how to get your products into File Manager):
> https://cwiki.apache.org/confluence/display/OODT/OODT+Crawler+Help
>
> Data Processing (i.e. transforming data already archived or to-be
> archived):
> * Workflow Manager Developer Guide:
> http://oodt.apache.org/components/maven/workflow/development/developer.html
> * CAS-PGE Learn By Example (i.e. how to wrap your external algorithms into
> workflows):
> https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Learn+by+Example
>
> Data Sharing (i.e. sharing and accessing your archive between machines)
> * Web-grid overview:
> http://oodt.apache.org/components/maven/grid/slides.pdf
>
>
> To answer your second question, of experiences dealing with OODT, I've
> personally been using it for archival management for at least two climate
> science projects at NASA Jet Propulsion Laboratory, and for data processing
> needs for two other projects. I think the integration of the Solr catalog
> makes OODT an attractive choice for metadata cataloging and the workflow
> manager makes the creation and execution of batch jobs involving external
> algorithms easier. There's sort of a high-learning curve for OODT (we are
> working on improving documentation!), but once you get the hang of the
> components, its definitely a useful software package.
>
> Hope that helps!
>
> Rishi
>
> --
> [1] https://cwiki.apache.org/confluence/display/OODT/Home
>
> On Aug 6, 2014, at 2:01 PM, Roger Carter wrote:
>
> Hi Everyone,
>
> I'm new to the apache scene; I have experience with Matlab and minimal
> experience with Python. This seems like a powerful tool and I'd like to
> learn more. If anyone is willing to provide reccomendations for resources
> or detail their experiences in learning Apache OODT, I would be most
> grateful.
>
> Thanks,
> Roger
>
>
--
*Lewis*
Re: Advice for new user
Posted by "Verma, Rishi (398J)" <Ri...@jpl.nasa.gov>.
Hi Roger,
Great to hear from you, and thanks for considering Apache OODT!
I would say, at a high-level, Apache OODT is a project centered around three themes:
1. Data management and archival
2. Data processing
3. Data sharing
Depending on your use case, it would make sense to first identify which of those you are interested in, and then investigate the relevant modules in-depth. OODT is a component-based architecture, so one can just use modules independently of one another if so desired (or use a packaged bundle, like mentioned in the Quick Start section below). Much of our documentation is currently on our wiki [1], so that is a good place to start.
Here are some resources:
Quick-start with OODT:
* Vagrant Virtual Machine with latest OODT (all components) pre-installed: https://cwiki.apache.org/confluence/display/OODT/Vagrant+Powered+OODT
* RADiX (i.e. OODT, all components, packaged together through a single Maven build): https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OODT
Data Management and Archival (i.e. taking raw products, extracting metadata, archiving metadata and products)
* File Manager Developer Guide: http://oodt.apache.org/components/maven/filemgr/development/developer.html
* File Manager Policy (i.e. describing the nature of your products for archival): https://cwiki.apache.org/confluence/display/OODT/Everything+you+want+to+know+about+File+Manager+Policy
* Crawler (i.e. how to get your products into File Manager): https://cwiki.apache.org/confluence/display/OODT/OODT+Crawler+Help
Data Processing (i.e. transforming data already archived or to-be archived):
* Workflow Manager Developer Guide: http://oodt.apache.org/components/maven/workflow/development/developer.html
* CAS-PGE Learn By Example (i.e. how to wrap your external algorithms into workflows): https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Learn+by+Example
Data Sharing (i.e. sharing and accessing your archive between machines)
* Web-grid overview: http://oodt.apache.org/components/maven/grid/slides.pdf
To answer your second question, of experiences dealing with OODT, I've personally been using it for archival management for at least two climate science projects at NASA Jet Propulsion Laboratory, and for data processing needs for two other projects. I think the integration of the Solr catalog makes OODT an attractive choice for metadata cataloging and the workflow manager makes the creation and execution of batch jobs involving external algorithms easier. There's sort of a high-learning curve for OODT (we are working on improving documentation!), but once you get the hang of the components, its definitely a useful software package.
Hope that helps!
Rishi
--
[1] https://cwiki.apache.org/confluence/display/OODT/Home
On Aug 6, 2014, at 2:01 PM, Roger Carter wrote:
Hi Everyone,
I'm new to the apache scene; I have experience with Matlab and minimal
experience with Python. This seems like a powerful tool and I'd like to
learn more. If anyone is willing to provide reccomendations for resources
or detail their experiences in learning Apache OODT, I would be most
grateful.
Thanks,
Roger