You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Kevin Bootz <kb...@caci.com> on 2011/01/21 15:27:25 UTC
[PIPELINE]
Is anyone monitoring/developing with the commons pipeline project? The few entries I've seen in the archives and the sandbox are years old and aside from the NOAA website(Ken Tanaka?) there are scarce resources for learning the suite. Any homegrown examples out there that do not incorporate the digester?
Thanks
RE: [PIPELINE]
Posted by Kevin Bootz <kb...@caci.com>.
Thanks Mark,
I've read several of your posts. Most recently those discussing contributions you were thinking of making.
-----Original Message-----
From: Mark Southern [mailto:southern@scripps.edu]
Sent: Friday, January 21, 2011 9:37 AM
To: Commons Users List
Subject: RE: [PIPELINE]
Hi Kevin,
I'm using it. I have some pipelines that pull down data from the web and insert into a hibernate based database. I have some generic Stages that can group/ungroup objects and handle hibernate sessions and committing to the database.
Generic Stages: http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2Fsrc%2Fmain%2Fjava%2Fedu%2Fscripps%2Ffl%2Fpipeline
App specific Stages: http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2Fsrc%2Fmain%2Fjava%2Fedu%2Fscripps%2Ffl%2Fpubchem%2Fapp
Pipeline xml examples: http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2Fsrc%2Fmain%2Fresources%2Fedu%2Fscripps%2Ffl%2Fpubchem%253Fstate%253Dclosed
Best,
~Mark.
-----Original Message-----
From: Kevin Bootz [mailto:kbootz@caci.com]
Sent: Friday, January 21, 2011 9:27 AM
To: user@commons.apache.org
Subject: [PIPELINE]
Is anyone monitoring/developing with the commons pipeline project? The few entries I've seen in the archives and the sandbox are years old and aside from the NOAA website(Ken Tanaka?) there are scarce resources for learning the suite. Any homegrown examples out there that do not incorporate the digester?
Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
RE: [PIPELINE]
Posted by Mark Southern <so...@scripps.edu>.
Thanks for the reference. I haven't. I came across commons-pipeline, it was easy and I looked no further!
-----Original Message-----
From: Gary Gregory [mailto:GGregory@seagullsoftware.com]
Sent: Friday, January 21, 2011 9:44 AM
To: Commons Users List
Subject: RE: [PIPELINE]
Hi Mark,
Just curious: have you considered using Spring Batch?
Gary Gregory
Senior Software Engineer
Rocket Software
3340 Peachtree Road, Suite 820 . Atlanta, GA 30326 . USA
Tel: +1.404.760.1560
Email: ggregory@seagullsoftware.com
Web: seagull.rocketsoftware.com
> -----Original Message-----
> From: Mark Southern [mailto:southern@scripps.edu]
> Sent: Friday, January 21, 2011 09:37
> To: Commons Users List
> Subject: RE: [PIPELINE]
>
> Hi Kevin,
>
> I'm using it. I have some pipelines that pull down data from the web and
> insert into a hibernate based database. I have some generic Stages that can
> group/ungroup objects and handle hibernate sessions and committing to the
> database.
>
> Generic Stages:
> http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2
> Fsrc%2Fmain%2Fjava%2Fedu%2Fscripps%2Ffl%2Fpipeline
>
> App specific Stages:
> http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2
> Fsrc%2Fmain%2Fjava%2Fedu%2Fscripps%2Ffl%2Fpubchem%2Fapp
>
> Pipeline xml examples:
> http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2
> Fsrc%2Fmain%2Fresources%2Fedu%2Fscripps%2Ffl%2Fpubchem%253Fstate%253Dclosed
>
> Best,
> ~Mark.
>
>
> -----Original Message-----
> From: Kevin Bootz [mailto:kbootz@caci.com]
> Sent: Friday, January 21, 2011 9:27 AM
> To: user@commons.apache.org
> Subject: [PIPELINE]
>
> Is anyone monitoring/developing with the commons pipeline project? The few
> entries I've seen in the archives and the sandbox are years old and aside
> from the NOAA website(Ken Tanaka?) there are scarce resources for learning
> the suite. Any homegrown examples out there that do not incorporate the
> digester?
>
> Thanks
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
RE: [PIPELINE]
Posted by Gary Gregory <GG...@seagullsoftware.com>.
Hi Mark,
Just curious: have you considered using Spring Batch?
Gary Gregory
Senior Software Engineer
Rocket Software
3340 Peachtree Road, Suite 820 . Atlanta, GA 30326 . USA
Tel: +1.404.760.1560
Email: ggregory@seagullsoftware.com
Web: seagull.rocketsoftware.com
> -----Original Message-----
> From: Mark Southern [mailto:southern@scripps.edu]
> Sent: Friday, January 21, 2011 09:37
> To: Commons Users List
> Subject: RE: [PIPELINE]
>
> Hi Kevin,
>
> I'm using it. I have some pipelines that pull down data from the web and
> insert into a hibernate based database. I have some generic Stages that can
> group/ungroup objects and handle hibernate sessions and committing to the
> database.
>
> Generic Stages:
> http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2
> Fsrc%2Fmain%2Fjava%2Fedu%2Fscripps%2Ffl%2Fpipeline
>
> App specific Stages:
> http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2
> Fsrc%2Fmain%2Fjava%2Fedu%2Fscripps%2Ffl%2Fpubchem%2Fapp
>
> Pipeline xml examples:
> http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2
> Fsrc%2Fmain%2Fresources%2Fedu%2Fscripps%2Ffl%2Fpubchem%253Fstate%253Dclosed
>
> Best,
> ~Mark.
>
>
> -----Original Message-----
> From: Kevin Bootz [mailto:kbootz@caci.com]
> Sent: Friday, January 21, 2011 9:27 AM
> To: user@commons.apache.org
> Subject: [PIPELINE]
>
> Is anyone monitoring/developing with the commons pipeline project? The few
> entries I've seen in the archives and the sandbox are years old and aside
> from the NOAA website(Ken Tanaka?) there are scarce resources for learning
> the suite. Any homegrown examples out there that do not incorporate the
> digester?
>
> Thanks
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
RE: [PIPELINE]
Posted by Mark Southern <so...@scripps.edu>.
Hi Kevin,
I'm using it. I have some pipelines that pull down data from the web and insert into a hibernate based database. I have some generic Stages that can group/ungroup objects and handle hibernate sessions and committing to the database.
Generic Stages: http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2Fsrc%2Fmain%2Fjava%2Fedu%2Fscripps%2Ffl%2Fpipeline
App specific Stages: http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2Fsrc%2Fmain%2Fjava%2Fedu%2Fscripps%2Ffl%2Fpubchem%2Fapp
Pipeline xml examples: http://code.google.com/p/pubchemdb/source/browse/#svn%2Ftrunk%2FPubChemDB%2Fsrc%2Fmain%2Fresources%2Fedu%2Fscripps%2Ffl%2Fpubchem%253Fstate%253Dclosed
Best,
~Mark.
-----Original Message-----
From: Kevin Bootz [mailto:kbootz@caci.com]
Sent: Friday, January 21, 2011 9:27 AM
To: user@commons.apache.org
Subject: [PIPELINE]
Is anyone monitoring/developing with the commons pipeline project? The few entries I've seen in the archives and the sandbox are years old and aside from the NOAA website(Ken Tanaka?) there are scarce resources for learning the suite. Any homegrown examples out there that do not incorporate the digester?
Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [PIPELINE]
Posted by Ken Tanaka <Ke...@noaa.gov>.
I sent this reply yesterday but haven't seen it show up yet, so I'm
resending this--hopefully this isn't a duplicate.
We still use the pipeline for several scientific data loading projects.
A number of which summarize the spatial geometry and insert records into
a geospatial database (Oracle) that backs our ArcIMS interactive maps. (
Multibeam Bathymetry is one example
http://map.ngdc.noaa.gov/website/mgg/multibeam/viewer.htm ) The pipeline
code has a few updates that haven't been integrated yet (including Mark
Southern's contributions), but there hasn't been much development here
other than keeping up to date with the current version of maven. I know
there are a number of alternatives out there, but the pipeline is fairly
easy to use since it's focus is narrow. Besides Spring Batch, maybe
Kepler <https://kepler-project.org/>, Lift (Scala) or Akka
<http://akka.io/> (Java and Scala) are worth looking into.
We still configure the pipeline with digester, but I'd be interested in
any examples of configuring the pipeline with Spring or Groovy.
-Ken
Kevin Bootz wrote:
> Is anyone monitoring/developing with the commons pipeline project? The few entries I've seen in the archives and the sandbox are years old and aside from the NOAA website(Ken Tanaka?) there are scarce resources for learning the suite. Any homegrown examples out there that do not incorporate the digester?
>
> Thanks
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org