Re: Best way to realize stateful services architecture with massive data?

[Aleksander Slominski <as...@cs.indiana.edu>]

hi Florian,

check out LEAD project [1] in particular GPEL engine (Grid Process 
Execution Language - strict subset of BPEL with Grid in mind and data 
intensive ops but leaving out business focused functionality etc) , GFac 
(Grid Factory), and XBaya - with we did exactly what you describe - GPEL 
engine executes subset of BPEL workflow and hands off long running jobs 
through WS invocation with WS-Addressing callback URL to GFac that then 
stages files, puts job  in queue, monitors execution, sends events about 
status (so we know after few hours it is still stuck in queue or takes 
long to finish), when jobs is finished it stages out files with URIs and 
URL for use in next stage of workflow (large data is passed by reference 
i.e. URLs we do nto tunnel GBs or TBs of data as base64 or MTOM 
attachments as it would be somewhat pointless :evil:), users can monitor 
jobs from portal and compose them in simple to use GUI (XBaya) that is 
specific to data workflows and so on.

best,

Alek

[1] links:
http://www.extreme.indiana.edu/portals/index.shtml
http:/www.extreme.indiana.edu/gfac/
http://www.extreme.indiana.edu/xgws/xbaya

On Oct 21, 2007 10:29 AM, Florian Georg <fl...@uni-konstanz.de> 
wrote:
 > I´m currently thinking of the best way to realize data-intensive
 > "workflows" composed of web services (similar e.g. to a BPEL engine).
 > I´ll looking at Axis2 engine for that, but I´m unsure how it could fit
 > my special needs:
 >
 > What I need to do is to be able to call subsequent operations on a
 > service (e.g. configure(), execute(), getNextResults(), reset(), ..).
 > Background is that these are very long running services which consume
 > and produce potentially some GBs of data.
 > The services are dependend of each other, e.g. execute() may only be
 > called after a predecessor service has successfully executed.
 > Produced data should be re-usable by differend nodes, e.g. passing
 > around and re-generate should be minimized.
 >
 > What do you think of this?
 > What does Axis provide me for this scenario?
 > Should I use, e.g. WS-Resource (Muse) for this? An ESB ?
 > Or should I even consider mapping this into BPEL and use a process
 > engine?
 > (ok, I´ll stop here before it becomes too off-topic)
 >
 > Any remarks would be greatly appreciated.
 >
 > thanks in advance
 >  -- Florian
 >
 >
 >
 > ---------------------------------------------------------------------
 > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
 > For additional commands, e-mail: axis-user-help@ws.apache.org
 >
 >


-- 
The best way to predict the future is to invent it - Alan Kay


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org