You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stanbol.apache.org by David Riccitelli <da...@insideout.io> on 2012/01/06 16:47:17 UTC
asynchronous jobs
Hello guys,
At the Zurich CreateJS Hackathon we started working on the asynchronous
jobs for Stanbol. More specifically we need a REST API that allows us to:
1. start an enhancement job (passing its configuration data),
2. check its status and progress,
3. retrieve the results once the process is complete [1],
4. cancel an ongoing job.
We saw recently the release of the long-term jobs for the Reasoning
Services by Enrico Daga [2,3], and found that it can mostly satisfy the
above mentioned requirements with the exception of monitoring the actual
job progress.
I talked with Rupert about it, and he's looking forward to the asynchronous
operations OSGI postEvent/sendEvent methods, which would allow us to
reshape the way the tasks are instantiated, their progress monitored and
managed.
I would like to define with you the requirements for the asynchronous jobs
and a path in order to reach this goal, if you agree.
In the meanwhile I've been implementing a raw EnhancerJob/EnhancerJobResult
implementation in order to enable the *commons.jobs* APIs to be used with
the *enhancer*, and I can share this work if you'd like.
BR,
David
--
David Riccitelli
********************************************************************************
InsideOut10 s.r.l.
P.IVA: IT-11381771002
Fax: +39 0110708239
---
LinkedIn: http://it.linkedin.com/in/riccitelli
Twitter: ziodave
---
Layar Partner Network<http://www.layar.com/publishing/developers/list/?page=1&country=&city=&keyword=insideout10&lpn=1>
********************************************************************************
Re: asynchronous jobs
Posted by David Riccitelli <da...@insideout.io>.
Ok,
I've put it here:
https://github.com/InsideOut10/stanbol/tree/trunk/exts/enhancerjobsmanager
In order to use it:
curl -v -X POST --data-urlencode content=@sample
http://localhost:8080/enhancerjobs/
where *sample* is a text file with the content to be analyzed.
As per long-term jobs operation specs, the response will contain a URL with
the JobId:
HTTP/1.1 201 Created
Location: http://localhost:8080/jobs/YOezXhUqT0wUawxO9ap8PA
...
The URL can be queried to know the state of the job and where to locate its
output:
curl -X GET http://localhost:8080/jobs/YOezXhUqT0wUawxO9ap8PA
e.g.:
{
"status": "finished",
"outputLocation": "http://localhost:8080/enhancerjobs/YOezXhUqT0wUawxO9ap8PA
",
"messages": [
"You can remove this job using DELETE",
]
}
The outputLocation will return the results following the same formats/specs
of the original synchronous HTTP REST call.
BR
David
On Fri, Jan 6, 2012 at 5:20 PM, Olivier Grisel <ol...@ensta.org>wrote:
> 2012/1/6 David Riccitelli <da...@insideout.io>:
> > Hello guys,
> >
> > At the Zurich CreateJS Hackathon we started working on the asynchronous
> > jobs for Stanbol. More specifically we need a REST API that allows us to:
> > 1. start an enhancement job (passing its configuration data),
> > 2. check its status and progress,
> > 3. retrieve the results once the process is complete [1],
> > 4. cancel an ongoing job.
> >
> > We saw recently the release of the long-term jobs for the Reasoning
> > Services by Enrico Daga [2,3], and found that it can mostly satisfy the
> > above mentioned requirements with the exception of monitoring the actual
> > job progress.
> >
> > I talked with Rupert about it, and he's looking forward to the
> asynchronous
> > operations OSGI postEvent/sendEvent methods, which would allow us to
> > reshape the way the tasks are instantiated, their progress monitored and
> > managed.
> >
> > I would like to define with you the requirements for the asynchronous
> jobs
> > and a path in order to reach this goal, if you agree.
> >
> > In the meanwhile I've been implementing a raw
> EnhancerJob/EnhancerJobResult
> > implementation in order to enable the *commons.jobs* APIs to be used
> with
> > the *enhancer*, and I can share this work if you'd like.
>
> I will also need that at some point: I am working with people
> developing an opensource tool chain for machine translation (Arabic to
> french or English) using http://wapiti.limsi.fr/ and
> http://www.statmt.org/moses/ that we could wrap as a stanbol service
> (by wrapping the command-line programs with temporary input and output
> files). However as the translation process can be much slower than
> shallow nlp analysis like NER and document categorization and having
> the ability to run engines asynchronous is definitely a prerequisite
> for such an integration.
>
> So please feel free to attach your code as a patch to jira or to push
> it online as a fork of https://github.com/apache/stanbol so that we
> can have a look and give comments towards merging such a feature into
> stanbol/commons & stanbol/enhancer.
>
> --
> Olivier
> http://twitter.com/ogrisel - http://github.com/ogrisel
>
--
David Riccitelli
********************************************************************************
InsideOut10 s.r.l.
P.IVA: IT-11381771002
Fax: +39 0110708239
---
LinkedIn: http://it.linkedin.com/in/riccitelli
Twitter: ziodave
---
Layar Partner Network<http://www.layar.com/publishing/developers/list/?page=1&country=&city=&keyword=insideout10&lpn=1>
********************************************************************************
Re: asynchronous jobs
Posted by Olivier Grisel <ol...@ensta.org>.
2012/1/6 David Riccitelli <da...@insideout.io>:
> Hello guys,
>
> At the Zurich CreateJS Hackathon we started working on the asynchronous
> jobs for Stanbol. More specifically we need a REST API that allows us to:
> 1. start an enhancement job (passing its configuration data),
> 2. check its status and progress,
> 3. retrieve the results once the process is complete [1],
> 4. cancel an ongoing job.
>
> We saw recently the release of the long-term jobs for the Reasoning
> Services by Enrico Daga [2,3], and found that it can mostly satisfy the
> above mentioned requirements with the exception of monitoring the actual
> job progress.
>
> I talked with Rupert about it, and he's looking forward to the asynchronous
> operations OSGI postEvent/sendEvent methods, which would allow us to
> reshape the way the tasks are instantiated, their progress monitored and
> managed.
>
> I would like to define with you the requirements for the asynchronous jobs
> and a path in order to reach this goal, if you agree.
>
> In the meanwhile I've been implementing a raw EnhancerJob/EnhancerJobResult
> implementation in order to enable the *commons.jobs* APIs to be used with
> the *enhancer*, and I can share this work if you'd like.
I will also need that at some point: I am working with people
developing an opensource tool chain for machine translation (Arabic to
french or English) using http://wapiti.limsi.fr/ and
http://www.statmt.org/moses/ that we could wrap as a stanbol service
(by wrapping the command-line programs with temporary input and output
files). However as the translation process can be much slower than
shallow nlp analysis like NER and document categorization and having
the ability to run engines asynchronous is definitely a prerequisite
for such an integration.
So please feel free to attach your code as a patch to jira or to push
it online as a fork of https://github.com/apache/stanbol so that we
can have a look and give comments towards merging such a feature into
stanbol/commons & stanbol/enhancer.
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Re: asynchronous jobs
Posted by Rupert Westenthaler <ru...@gmail.com>.
Hi all
As some might know there was an IKS Hackathon about CreateJS (http://createjs.org/) last week in Zürich (http://lanyrd.com/2012/createjs-hackathon/).
While Stanbol played only a side role in this meet up David, Reto and me still used this come together to discuss about the currently ongoing extensions to the Stanbol Enhancer
especially
* STANBOL-414, STANBOL-431: Multipart ContentItems, Enhancement Chains, Enhancement Engines REST API
* STANBOL-46: Asynchronous Enhancements
I have documented - my interpretation - of the results such discussion within
http://stanbol.staging.apache.org/stanbol/docs/trunk/enhancer/STANBOL-414-specification.html
with the request for feedback and further discussion
best
Rupert
On 06.01.2012, at 16:48, David Riccitelli wrote:
> I forgot to send the reference links:
>
> 1. Asynchronous enhancements:
> https://issues.apache.org/jira/browse/STANBOL-46
> 2. Long term operations:
> https://issues.apache.org/jira/browse/STANBOL-343
> 3. Stanbol jobs: https://issues.apache.org/jira/browse/STANBOL-383
>
> D.
>
> On Fri, Jan 6, 2012 at 4:47 PM, David Riccitelli <da...@insideout.io> wrote:
>
>> Hello guys,
>>
>> At the Zurich CreateJS Hackathon we started working on the asynchronous
>> jobs for Stanbol. More specifically we need a REST API that allows us to:
>> 1. start an enhancement job (passing its configuration data),
>> 2. check its status and progress,
>> 3. retrieve the results once the process is complete [1],
>> 4. cancel an ongoing job.
>>
>> We saw recently the release of the long-term jobs for the Reasoning
>> Services by Enrico Daga [2,3], and found that it can mostly satisfy the
>> above mentioned requirements with the exception of monitoring the actual
>> job progress.
>>
>> I talked with Rupert about it, and he's looking forward to the
>> asynchronous operations OSGI postEvent/sendEvent methods, which would allow
>> us to reshape the way the tasks are instantiated, their progress monitored
>> and managed.
>>
>> I would like to define with you the requirements for the asynchronous jobs
>> and a path in order to reach this goal, if you agree.
>>
>> In the meanwhile I've been implementing a raw
>> EnhancerJob/EnhancerJobResult implementation in order to enable the *
>> commons.jobs* APIs to be used with the *enhancer*, and I can share this
>> work if you'd like.
>>
>> BR,
>>
>> David
>>
>> --
>> David Riccitelli
>>
>>
>> ********************************************************************************
>> InsideOut10 s.r.l.
>> P.IVA: IT-11381771002
>> Fax: +39 0110708239
>> ---
>> LinkedIn: http://it.linkedin.com/in/riccitelli
>> Twitter: ziodave
>> ---
>> Layar Partner Network<http://www.layar.com/publishing/developers/list/?page=1&country=&city=&keyword=insideout10&lpn=1>
>>
>> ********************************************************************************
>>
>>
>
>
> --
> David Riccitelli
>
> ********************************************************************************
> InsideOut10 s.r.l.
> P.IVA: IT-11381771002
> Fax: +39 0110708239
> ---
> LinkedIn: http://it.linkedin.com/in/riccitelli
> Twitter: ziodave
> ---
> Layar Partner Network<http://www.layar.com/publishing/developers/list/?page=1&country=&city=&keyword=insideout10&lpn=1>
> ********************************************************************************
Re: asynchronous jobs
Posted by David Riccitelli <da...@insideout.io>.
I forgot to send the reference links:
1. Asynchronous enhancements:
https://issues.apache.org/jira/browse/STANBOL-46
2. Long term operations:
https://issues.apache.org/jira/browse/STANBOL-343
3. Stanbol jobs: https://issues.apache.org/jira/browse/STANBOL-383
D.
On Fri, Jan 6, 2012 at 4:47 PM, David Riccitelli <da...@insideout.io> wrote:
> Hello guys,
>
> At the Zurich CreateJS Hackathon we started working on the asynchronous
> jobs for Stanbol. More specifically we need a REST API that allows us to:
> 1. start an enhancement job (passing its configuration data),
> 2. check its status and progress,
> 3. retrieve the results once the process is complete [1],
> 4. cancel an ongoing job.
>
> We saw recently the release of the long-term jobs for the Reasoning
> Services by Enrico Daga [2,3], and found that it can mostly satisfy the
> above mentioned requirements with the exception of monitoring the actual
> job progress.
>
> I talked with Rupert about it, and he's looking forward to the
> asynchronous operations OSGI postEvent/sendEvent methods, which would allow
> us to reshape the way the tasks are instantiated, their progress monitored
> and managed.
>
> I would like to define with you the requirements for the asynchronous jobs
> and a path in order to reach this goal, if you agree.
>
> In the meanwhile I've been implementing a raw
> EnhancerJob/EnhancerJobResult implementation in order to enable the *
> commons.jobs* APIs to be used with the *enhancer*, and I can share this
> work if you'd like.
>
> BR,
>
> David
>
> --
> David Riccitelli
>
>
> ********************************************************************************
> InsideOut10 s.r.l.
> P.IVA: IT-11381771002
> Fax: +39 0110708239
> ---
> LinkedIn: http://it.linkedin.com/in/riccitelli
> Twitter: ziodave
> ---
> Layar Partner Network<http://www.layar.com/publishing/developers/list/?page=1&country=&city=&keyword=insideout10&lpn=1>
>
> ********************************************************************************
>
>
--
David Riccitelli
********************************************************************************
InsideOut10 s.r.l.
P.IVA: IT-11381771002
Fax: +39 0110708239
---
LinkedIn: http://it.linkedin.com/in/riccitelli
Twitter: ziodave
---
Layar Partner Network<http://www.layar.com/publishing/developers/list/?page=1&country=&city=&keyword=insideout10&lpn=1>
********************************************************************************