You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Oliver Lietz <ap...@oliverlietz.de> on 2020/08/11 12:11:34 UTC
[DISCUSS] New Content API/Modules
Hi *,
I would like to discuss new API and modules for content analyzing and
processing. See below:
org.apache.sling.commons.content.analyzing.Analyzer
Future<Void> analyze(InputStream in, Map<String, Object> report) throws
AnalyzingException
org.apache.sling.commons.content.processing.Processor
Future<Void> process(InputStream in, OutputStream out, Map<String, Object>
report) throws ProcessingException
The idea is to create an Anti-Virus/Malware API which also supports async
malware scan engines and retrofit the existing Commons Clam API *and* also
have a general API for other modules working on content like compressors or
content detection.
WDYT?
Regards,
O.
Re: [DISCUSS] New Content API/Modules
Posted by Paul Bjorkstrand <pa...@gmail.com>.
Glad to see that :-) I am a big fan of the promise-like feel of
CompletableFuture (at least until Project Loom [1] is complete).
-Paul
[1]: https://openjdk.java.net/projects/loom/
On Tue, Aug 11, 2020 at 1:35 PM Oliver Lietz <ap...@oliverlietz.de> wrote:
> On Tuesday, August 11, 2020 6:42:32 PM CEST Paul Bjorkstrand wrote:
> > If I may suggest, use CompletableFuture instead of Future. IMO it is a
> more
> > natural API, and helps to steer people away from using the blocking
> method
> > Future.get(). It's hard enough to ensure features are developed properly
> > for a multithreaded application without adding in thread-blocking "async"
> > operations. (This assumes you were intending to use Future as a means of
> > denoting an asynchronous operation.)
> >
> > A side benefit: you may be able to get away with not building a custom
> > pipeline API, and just make methods that return a CompletionStage. The
> > benefits of this are questionable, but it may help with prototyping or
> > implementing under the covers.
>
> Thanks, Paul! Completable just got lost somewhere. I had CompletableFuture
> in
> my mind as I'm quite happy with what we have in Commons Messaging Mail:
>
>
> https://github.com/apache/sling-org-apache-sling-commons-messaging-mail/blob/
>
> master/src/main/java/org/apache/sling/commons/messaging/mail/MailService.java
> <https://github.com/apache/sling-org-apache-sling-commons-messaging-mail/blob/master/src/main/java/org/apache/sling/commons/messaging/mail/MailService.java>
>
> Regards,
> O.
>
>
> > -Paul
> >
> >
> > On Tue, Aug 11, 2020 at 8:07 AM Bertrand Delacretaz <
> bdelacretaz@apache.org>
> > wrote:
> > > Hi,
> > >
> > > On Tue, Aug 11, 2020 at 2:11 PM Oliver Lietz <ap...@oliverlietz.de>
> > >
> > > wrote:
> > > > ...I would like to discuss new API and modules for content analyzing
> and
> > > > processing...
> > >
> > > I think it's often useful to be able to chain such things in a similar
> > > way to Unix Pipes.
> > >
> > > Which might speak for an API like
> > >
> > > FilePipeStatus s = FilePipe.
> > >
> > > .withProcessor(grep)
> > > .withProcessor(sed)
> > > .withProcessor(antivirus)
> > > .withInput(someInputStream)
> > > .withOutput(someOutputStream)
> > > .start()
> > >
> > > WhereFilePipeStatus provides access to the completion Future and to
> > > the processing report, which might also be modeled along the lines of
> > > standard output / error.
> > >
> > > Just my 2 cents - but I think a pipeline model can be useful here (and
> > > we already have something named Sling Pipes)
> > >
> > > -Bertrand
>
>
>
>
>
Re: [DISCUSS] New Content API/Modules
Posted by Oliver Lietz <ap...@oliverlietz.de>.
On Tuesday, August 11, 2020 6:42:32 PM CEST Paul Bjorkstrand wrote:
> If I may suggest, use CompletableFuture instead of Future. IMO it is a more
> natural API, and helps to steer people away from using the blocking method
> Future.get(). It's hard enough to ensure features are developed properly
> for a multithreaded application without adding in thread-blocking "async"
> operations. (This assumes you were intending to use Future as a means of
> denoting an asynchronous operation.)
>
> A side benefit: you may be able to get away with not building a custom
> pipeline API, and just make methods that return a CompletionStage. The
> benefits of this are questionable, but it may help with prototyping or
> implementing under the covers.
Thanks, Paul! Completable just got lost somewhere. I had CompletableFuture in
my mind as I'm quite happy with what we have in Commons Messaging Mail:
https://github.com/apache/sling-org-apache-sling-commons-messaging-mail/blob/
master/src/main/java/org/apache/sling/commons/messaging/mail/MailService.java
Regards,
O.
> -Paul
>
>
> On Tue, Aug 11, 2020 at 8:07 AM Bertrand Delacretaz <bd...@apache.org>
> wrote:
> > Hi,
> >
> > On Tue, Aug 11, 2020 at 2:11 PM Oliver Lietz <ap...@oliverlietz.de>
> >
> > wrote:
> > > ...I would like to discuss new API and modules for content analyzing and
> > > processing...
> >
> > I think it's often useful to be able to chain such things in a similar
> > way to Unix Pipes.
> >
> > Which might speak for an API like
> >
> > FilePipeStatus s = FilePipe.
> >
> > .withProcessor(grep)
> > .withProcessor(sed)
> > .withProcessor(antivirus)
> > .withInput(someInputStream)
> > .withOutput(someOutputStream)
> > .start()
> >
> > WhereFilePipeStatus provides access to the completion Future and to
> > the processing report, which might also be modeled along the lines of
> > standard output / error.
> >
> > Just my 2 cents - but I think a pipeline model can be useful here (and
> > we already have something named Sling Pipes)
> >
> > -Bertrand
Re: [DISCUSS] New Content API/Modules
Posted by Paul Bjorkstrand <pa...@gmail.com>.
If I may suggest, use CompletableFuture instead of Future. IMO it is a more
natural API, and helps to steer people away from using the blocking method
Future.get(). It's hard enough to ensure features are developed properly
for a multithreaded application without adding in thread-blocking "async"
operations. (This assumes you were intending to use Future as a means of
denoting an asynchronous operation.)
A side benefit: you may be able to get away with not building a custom
pipeline API, and just make methods that return a CompletionStage. The
benefits of this are questionable, but it may help with prototyping or
implementing under the covers.
-Paul
On Tue, Aug 11, 2020 at 8:07 AM Bertrand Delacretaz <bd...@apache.org>
wrote:
> Hi,
>
> On Tue, Aug 11, 2020 at 2:11 PM Oliver Lietz <ap...@oliverlietz.de>
> wrote:
> > ...I would like to discuss new API and modules for content analyzing and
> > processing...
>
> I think it's often useful to be able to chain such things in a similar
> way to Unix Pipes.
>
> Which might speak for an API like
>
> FilePipeStatus s = FilePipe.
> .withProcessor(grep)
> .withProcessor(sed)
> .withProcessor(antivirus)
> .withInput(someInputStream)
> .withOutput(someOutputStream)
> .start()
>
> WhereFilePipeStatus provides access to the completion Future and to
> the processing report, which might also be modeled along the lines of
> standard output / error.
>
> Just my 2 cents - but I think a pipeline model can be useful here (and
> we already have something named Sling Pipes)
>
> -Bertrand
>
Re: [DISCUSS] New Content API/Modules
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Tue, Aug 11, 2020 at 2:11 PM Oliver Lietz <ap...@oliverlietz.de> wrote:
> ...I would like to discuss new API and modules for content analyzing and
> processing...
I think it's often useful to be able to chain such things in a similar
way to Unix Pipes.
Which might speak for an API like
FilePipeStatus s = FilePipe.
.withProcessor(grep)
.withProcessor(sed)
.withProcessor(antivirus)
.withInput(someInputStream)
.withOutput(someOutputStream)
.start()
WhereFilePipeStatus provides access to the completion Future and to
the processing report, which might also be modeled along the lines of
standard output / error.
Just my 2 cents - but I think a pipeline model can be useful here (and
we already have something named Sling Pipes)
-Bertrand
Re: [DISCUSS] New Content API/Modules
Posted by Oliver Lietz <ap...@oliverlietz.de>.
On Tuesday, August 11, 2020 2:11:34 PM CEST Oliver Lietz wrote:
> Hi *,
>
> I would like to discuss new API and modules for content analyzing and
> processing. See below:
>
> org.apache.sling.commons.content.analyzing.Analyzer
>
> Future<Void> analyze(InputStream in, Map<String, Object> report) throws
> AnalyzingException
>
> org.apache.sling.commons.content.processing.Processor
>
> Future<Void> process(InputStream in, OutputStream out, Map<String,
> Object> report) throws ProcessingException
>
> The idea is to create an Anti-Virus/Malware API which also supports async
> malware scan engines and retrofit the existing Commons Clam API *and* also
> have a general API for other modules working on content like compressors or
> content detection.
For the initial API see:
https://github.com/apache/sling-org-apache-sling-commons-content-analyzing
https://github.com/apache/sling-org-apache-sling-commons-content-processing
and ClamdService implementing ContentAnalyzer:
https://github.com/apache/sling-org-apache-sling-commons-clam
O.
> WDYT?
>
> Regards,
> O.