You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by Leo Romanoff <ro...@yahoo.com> on 2010/04/18 09:49:19 UTC

Server-side asynchrony for SOAP web services in CXF

Hi,

Could someone provide an update of the current status of Server-side
asynchrony for SOAP web services in CXF? Which transports support it? How it
can be done?

As far as I understand from looking at various descriptions, JMS transport
supports it. 
Using WS-Adressing and providing a callback from the client-side that will
be called once server-side is ready should be also a no brainer.

But what about server-side asynchrony that happens purely on the
server-side, completely transparent for the client? Which transports support
it and how?

I've found some links about CXF continuations that can be used for that, but
it is not quite clear how to do that and what is the current status:
 https://issues.apache.org/jira/browse/CXF-2002
 http://sberyozkin.blogspot.com/2008/12/continuations-in-cxf.html
 http://www.mail-archive.com/dev@cxf.apache.org/msg00801.html

It would be also interesting to see how server-side asynchrony support in
CXF relates to the Sun's Metro SOAP stack support for the same future? There
there are some special interfaces like AsyncProvider and some alternative
proposals:
 http://www.docjar.com/docs/api/com/sun/xml/ws/api/server/AsyncProvider.html
 https://jax-ws.dev.java.net/issues/show_bug.cgi?id=787    (motivation part
is very interesting)

https://jax-ws.dev.java.net/nonav/issues/showattachment.cgi/266/ResponseDispatch.html

Basically, it is very important for many enterprise server-side services to
be able to handle tens and hundreds of thousends requests, where many of
them require a very long time for their completion (due to invoking other
services, doing DBs, etc). Obviously, thread-per-request model would not
work in such situations and would be inefficient anyway. Therefore,
something like continuations in CXF or Fibers/Tubes in Metro are most likely
required to solve the problem. Requests need to be mapped to tasks
representing them and those task should be scheduled in a controlled way on
thread pools with limited number of threads or something similar.
Additionally, some features like timeouts and persistence of such tasks for
long-running operations may be required. Having a common mechanism that is
(mostly) transport independent (HTTP, HTTPs, JMS, etc) and eventually
supports both SOAP and REST services (i.e. JAX-WS and JAX-RS) would be even
better, of course.

Even if server-side asynchrony is not supported out of the box, it would be
interesting to hear if there are any plans regarding it. In any case, I
would appriciate any advices on implementing scenarios similar to the
described above in a client-transparent scalable way (so, not WS-Addressing
or polling).


-- 
View this message in context: http://old.nabble.com/Server-side-asynchrony-for-SOAP-web-services-in-CXF-tp28280821p28280821.html
Sent from the cxf-user mailing list archive at Nabble.com.


Re: Server-side asynchrony for SOAP web services in CXF

Posted by Roman Levenstein <ro...@yahoo.com>.
Daniel, thanks for this very elaborative explanation!

On Thu, Apr 22, 2010 at 4:33 PM, Daniel Kulp <dk...@apache.org> wrote:
> On Wednesday 21 April 2010 10:35:12 am Roman Levenstein wrote:
>> > We've talked about adding a @UseAyncMethod(timeout = "###") annotation or
>> > similar to let the runtime know to use the async method sigs instead of
>> > the normal one and layer that on top of the continuations.   That would
>> > simplify things a lot.  Just haven't gotten around to implementing it.
>> >  That said, if layered on the continuations, it would have the same
>> > transport restrictions.
>>
>> Thanks for the explanation. What bothers me a bit, is that you are
>> very implementation centric (continuations, Servlets 3.0).
>> But what about the asynchrony support in the JSR spec?
>> What about compatibility with other JAX-WS implementations?
>
> Well, that would require updates and enhancements to the JAX-WS spec.  :-)

Yes. This may be the case. But even if support for asynchrony would
not be part of the standard (yet), having a "de facto" standard
supported by major Java SOAP stacks would be a big advantage for
developers and make their life easier.

> More in a sec...
>
>
>> Why I'm wondering about are two things:
>> 1) Why there is no co-operation into bringing up server-side
>> asynchrony issues at the JSR level?
>
> Well, when producing the JAX-WS spec, everything presented in the spec must be
> "doable" using the standard technologies and normal deployment environments.
> In the case of JAX-WS, the "normal" environment is to have the services
> deployed in a war with some sort of Servlet frontend.  (btw: the TCK runs by
> deploying wars in tomcat)    Thus, any  requirements of the spec MUST be
> implementable on that technology.

This is a very interesting comment. I'm not part of the JSR
standardization team and was not aware of this requirement.
This explains a lot of limitations present in some specifications.

> However, the Async stuff you describe is NOT implementable on a Servlet 2.x
> container.    When the servlet container calls the servlet, when the servlet
> returns, it assumes it has completed and will flush the response (and close
> the connection if no keep-alive).   Thus, there is no way for the servlet
> container to "wait" for a response via a Future object without consuming the
> request thread.   It's just not possible.   That is one main reason it's not
> addressed as part of the JAX-WS spec as it would not be implementable on
> current technology.

I see.

> In glassfish, Metro doesn't use a servlet and can hook into the HTTP stack
> furthur down and thus it works there.   However, if you deploy the same Async
> service as a war into tomcat, the async stuff would no longer work.

This is sad, that the least capable of all modern app. servers should
be taken as a common platform ;-(

> Basically, for any async stuff to work, there HAS to be support from the
> underlying transport for it to even be possible.  With Servlet 2.x, there
> isn't support for it.    With servlet 3.x, there is some support.  Maybe a
> future version of the spec would add support now that Servlet 3.x is final.
> I don't really know.

I hope so. With all the movement towards AJAX, Comet, long-lived HTTP
connections, ARP and recently WebSockets,
it would be nice to have a spec that is more up-to-date.

BTW, usage of servlets as a basis has different kinds of limitations:
 - The facts that it uses one thread per request approach means that
it most likely does not scale to serve let's say 50000 or 100000
simultaneous requests, because Java just does not support that many
threads. Of course, requests can be queued in this case.
- If request processing takes longer, because all threads are occupied
and wait for results of the IO (e.g. DB query or invocation of another
external WS) , then no new requests can be processed, even though the
CPU load is almost 0 and all threads are just waiting for IO results.
This of course does not contribute to a great scalability.

>>      Or at least working on a cross-implementation interoperability or
>> reuse of "user-space" code developed by JAX-WS users, when they want
>> to use the server-side asynchrony support. It would be very nice, if
>> the same code can be run on Metro, CXF, Axis and other JAX-WS
>> implementations. Today we have to write JAX-WS implementation specific
>> code.
>>
>> 2) Why asynchrony support is so low on the JAX-WS implementors agenda?
>
> See above.    A spec must be implementable or it's a pretty useless spec.
>
>>     I discovered massive problems in Metro and CXF when it comes to
>> this topic (both client-side and server-side).
>>     Does it mean that there are other ways for achieving
>> high-performance and scalable (e.g. able to support thousands of
>> simultaneous requests) server-side WS processing?
>
> JMS?

JMS is a good solution, no doubts. But this is not very Web friendly,
so to say.
First of all, it is a Java-only solution. Then it may problems with firewalls.
And invoking it from Web 2.0 apps is not so easy and straight forward, AFAIK.

The scalability of JMS is eventually also not so scalable, at least
when used inside a JavaEE server.
Standalone JMS-based apps probably have better performance.

BTW, I experimented a bit recently with a few JMS implementations
(usual OSS suspects). I got up-to 11000 requests per sec on a
commodity hardware (4 GB RAM, Dual Core). This is of course a way
better than what I get with JAX-WS on the same hardware: around 350
processed invocations/sec. But for our apps we need a much better
performance.

> Seriously, I think a semi-normal way of handling such stuff would involve
> either using more of a polling method or callback where a client would start a
> long running process and either pass a reference to receive a callback when
> complete or get back some sort of process ID that it could then invoke another
> "getResult(id)" method later to see if it's done and get the result.
>
> I guess it requires thinking of it more from a messaging standpoint instead of
> a request/reply standpoint, even on the client side.

In principle, I agree with you. I'd even say that for the systems that
are completely under your control, this would be a right way to go.
But it is important that clients should be aware of this architectural choice.
Now, if you have only legacy clients supporing only the old fashioned
WS invocations and written without callback or polling in mind? Or if
those client are all third-party and you cannot force them to use this
architectural pattern?
In such cases, having a better scalability transparently on the
server-side is almost the only solution.

>>     Or does it mean that there is not real interest in scaling WS
>> processing that high?
>>     Were any JAX-WS frameworks  used AT ALL for such kinds of scalable
>> processing? If so, do you have any references and performance figures?
>>     Or are these frameworks good only for mostly low- or medium-load
>> scenarios? Or may be I just expect too much from SOAP stacks and something
>> else should be used instead for really high-performance scenarios, e.g.
>> JMS, REST, ESBs (non SOAP based ;-) etc?
>
> Well, REST in Java would have much of the same problems.  If it's implemented
> via a servlet type thing, same issue.    Likewise for ESB if only dealing with
> HTTP.   JMS is possibly the right solution or using a ESB along with JMS.

If you are under JavaEE constraints, yes - you are right.

But recently many frameworks based on NIO or AIO were developed
(netty, etc). They can scale up to hundreds of thousands incoming
requests. There are also new languages like Scala, Clojure, Groovy or
frameworks like Kilim or JetLang. Almost all of them are not JavaEE
based and even ditch it, avoiding many of JavaEE limitations (control
over multi-threading, one thread per request model, etc),  and yet
they are using Java or JVM.

Solutions developed using such languages and frameworks also need ways
for using Web Services, be it SOAP or REST.
And they do not limit SOAP stacks via artificial constraints  like
JavaEE (no explicit thread mgmt, etc) and may require improved support
for scalability, asynchrony, light-weight threads and other
alternatives like actors, etc.

I'm a strong believer that some important and big JSRs (like JAX-WS,
JAX-RS), or at least WS stacks should try to liberate themselves from
the idea of being purely JavaEE based. The future may be somewhere
else (see above). And it is probably better for WS stacks to be part
of that future by means of supporting it, instead of just stating
that they are non JavaEE and therefore not part of the standard. I
think, this attitude may result in a situation, where many of those
new technologies due to the lack of support would introduce their own
solutions. Eventually these solutions would not be WS based at all,
thus eliminating the overall need for WS as such or significantely
reducing its popularity...

Just my 2 cents ;-)

Re: Server-side asynchrony for SOAP web services in CXF

Posted by Daniel Kulp <dk...@apache.org>.
On Wednesday 21 April 2010 10:35:12 am Roman Levenstein wrote:
> > We've talked about adding a @UseAyncMethod(timeout = "###") annotation or
> > similar to let the runtime know to use the async method sigs instead of
> > the normal one and layer that on top of the continuations.   That would
> > simplify things a lot.  Just haven't gotten around to implementing it.
> >  That said, if layered on the continuations, it would have the same
> > transport restrictions.
> 
> Thanks for the explanation. What bothers me a bit, is that you are
> very implementation centric (continuations, Servlets 3.0).
> But what about the asynchrony support in the JSR spec?
> What about compatibility with other JAX-WS implementations?

Well, that would require updates and enhancements to the JAX-WS spec.  :-)

More in a sec...


> Why I'm wondering about are two things:
> 1) Why there is no co-operation into bringing up server-side
> asynchrony issues at the JSR level?

Well, when producing the JAX-WS spec, everything presented in the spec must be 
"doable" using the standard technologies and normal deployment environments.   
In the case of JAX-WS, the "normal" environment is to have the services 
deployed in a war with some sort of Servlet frontend.  (btw: the TCK runs by 
deploying wars in tomcat)    Thus, any  requirements of the spec MUST be 
implementable on that technology.     

However, the Async stuff you describe is NOT implementable on a Servlet 2.x 
container.    When the servlet container calls the servlet, when the servlet 
returns, it assumes it has completed and will flush the response (and close 
the connection if no keep-alive).   Thus, there is no way for the servlet 
container to "wait" for a response via a Future object without consuming the 
request thread.   It's just not possible.   That is one main reason it's not 
addressed as part of the JAX-WS spec as it would not be implementable on 
current technology.

In glassfish, Metro doesn't use a servlet and can hook into the HTTP stack 
furthur down and thus it works there.   However, if you deploy the same Async 
service as a war into tomcat, the async stuff would no longer work.   

Basically, for any async stuff to work, there HAS to be support from the 
underlying transport for it to even be possible.  With Servlet 2.x, there 
isn't support for it.    With servlet 3.x, there is some support.  Maybe a 
future version of the spec would add support now that Servlet 3.x is final.   
I don't really know.

>      Or at least working on a cross-implementation interoperability or
> reuse of "user-space" code developed by JAX-WS users, when they want
> to use the server-side asynchrony support. It would be very nice, if
> the same code can be run on Metro, CXF, Axis and other JAX-WS
> implementations. Today we have to write JAX-WS implementation specific
> code.
> 
> 2) Why asynchrony support is so low on the JAX-WS implementors agenda?

See above.    A spec must be implementable or it's a pretty useless spec.

>     I discovered massive problems in Metro and CXF when it comes to
> this topic (both client-side and server-side).
>     Does it mean that there are other ways for achieving
> high-performance and scalable (e.g. able to support thousands of
> simultaneous requests) server-side WS processing?

JMS?   

Seriously, I think a semi-normal way of handling such stuff would involve 
either using more of a polling method or callback where a client would start a 
long running process and either pass a reference to receive a callback when 
complete or get back some sort of process ID that it could then invoke another 
"getResult(id)" method later to see if it's done and get the result.   

I guess it requires thinking of it more from a messaging standpoint instead of 
a request/reply standpoint, even on the client side.   

>     Or does it mean that there is not real interest in scaling WS
> processing that high?
>     Were any JAX-WS frameworks  used AT ALL for such kinds of scalable
> processing? If so, do you have any references and performance figures?
>     Or are these frameworks good only for mostly low- or medium-load
> scenarios? Or may be I just expect too much from SOAP stacks and something
> else should be used instead for really high-performance scenarios, e.g.
> JMS, REST, ESBs (non SOAP based ;-) etc?

Well, REST in Java would have much of the same problems.  If it's implemented 
via a servlet type thing, same issue.    Likewise for ESB if only dealing with 
HTTP.   JMS is possibly the right solution or using a ESB along with JMS.


-- 
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog

Re: Server-side asynchrony for SOAP web services in CXF

Posted by Roman Levenstein <ro...@yahoo.com>.
Hi Daniel,

Thanks for your reply!

On Wed, Apr 21, 2010 at 3:33 PM, Daniel Kulp <dk...@apache.org> wrote:
> Sergey and I have chatted off and on about some of this, but never really had
> time to put it to paper or start working on it.
>
> Currently, at the transport level, we support the continuations which, if the
> transport supports it, allows the service to suspend the processing on the
> request thread and pick it up later.   Thus, for long running processes, the
> servant can push it onto a workqueue or something and more or less suspend
> things and resume later.    It IS a bit complicated.  :-(    Also, not all
> transport support it.   Only JMS and HTTP if using the embedded Jetty.   HTTPs
> doesn't work (no continuation support for https with Jetty6, haven't checked
> jetty 7 yet) and not the servlet version either.  (new 3.0 servlet spec would
> allow it.   We haven't updated to the 3.0 version for our servlet yet).
>
> We've talked about adding a @UseAyncMethod(timeout = "###") annotation or
> similar to let the runtime know to use the async method sigs instead of the
> normal one and layer that on top of the continuations.   That would simplify
> things a lot.  Just haven't gotten around to implementing it.  That said, if
> layered on the continuations, it would have the same transport restrictions.

Thanks for the explanation. What bothers me a bit, is that you are
very implementation centric (continuations, Servlets 3.0).
But what about the asynchrony support in the JSR spec?
What about compatibility with other JAX-WS implementations?

> If you'd like to help out in this area, I'd be happy to provide pointers. :-)

Well, I'm afraid I cannot spend any time on these issues. Basically,
I'm the user of JAX-WS, not its implementor ;-)
To be frank, I looked at the CXF JAX-WS implementation as a fallback
solution. Many of our projects are based on Glassfish and therefore we
used Metro.
But after hitting certain limitations in Metro, when it comes to the
server-side asynchrony (and also client-side asynchrony),
I thought that may be another JAX-WS implementation can do it better.
So, I started looking into CXF.
But it turns out that the situation with CXF is not much better, if
not worse in this regard ;-)

Why I'm wondering about are two things:
1) Why there is no co-operation into bringing up server-side
asynchrony issues at the JSR level?
     Or at least working on a cross-implementation interoperability or
reuse of "user-space" code developed by JAX-WS users, when they want
to use the server-side asynchrony support. It would be very nice, if
the same code can be run on Metro, CXF, Axis and other JAX-WS
implementations. Today we have to write JAX-WS implementation specific
code.

2) Why asynchrony support is so low on the JAX-WS implementors agenda?
    I discovered massive problems in Metro and CXF when it comes to
this topic (both client-side and server-side).
    Does it mean that there are other ways for achieving
high-performance and scalable (e.g. able to support thousands of
simultaneous requests) server-side WS processing?
    Or does it mean that there is not real interest in scaling WS
processing that high?
    Were any JAX-WS frameworks  used AT ALL for such kinds of scalable
processing? If so, do you have any references and performance figures?
    Or are these frameworks good only for mostly low- or medium-load scenarios?
    Or may be I just expect too much from SOAP stacks and something
else should be used instead for really high-performance scenarios,
e.g. JMS, REST, ESBs (non SOAP based ;-) etc?

Thanks in advance for any feedback!

Re: Server-side asynchrony for SOAP web services in CXF

Posted by Daniel Kulp <dk...@apache.org>.
Sergey and I have chatted off and on about some of this, but never really had 
time to put it to paper or start working on it.

Currently, at the transport level, we support the continuations which, if the 
transport supports it, allows the service to suspend the processing on the 
request thread and pick it up later.   Thus, for long running processes, the 
servant can push it onto a workqueue or something and more or less suspend 
things and resume later.    It IS a bit complicated.  :-(    Also, not all 
transport support it.   Only JMS and HTTP if using the embedded Jetty.   HTTPs 
doesn't work (no continuation support for https with Jetty6, haven't checked 
jetty 7 yet) and not the servlet version either.  (new 3.0 servlet spec would 
allow it.   We haven't updated to the 3.0 version for our servlet yet).

We've talked about adding a @UseAyncMethod(timeout = "###") annotation or 
similar to let the runtime know to use the async method sigs instead of the 
normal one and layer that on top of the continuations.   That would simplify 
things a lot.  Just haven't gotten around to implementing it.  That said, if 
layered on the continuations, it would have the same transport restrictions.

If you'd like to help out in this area, I'd be happy to provide pointers.   
:-)

Dan


On Sunday 18 April 2010 3:49:19 am Leo Romanoff wrote:
> Hi,
> 
> Could someone provide an update of the current status of Server-side
> asynchrony for SOAP web services in CXF? Which transports support it? How
> it can be done?
> 
> As far as I understand from looking at various descriptions, JMS transport
> supports it.
> Using WS-Adressing and providing a callback from the client-side that will
> be called once server-side is ready should be also a no brainer.
> 
> But what about server-side asynchrony that happens purely on the
> server-side, completely transparent for the client? Which transports
> support it and how?
> 
> I've found some links about CXF continuations that can be used for that,
> but it is not quite clear how to do that and what is the current status:
> https://issues.apache.org/jira/browse/CXF-2002
>  http://sberyozkin.blogspot.com/2008/12/continuations-in-cxf.html
>  http://www.mail-archive.com/dev@cxf.apache.org/msg00801.html
> 
> It would be also interesting to see how server-side asynchrony support in
> CXF relates to the Sun's Metro SOAP stack support for the same future?
> There there are some special interfaces like AsyncProvider and some
> alternative proposals:
>  http://www.docjar.com/docs/api/com/sun/xml/ws/api/server/AsyncProvider.htm
> l https://jax-ws.dev.java.net/issues/show_bug.cgi?id=787    (motivation
> part is very interesting)
> 
> https://jax-ws.dev.java.net/nonav/issues/showattachment.cgi/266/ResponseDis
> patch.html
> 
> Basically, it is very important for many enterprise server-side services to
> be able to handle tens and hundreds of thousends requests, where many of
> them require a very long time for their completion (due to invoking other
> services, doing DBs, etc). Obviously, thread-per-request model would not
> work in such situations and would be inefficient anyway. Therefore,
> something like continuations in CXF or Fibers/Tubes in Metro are most
> likely required to solve the problem. Requests need to be mapped to tasks
> representing them and those task should be scheduled in a controlled way
> on thread pools with limited number of threads or something similar.
> Additionally, some features like timeouts and persistence of such tasks for
> long-running operations may be required. Having a common mechanism that is
> (mostly) transport independent (HTTP, HTTPs, JMS, etc) and eventually
> supports both SOAP and REST services (i.e. JAX-WS and JAX-RS) would be even
> better, of course.
> 
> Even if server-side asynchrony is not supported out of the box, it would be
> interesting to hear if there are any plans regarding it. In any case, I
> would appriciate any advices on implementing scenarios similar to the
> described above in a client-transparent scalable way (so, not WS-Addressing
> or polling).

-- 
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog

Re: Server-side asynchrony for SOAP web services in CXF

Posted by Leo Romanoff <ro...@yahoo.com>.
Anyone has an idea on this subject?

-- 
View this message in context: http://old.nabble.com/Server-side-asynchrony-for-SOAP-web-services-in-CXF-tp28280821p28288007.html
Sent from the cxf-user mailing list archive at Nabble.com.


Re: Server-side asynchrony for SOAP web services in CXF

Posted by Glen Mazza <gl...@gmail.com>.
The fact that you're drawing silence on this list and Metro's list may
indicate you're thinking of this problem differently from most others.  I've
been on both lists off-and-on for a few years and you're the only one I've
seen so far bringing up this issue.

A web service provider, like a web application, is really just a servlet
sitting in a servlet container.  Are not your synchronicity issues better
handled at the servlet container level, just as it would for a web
application?  You have a servlet (whatever CXF or Metro uses
behind-the-scenes) and you want to know how best to have it handle 1,000's
of accesses--this sounds like something best handled on the Tomcat user's
list.

All I think that you're asking is, for a service implementation bean method,
such as this:

public int doSomething(int inValue) {
   return 10 * inValue;
}

is how to launch a new thread within this SIB method to do my processing of
doSomething() on a separate thread?  I do not know the consequences of doing
your multithreading this way but wonder if it would be actually better to
start thinking "how do I get the servlet container to clone (or provide
enough threads for) the servlet activating this SIB so it can efficiently
handle all the requests I am expecting for it?

Stated another way, I suspect the reason why your question is not occurring
too often on the CXF and Metro lists is that most people in your situation
think in terms of the servlet container rather than the web service call for
its effective solution.

Glen


Leo Romanoff wrote:
> 
> Any information about the subject?
> 

-- 
View this message in context: http://old.nabble.com/Server-side-asynchrony-for-SOAP-web-services-in-CXF-tp28280821p28302371.html
Sent from the cxf-user mailing list archive at Nabble.com.


Re: Server-side asynchrony for SOAP web services in CXF

Posted by Leo Romanoff <ro...@yahoo.com>.
Any information about the subject?
-- 
View this message in context: http://old.nabble.com/Server-side-asynchrony-for-SOAP-web-services-in-CXF-tp28280821p28288010.html
Sent from the cxf-user mailing list archive at Nabble.com.


Re: Server-side asynchrony for SOAP web services in CXF

Posted by Leo Romanoff <ro...@yahoo.com>.
Anyone has an idea on this subject?
-- 
View this message in context: http://old.nabble.com/Server-side-asynchrony-for-SOAP-web-services-in-CXF-tp28280821p28287830.html
Sent from the cxf-user mailing list archive at Nabble.com.