REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Hi to all Apache CXF users!,

I would kindly like to bring to your attention a particular problem that we are currently being stuck into. We are experimenting with a simple web application, which exposes its functionality via REST. We are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as REST frameworks.

For calculating our performance measurements, we simply execute REST requests, and in the client we measure the time that the roundtrip takes. Apart from that, on the server side, we measure as well the time that our REST resource consumes, i.e. the method call time. In summary, we have some huge differences between CXF and Jersey - in favor of Jersey being much faster - in the cases where we test against non-existing resources. In all other cases, the numbers we got were really very close for the two REST frameworks.

But when testing against non-existing REST paths, the roundtrip time in CXF case is approximately 0.5 seconds, while in the case of Jersey it is 0.01 seconds. We think that this is quite strange, and probably we are not using CXF in its proper configuration. Could you please give us some hints ? Maybe we have to tune and tweak it a little bit, so that we get the best out of it.

The server side time, in the case of CXF, is rather small - 0.001 seconds, so we wonder where is that half second spent in ? In our application logic, in the case when the path is not correct, we throw javax.ws.rs.WebApplicationException and with a debugger we saw that this exception is getting caught in CXF, and heavily processed further down the stack.

Are you aware of such kind of issues, with CXF handling the 404 situation in a very heavyweight manner ? Maybe it does some thorough processing to try to map the URI to some existing REST resource, I don't know ... maybe performance here could depend on the particular JSON framework that is being used. ...


Kindest Regards,
Krum.

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi Krum
On 22/02/12 09:35, Bakalsky, Krum wrote:
> Hi Sergey,
>
> I have just made a test run with the modified WebApplicationException mapper that you have given a link to below, and it works fine: it also shows 10 times decrease of roundtrip duration as well. So, it seems that you have really optimized it pretty good already.
>
ok, cool,
> Sure, I don't have any particular objections or concerns as to how to approach the change that we are talking about. I'd guess that as long as the application has a way to configure its own exception mapper (which is already the case) so that 404 overhead is gone for it, things are fine. I wouldn't know for those implications and complications that adding a property could lead to. Thanks for sharing.
>
> We haven't identified particular 'slow' code fragments, other than the 404 processing. If we do, we will sure let you know. (or if you want, you could share your suspicions and give us some hint of where to take a look at ?)
>
I was aware of the particular issue where the active request path was 
recalculated few times instead of caching the calculated value on the 
current message, I don't think it was a major issue, but I finally got 
it resolved yesterday, inspired by all the conversation about the 
performance :-)
We've had a few good relevant patches provided by KL before so the 
'main' path should do pretty well, we can obviously put more attention 
into the individual edge or regular cases if something is spotted,

thanks, Sergey

> Kindest Regards,
> Krum.
>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Tuesday, February 21, 2012 8:34 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi Krum,
> On 20/02/12 16:47, Bakalsky, Krum wrote:
>> Hi,
>>
>> Yes, thanks for the comments, they sound very logical to explaining the numbers from all the variants we tested.
>> Didn't know that the default exception mapper was developed as a result of a user demand as well.
>>
>> For the record: how could I disable it ? I am registering the Simple mapper within the returned set of the Application#getClasses() method call - so maybe I should remove it there, i.e. from the super.getClasses() list, or it is in a different manner ?
>>
>> Yes, I think that a servlet context parameter would be the most flexible and elegant solution, for configuring the desired behavior for non-200/exceptional cases.
>>
> I've spent some today on optimizing a couple of things,
> see the updated default WebApplicationException mapper:
>
> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>
> I think it's more effective now, yes still it is configurable and able
> to interact with CXF FaultLoggers and optionally fine-log the exception
> message.
>
> I was thinking yesterday of adding a property to get this default mapper
> ignored. I'm not sure now it's worth going this route.
>
> One way to 'disable' it is to offer a custom WebApplicationException
> mapper, something which we've explored. Another one - is to add this
> would be property but I've realized that it's not that good for CXF
> after all:  this would require the CXF servlet to process unmapped
> WebApplicationExceptions to be compliant and this would effectively mean
> repeating at the servlet level what is done at the CXF chain level but
> with limitations, ex, the Response produced by the mapper would still be
> routed through the out filters and indeed the optional CXF out faulty
> interceptors which can be handy. Say we can have an out filter which
> time-stamps all the responses - this would be ignored at the servlet
> level and would effectively have to be duplicated if needed...
>
> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
> will not be necessarily installed in some OSGI containers by default.
>
> So it appears to be the best way is to optimize the default mapper which
> I hope I did, and it would be always possible to offer custom mappers
> instead on per-Application basis.
>
> Let me know please if you have any concerns with the above,
>
> In meantime I also updated some code on the in chain to optimize the way
> the HTTP path to match is calculated - it may well save us few
> milliseconds :-)
>
> If you can share some more results from your tests which point to few
> unoptimized code spots then it would be good; we are going to review few
> code paths anyway,
>
> Thanks, Sergey
>
>>
>> Kindest Regards and many thanks again,
>> Krum.
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Monday, February 20, 2012 5:49 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi,
>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>> Hi Krum
>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>> Hi Sergey,
>>>>
>>>> Wow! I tested your idea and it produced dramatically better results!
>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>> investigation, and I believe that our observation is really true that
>>>> the overhead is spent in the error handling code. Just out of
>>>> curiosity: with the same SimpleApplicationMapper:
>>>>
>>> Thanks for the confirmation, so we've identified the spot where the
>>> problem is :-)
>>>
>>>> public class SimpleApplicationExceptionMapper implements
>>>> ExceptionMapper<WebApplicationException>   {
>>>>
>>>> @Override
>>>> public Response toResponse(WebApplicationException exception) {
>>>> return Response.status(404).build();
>>>> }
>>>>
>>>> }
>>>>
>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>
>>>>
>>>> However, without this class, Jersey performed even faster - most of
>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>
>>
>> Forgot to comment to this one... I think the fact that both frameworks
>> show approximately the same time with SimpleApplicationExceptionMapper
>> in place and that Jersey scales really well without it is that Jersey
>> basically either writes  to HTTP out stream or propagates the exception
>> up to the ServletException filter immediately if no custom mapper is
>> available.
>> In case of CXF the immediate propagation will also happen if no custom
>> mapper is available. The reason the default WebApplicationException
>> Mapper was installed was to do with the fact that few users were not
>> quite happy with having to write their custom mappers in order to report
>> differently to the exceptions generated at the runtime level or some of
>> the providers' level or indeed having to write a filter catching the
>> escaped exceptions there...
>>
>> What I might need to do additionally is to introduce a contextual
>> property that will tell the runtime to ignore even the default
>> WebApplicationException mapper, for example, when the user wishes to
>> get all the exceptions propagated to the filter level, to simplify the
>> performance data comparison, etc
>>
>> Cheers, Sergey
>>
>>>>
>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>> where the discrepancies come from.
>>>>
>>> I've opened
>>> https://issues.apache.org/jira/browse/CXF-4121
>>>
>>> I suspect now that it is the specific lines which get the basic warning
>>> message from the resource bundle are main 'culprits', I guess I will
>>> just have an internal final static String - will need to experiment a
>>> bit...It is good even the default providers can be customized/replaced,
>>> but I'll look into trying to make the default exception mapper
>>> performing a bit better :-)
>>>
>>> Thanks, Sergey
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>> To: users@cxf.apache.org
>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi Krum,
>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>> Hi to all Apache CXF users!,
>>>>>
>>>>> I would kindly like to bring to your attention a particular problem
>>>>> that we are currently being stuck into. We are experimenting with a
>>>>> simple web application, which exposes its functionality via REST. We
>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>> REST frameworks.
>>>>>
>>>>> For calculating our performance measurements, we simply execute REST
>>>>> requests, and in the client we measure the time that the roundtrip
>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>> favor of Jersey being much faster - in the cases where we test
>>>>> against non-existing resources. In all other cases, the numbers we
>>>>> got were really very close for the two REST frameworks.
>>>>>
>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>> we are not using CXF in its proper configuration. Could you please
>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>> so that we get the best out of it.
>>>>>
>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>> application logic, in the case when the path is not correct, we throw
>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>> this exception is getting caught in CXF, and heavily processed
>>>>> further down the stack.
>>>>>
>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>> processing to try to map the URI to some existing REST resource, I
>>>>> don't know ... maybe performance here could depend on the particular
>>>>> JSON framework that is being used. ...
>>>>>
>>>>
>>>> I suspect it could be the default WebApplicationExceptionMapper which is
>>>> to blame.
>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>> any other regular response, but I'm not sure it would explain the
>>>> difference.
>>>>
>>>> Can you give me a favor please and register a custom
>>>> WebApplicationException mapper which would only return:
>>>>
>>>> Response.status(exception.getStatus()).build() ?
>>>>
>>>> Have a look please at the default mapper's code:
>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>
>>>>
>>>> As you can see, we are trying to go to some length there to figure out
>>>> what and how to report a given exception and I reckon it all adds up to
>>>> the response time.
>>>>
>>>> Please try eliminating the default mapper and let us know the results
>>>>
>>>> Thanks, Sergey
>>>>
>>>>>
>>>>> Kindest Regards,
>>>>> Krum.
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

RE: REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Hi Sergey,

I have just made a test run with the modified WebApplicationException mapper that you have given a link to below, and it works fine: it also shows 10 times decrease of roundtrip duration as well. So, it seems that you have really optimized it pretty good already.

Sure, I don't have any particular objections or concerns as to how to approach the change that we are talking about. I'd guess that as long as the application has a way to configure its own exception mapper (which is already the case) so that 404 overhead is gone for it, things are fine. I wouldn't know for those implications and complications that adding a property could lead to. Thanks for sharing.

We haven't identified particular 'slow' code fragments, other than the 404 processing. If we do, we will sure let you know. (or if you want, you could share your suspicions and give us some hint of where to take a look at ?)

Kindest Regards,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:sberyozkin@gmail.com] 
Sent: Tuesday, February 21, 2012 8:34 PM
To: users@cxf.apache.org
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi Krum,
On 20/02/12 16:47, Bakalsky, Krum wrote:
> Hi,
>
> Yes, thanks for the comments, they sound very logical to explaining the numbers from all the variants we tested.
> Didn't know that the default exception mapper was developed as a result of a user demand as well.
>
> For the record: how could I disable it ? I am registering the Simple mapper within the returned set of the Application#getClasses() method call - so maybe I should remove it there, i.e. from the super.getClasses() list, or it is in a different manner ?
>
> Yes, I think that a servlet context parameter would be the most flexible and elegant solution, for configuring the desired behavior for non-200/exceptional cases.
>
I've spent some today on optimizing a couple of things,
see the updated default WebApplicationException mapper:

http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java

I think it's more effective now, yes still it is configurable and able 
to interact with CXF FaultLoggers and optionally fine-log the exception 
message.

I was thinking yesterday of adding a property to get this default mapper 
ignored. I'm not sure now it's worth going this route.

One way to 'disable' it is to offer a custom WebApplicationException 
mapper, something which we've explored. Another one - is to add this 
would be property but I've realized that it's not that good for CXF 
after all:  this would require the CXF servlet to process unmapped 
WebApplicationExceptions to be compliant and this would effectively mean 
repeating at the servlet level what is done at the CXF chain level but 
with limitations, ex, the Response produced by the mapper would still be 
routed through the out filters and indeed the optional CXF out faulty 
interceptors which can be handy. Say we can have an out filter which 
time-stamps all the responses - this would be ignored at the servlet 
level and would effectively have to be duplicated if needed...

The other thing is that CXFNonSpringJaxrsServlet and servlet filters 
will not be necessarily installed in some OSGI containers by default.

So it appears to be the best way is to optimize the default mapper which 
I hope I did, and it would be always possible to offer custom mappers 
instead on per-Application basis.

Let me know please if you have any concerns with the above,

In meantime I also updated some code on the in chain to optimize the way 
the HTTP path to match is calculated - it may well save us few 
milliseconds :-)

If you can share some more results from your tests which point to few 
unoptimized code spots then it would be good; we are going to review few 
code paths anyway,

Thanks, Sergey

>
> Kindest Regards and many thanks again,
> Krum.
>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Monday, February 20, 2012 5:49 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi,
> On 20/02/12 15:28, Sergey Beryozkin wrote:
>> Hi Krum
>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>> Hi Sergey,
>>>
>>> Wow! I tested your idea and it produced dramatically better results!
>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>> investigation, and I believe that our observation is really true that
>>> the overhead is spent in the error handling code. Just out of
>>> curiosity: with the same SimpleApplicationMapper:
>>>
>> Thanks for the confirmation, so we've identified the spot where the
>> problem is :-)
>>
>>> public class SimpleApplicationExceptionMapper implements
>>> ExceptionMapper<WebApplicationException>  {
>>>
>>> @Override
>>> public Response toResponse(WebApplicationException exception) {
>>> return Response.status(404).build();
>>> }
>>>
>>> }
>>>
>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>
>>>
>>> However, without this class, Jersey performed even faster - most of
>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>> the CXF case ... you already know it :) - 0.500 seconds
>>>
>
> Forgot to comment to this one... I think the fact that both frameworks
> show approximately the same time with SimpleApplicationExceptionMapper
> in place and that Jersey scales really well without it is that Jersey
> basically either writes  to HTTP out stream or propagates the exception
> up to the ServletException filter immediately if no custom mapper is
> available.
> In case of CXF the immediate propagation will also happen if no custom
> mapper is available. The reason the default WebApplicationException
> Mapper was installed was to do with the fact that few users were not
> quite happy with having to write their custom mappers in order to report
> differently to the exceptions generated at the runtime level or some of
> the providers' level or indeed having to write a filter catching the
> escaped exceptions there...
>
> What I might need to do additionally is to introduce a contextual
> property that will tell the runtime to ignore even the default
> WebApplicationException mapper, for example, when the user wishes to
> get all the exceptions propagated to the filter level, to simplify the
> performance data comparison, etc
>
> Cheers, Sergey
>
>>>
>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>> where the discrepancies come from.
>>>
>> I've opened
>> https://issues.apache.org/jira/browse/CXF-4121
>>
>> I suspect now that it is the specific lines which get the basic warning
>> message from the resource bundle are main 'culprits', I guess I will
>> just have an internal final static String - will need to experiment a
>> bit...It is good even the default providers can be customized/replaced,
>> but I'll look into trying to make the default exception mapper
>> performing a bit better :-)
>>
>> Thanks, Sergey
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Friday, February 17, 2012 6:14 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi Krum,
>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>> Hi to all Apache CXF users!,
>>>>
>>>> I would kindly like to bring to your attention a particular problem
>>>> that we are currently being stuck into. We are experimenting with a
>>>> simple web application, which exposes its functionality via REST. We
>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>> REST frameworks.
>>>>
>>>> For calculating our performance measurements, we simply execute REST
>>>> requests, and in the client we measure the time that the roundtrip
>>>> takes. Apart from that, on the server side, we measure as well the
>>>> time that our REST resource consumes, i.e. the method call time. In
>>>> summary, we have some huge differences between CXF and Jersey - in
>>>> favor of Jersey being much faster - in the cases where we test
>>>> against non-existing resources. In all other cases, the numbers we
>>>> got were really very close for the two REST frameworks.
>>>>
>>>> But when testing against non-existing REST paths, the roundtrip time
>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>> we are not using CXF in its proper configuration. Could you please
>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>> so that we get the best out of it.
>>>>
>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>> seconds, so we wonder where is that half second spent in ? In our
>>>> application logic, in the case when the path is not correct, we throw
>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>> this exception is getting caught in CXF, and heavily processed
>>>> further down the stack.
>>>>
>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>> processing to try to map the URI to some existing REST resource, I
>>>> don't know ... maybe performance here could depend on the particular
>>>> JSON framework that is being used. ...
>>>>
>>>
>>> I suspect it could be the default WebApplicationExceptionMapper which is
>>> to blame.
>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>> any other regular response, but I'm not sure it would explain the
>>> difference.
>>>
>>> Can you give me a favor please and register a custom
>>> WebApplicationException mapper which would only return:
>>>
>>> Response.status(exception.getStatus()).build() ?
>>>
>>> Have a look please at the default mapper's code:
>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>
>>>
>>> As you can see, we are trying to go to some length there to figure out
>>> what and how to report a given exception and I reckon it all adds up to
>>> the response time.
>>>
>>> Please try eliminating the default mapper and let us know the results
>>>
>>> Thanks, Sergey
>>>
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>>
>>>
>>
>>
>
>

RE: REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Hi Sergey,

The client side measurements are taken as follows:

                WebConversation conversation = new WebConversation();
                conversation.setExceptionsThrownOnErrorStatus(false);

                long start = System.currentTimeMillis();

                WebRequest request = new GetMethodWebRequest(url);
                WebResponse response = conversation.getResponse(request);

                long end = System.currentTimeMillis();

The classes that you see here are from com.meterware.httpunit.

The time which I am referring to being too big in the 404 case, is exactly the (end - start) value (the 0.500 seconds time).

For the server side measurements, we just do something similar, but within the methods of our REST resources. If you recall my very first mail on this thread, they showed to be negligibly small - around 0.001 seconds. This was the reason why we thought that the CXF processing could be taking that long.


Kindest Regards,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
Sent: Wednesday, February 22, 2012 4:19 PM
To: users@cxf.apache.org
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi Krum,

Before commenting further, let me ask you how do you take the
measurements ? Do you check on the client side ? On the server side ?

Thanks, Sergey


On 22/02/12 13:49, Bakalsky, Krum wrote:
> Hi Sergey, again,
>
> I am afraid that we are in the middle of some big misunderstanding, me being the one who has started it. Unfortunately, it turned out that I didn't quite realize, that javax.ws.rs.core.Application#getClasses() returns a non-modifiable collection (Collections.emptySet()), and in my exception mapper I used:
>
>          public Set<Class<?>>  getClasses() {
>                  // TODO Auto-generated method stub
>                  Set<Class<?>>  classes = super.getClasses();
>                  classes.add(SimpleApplicationExceptionMapper.class);
>                  return classes;
>          }
>
> which probably leads to an java.lang.UnsupportedOperationException at runtime, and thus to some work flow/behavior that I am not aware of. This made mu numbers faster, but obviously the scenario is broken, so the results are irrelevant.
>
> Next, I took a look at some examples, like this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>  one, and modified my code accordingly. Unfortunately, I got the same results: 0.500 seconds for the 404 cases. I am not even sure that my exception mapper was really used, since when I inserted some dumps, and even when I threw some exceptions in my toResponse() method, I didn't see them nowhere.
>
> Next, I made a step further, and patched the WebApplicationExceptionMapper that comes with CXF to look like this:
>
> public class WebApplicationExceptionMapper implements ExceptionMapper<WebApplicationException>  {
>
>          public Response toResponse(WebApplicationException ex) {
>                  return ex.getResponse();
>          }
> }
>
> I made sure that it was really used. Unfortunately, I got the same results: roundtrip time was again ~0.500 seconds.
>
> I am really not sure whether the CXF-4121 bug makes sense any more. Problem is still here ;(
>
>
> Kindest Regards,
> Krum.
>
> -----Original Message-----
> From: Bakalsky, Krum
> Sent: Wednesday, February 22, 2012 11:35 AM
> To: 'users@cxf.apache.org'
> Subject: RE: REST performance discrepancies between CXF and Jersey
>
> Hi Sergey,
>
> I have just made a test run with the modified WebApplicationException mapper that you have given a link to below, and it works fine: it also shows 10 times decrease of roundtrip duration as well. So, it seems that you have really optimized it pretty good already.
>
> Sure, I don't have any particular objections or concerns as to how to approach the change that we are talking about. I'd guess that as long as the application has a way to configure its own exception mapper (which is already the case) so that 404 overhead is gone for it, things are fine. I wouldn't know for those implications and complications that adding a property could lead to. Thanks for sharing.
>
> We haven't identified particular 'slow' code fragments, other than the 404 processing. If we do, we will sure let you know. (or if you want, you could share your suspicions and give us some hint of where to take a look at ?)
>
> Kindest Regards,
> Krum.
>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Tuesday, February 21, 2012 8:34 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi Krum,
> On 20/02/12 16:47, Bakalsky, Krum wrote:
>> Hi,
>>
>> Yes, thanks for the comments, they sound very logical to explaining the numbers from all the variants we tested.
>> Didn't know that the default exception mapper was developed as a result of a user demand as well.
>>
>> For the record: how could I disable it ? I am registering the Simple mapper within the returned set of the Application#getClasses() method call - so maybe I should remove it there, i.e. from the super.getClasses() list, or it is in a different manner ?

>>
>> Yes, I think that a servlet context parameter would be the most flexible and elegant solution, for configuring the desired behavior for non-200/exceptional cases.
>>
> I've spent some today on optimizing a couple of things,
> see the updated default WebApplicationException mapper:
>
> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>
> I think it's more effective now, yes still it is configurable and able
> to interact with CXF FaultLoggers and optionally fine-log the exception
> message.
>
> I was thinking yesterday of adding a property to get this default mapper
> ignored. I'm not sure now it's worth going this route.
>
> One way to 'disable' it is to offer a custom WebApplicationException
> mapper, something which we've explored. Another one - is to add this
> would be property but I've realized that it's not that good for CXF
> after all:  this would require the CXF servlet to process unmapped
> WebApplicationExceptions to be compliant and this would effectively mean
> repeating at the servlet level what is done at the CXF chain level but
> with limitations, ex, the Response produced by the mapper would still be
> routed through the out filters and indeed the optional CXF out faulty
> interceptors which can be handy. Say we can have an out filter which
> time-stamps all the responses - this would be ignored at the servlet
> level and would effectively have to be duplicated if needed...
>
> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
> will not be necessarily installed in some OSGI containers by default.
>
> So it appears to be the best way is to optimize the default mapper which
> I hope I did, and it would be always possible to offer custom mappers
> instead on per-Application basis.
>
> Let me know please if you have any concerns with the above,
>
> In meantime I also updated some code on the in chain to optimize the way
> the HTTP path to match is calculated - it may well save us few
> milliseconds :-)
>
> If you can share some more results from your tests which point to few
> unoptimized code spots then it would be good; we are going to review few
> code paths anyway,
>
> Thanks, Sergey
>
>>
>> Kindest Regards and many thanks again,
>> Krum.
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Monday, February 20, 2012 5:49 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi,
>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>> Hi Krum
>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>> Hi Sergey,
>>>>
>>>> Wow! I tested your idea and it produced dramatically better results!
>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>> investigation, and I believe that our observation is really true that
>>>> the overhead is spent in the error handling code. Just out of
>>>> curiosity: with the same SimpleApplicationMapper:
>>>>
>>> Thanks for the confirmation, so we've identified the spot where the
>>> problem is :-)
>>>
>>>> public class SimpleApplicationExceptionMapper implements
>>>> ExceptionMapper<WebApplicationException>   {
>>>>
>>>> @Override
>>>> public Response toResponse(WebApplicationException exception) {
>>>> return Response.status(404).build();
>>>> }
>>>>
>>>> }
>>>>
>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>
>>>>
>>>> However, without this class, Jersey performed even faster - most of
>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>
>>
>> Forgot to comment to this one... I think the fact that both frameworks
>> show approximately the same time with SimpleApplicationExceptionMapper
>> in place and that Jersey scales really well without it is that Jersey
>> basically either writes  to HTTP out stream or propagates the exception
>> up to the ServletException filter immediately if no custom mapper is
>> available.
>> In case of CXF the immediate propagation will also happen if no custom
>> mapper is available. The reason the default WebApplicationException
>> Mapper was installed was to do with the fact that few users were not
>> quite happy with having to write their custom mappers in order to report
>> differently to the exceptions generated at the runtime level or some of
>> the providers' level or indeed having to write a filter catching the
>> escaped exceptions there...
>>
>> What I might need to do additionally is to introduce a contextual
>> property that will tell the runtime to ignore even the default
>> WebApplicationException mapper, for example, when the user wishes to
>> get all the exceptions propagated to the filter level, to simplify the
>> performance data comparison, etc
>>
>> Cheers, Sergey
>>
>>>>
>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>> where the discrepancies come from.
>>>>
>>> I've opened
>>> https://issues.apache.org/jira/browse/CXF-4121
>>>
>>> I suspect now that it is the specific lines which get the basic warning
>>> message from the resource bundle are main 'culprits', I guess I will
>>> just have an internal final static String - will need to experiment a
>>> bit...It is good even the default providers can be customized/replaced,
>>> but I'll look into trying to make the default exception mapper
>>> performing a bit better :-)
>>>
>>> Thanks, Sergey
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>> To: users@cxf.apache.org
>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi Krum,
>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>> Hi to all Apache CXF users!,
>>>>>
>>>>> I would kindly like to bring to your attention a particular problem
>>>>> that we are currently being stuck into. We are experimenting with a
>>>>> simple web application, which exposes its functionality via REST. We
>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>> REST frameworks.
>>>>>
>>>>> For calculating our performance measurements, we simply execute REST
>>>>> requests, and in the client we measure the time that the roundtrip
>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>> favor of Jersey being much faster - in the cases where we test
>>>>> against non-existing resources. In all other cases, the numbers we
>>>>> got were really very close for the two REST frameworks.
>>>>>
>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>> we are not using CXF in its proper configuration. Could you please
>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>> so that we get the best out of it.
>>>>>
>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>> application logic, in the case when the path is not correct, we throw
>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>> this exception is getting caught in CXF, and heavily processed
>>>>> further down the stack.
>>>>>
>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>> processing to try to map the URI to some existing REST resource, I
>>>>> don't know ... maybe performance here could depend on the particular
>>>>> JSON framework that is being used. ...
>>>>>
>>>>
>>>> I suspect it could be the default WebApplicationExceptionMapper which is
>>>> to blame.
>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>> any other regular response, but I'm not sure it would explain the
>>>> difference.
>>>>
>>>> Can you give me a favor please and register a custom
>>>> WebApplicationException mapper which would only return:
>>>>
>>>> Response.status(exception.getStatus()).build() ?
>>>>
>>>> Have a look please at the default mapper's code:
>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>
>>>>
>>>> As you can see, we are trying to go to some length there to figure out
>>>> what and how to report a given exception and I reckon it all adds up to
>>>> the response time.
>>>>
>>>> Please try eliminating the default mapper and let us know the results
>>>>
>>>> Thanks, Sergey
>>>>
>>>>>
>>>>> Kindest Regards,
>>>>> Krum.
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>
>


--
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi Krum, btw, can you please
attach a test maven project that is used to build the CXF-based war to
https://issues.apache.org/jira/browse/CXF-4121 ?

I'll run my own tests against it too


Thanks, Sergey
On 23/02/12 15:51, Sergey Beryozkin wrote:
> Hi,
> On 23/02/12 14:50, Bakalsky, Krum wrote:
>> Hi Sergey,
>>
>> Thanks for the experimenting you have performed.
>>
>> [Yes, I checked the WAR archive, and it doesn't contain any Jersey
>> stuff inside.]
>>
> The reason I asked is that I recall Jersey Servlet scanning custom wars
> for Applications on Glassfish, but indeed, should not be the case in the
> Tomcat installation.
>
>> Which client/testing framework do you use in the pasted code snippet ?
>
> I'm using CXF client API:
> http://cxf.apache.org/docs/jax-rs-client-api.html.
>
> it uses HttpUrlConnection... The client code runs in a diff process...
>
>> I am asking that, because strangely enough we got much better
>> performance when switching the client framework from Meterware's
>> HttpUnit to Apache HttpClient. Could that really bring the significant
>> difference ?
>
> I'm not sure, given it is the basic processing on the client side,
>
>>
>> Maybe the responses that CXF returns are not exactly the same as those
>> that Jersey returns, and are somehow processed much slower by the
>> HttpUnit framework (maybe we have some issue of lazy loading the
>> response body, headers ... or just the HTTP parser is more mature in
>> Apache HttpClient...).
>>
>
> FYI, here is tcp trace fragment of what CXF returns in case of negative
> 404:
>
> HTTP/1.1 404 Not Found
> Date: Thu, 23 Feb 2012 15:47:20 GMT
> Content-Length: 0
> Server: Jetty(6.1.15)
>
> similarly for Tomcat
>
> HTTP/1.1 404 Not Found
> Server: Apache-Coyote/1.1
> Date: Thu, 23 Feb 2012 15:49:07 GMT
> Content-Length: 0
>
>> Thanks a lot once again for the serious attention that you are paying
>> on that topic!
>>
> not at all, I'm motivated to get CXF JAX-RS producing good perf results
> :-),
>
> thanks for the feedback,
>
> Sergey
>
>
>> Kindest Regards,
>> Krum.
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Thursday, February 23, 2012 2:16 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi Krum
>> On 22/02/12 16:09, Bakalsky, Krum wrote:
>>> Just a short update: the only way I could successfully enter my
>>> custom exception mapper with the debugger, was to annotate it with
>>> the @Provider annotation: I found that after debugging
>>> this<http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cxf/cxf-rt-frontend-jaxrs/2.5.1/org/apache/cxf/jaxrs/utils/ResourceUtils.java>
>>> code (if
>>> (c.getAnnotation<http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Class.java>(Provider<http://grepcode.com/file/repo1.maven.org/maven2/javax.ws.rs/jsr311-api/1.1.1/javax/ws/rs/ext/Provider.java>.class)
>>> != null) )
>>>
>>> Still 404 is slow ... So seems to be not an exception mapper issue...
>>
>>
>> My tests show 0.006 - 0.007 for the negative case.
>> I've deployed the war to the jetty server, run the 100 iterations first
>> to warm things up - that gave me 0.009
>>
>> from then on I get 0.006 - 0.007 all the time which I guess is still not
>> perfect but definitely not 0.5 :-)
>>
>> In the server I just do
>> throw new WebApplicationException(404) from within the resource method,
>>
>> on the client:
>>
>> long total = 0;
>> for (int i = 0; i< 100; i++) {
>> long l1 = System.currentTimeMillis();
>> try {
>> client.useSearchService();
>> } catch (Exception ex) {
>> }
>> long l2 = System.currentTimeMillis();
>> total += l2-l1;
>> }
>> System.out.println("Totall in milliseconds: " + total / 100);
>>
>> where
>> useSearchService() looks like this:
>>
>> WebClient wc = WebClient.create("http://localhost:" + port +
>> "/services/personservice/search");
>> wc.accept(MediaType.APPLICATION_XML);
>> wc.query("name", "Fred", "Lorraine");
>> wc.get(Response.class);
>>
>> so nothing unusual there...
>>
>> What I'm still curious about is why you got the unexpected improvements
>> after erroneously registering the custom mapper - there must be some
>> explanation for that :-). Can you also please check Jersey is not loaded
>> in the process which runs the CXF endpoint ?
>>
>> btw, run the war in Tomcat too - 0.007 all the time
>>
>> Thanks, Sergey
>>
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Wednesday, February 22, 2012 5:17 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> On 22/02/12 15:11, Bakalsky, Krum wrote:
>>>> Yes, that's right :)
>>>>
>>>> So, maybe the recipe for being faster here is: just throw some
>>>> exception within getClasses() ;-) - just kidding...
>>>
>>> LOL :-)
>>>
>>>>
>>>> Maybe I should try to see with some breakpoints where exactly the
>>>> work flow goes through ...
>>>
>>> I'm really intrigued now in seeing where the cost savings are coming
>>> from in this case :-), please try to check if you can get a chance...
>>>>
>>>> Anyway, do you know how to register custom exception mappers, in the
>>>> non-Spring case ? Is the getClasses() result the only way possible
>>>> for that ?
>>>
>>> as far as providers are concerned, they can be returned in either
>>> getClasses() - the runtime will have to load them or in getSingletons(),
>>> alongside the root resource classes if any...
>>>
>>> I'm finalizing my current main task soon, so will be looking into it all
>>> seriously soon
>>>
>>> Cheers, Sergey
>>>
>>>>
>>>> -----Original Message-----
>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>> Sent: Wednesday, February 22, 2012 4:30 PM
>>>> To: users@cxf.apache.org
>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi, By the way,
>>>>
>>>> On 22/02/12 14:19, Sergey Beryozkin wrote:
>>>>> Hi Krum,
>>>>>
>>>>> Before commenting further, let me ask you how do you take the
>>>>> measurements ? Do you check on the client side ? On the server side ?
>>>>>
>>>>> Thanks, Sergey
>>>>>
>>>>>
>>>>> On 22/02/12 13:49, Bakalsky, Krum wrote:
>>>>>> Hi Sergey, again,
>>>>>>
>>>>>> I am afraid that we are in the middle of some big
>>>>>> misunderstanding, me
>>>>>> being the one who has started it. Unfortunately, it turned out that I
>>>>>> didn't quite realize, that javax.ws.rs.core.Application#getClasses()
>>>>>> returns a non-modifiable collection (Collections.emptySet()), and in
>>>>>> my exception mapper I used:
>>>>>>
>>>>>> public Set<Class<?>> getClasses() {
>>>>>> // TODO Auto-generated method stub
>>>>>> Set<Class<?>> classes = super.getClasses();
>>>>>> classes.add(SimpleApplicationExceptionMapper.class);
>>>>>> return classes;
>>>>>> }
>>>>>>
>>>>>> which probably leads to an java.lang.UnsupportedOperationException at
>>>>>> runtime, and thus to some work flow/behavior that I am not aware of.
>>>>>> This made mu numbers faster, but obviously the scenario is broken, so
>>>>>> the results are irrelevant.
>>>>
>>>> This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored
>>>> getClasses() due to this exception, then it means that the default CXF
>>>> mapper should've been used, otherwise you'd not see 404 on the client
>>>> side, you'd just get 500 due to your custom exception simply escaping.
>>>> Yet apparently the results have become better :-) - thins is something
>>>> I'd like to understand...
>>>>
>>>>
>>>>>>
>>>>>> Next, I took a look at some examples, like
>>>>>> this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
>>>>>>
>>>>>> one, and modified my code accordingly. Unfortunately, I got the same
>>>>>> results: 0.500 seconds for the 404 cases. I am not even sure that my
>>>>>> exception mapper was really used, since when I inserted some dumps,
>>>>>> and even when I threw some exceptions in my toResponse() method, I
>>>>>> didn't see them nowhere.
>>>>>>
>>>>>> Next, I made a step further, and patched the
>>>>>> WebApplicationExceptionMapper that comes with CXF to look like this:
>>>>>>
>>>>>> public class WebApplicationExceptionMapper implements
>>>>>> ExceptionMapper<WebApplicationException> {
>>>>>>
>>>>>> public Response toResponse(WebApplicationException ex) {
>>>>>> return ex.getResponse();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> I made sure that it was really used. Unfortunately, I got the same
>>>>>> results: roundtrip time was again ~0.500 seconds.
>>>>>>
>>>>>> I am really not sure whether the CXF-4121 bug makes sense any more.
>>>>>> Problem is still here ;(
>>>>>>
>>>>
>>>> Sure, we'll get to the bottom of it, however, lets try to see where the
>>>> unexpected improvement after registering a custom mapper came from...
>>>>
>>>> Actually, I think I may know why your custom mapper may be ignored. It
>>>> should have a @Provider annotation, for it to be recognized as a
>>>> provider in the list returned from getClasses()
>>>>
>>>> Cheers, Sergey
>>>>
>>>>>>
>>>>>> Kindest Regards,
>>>>>> Krum.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Bakalsky, Krum
>>>>>> Sent: Wednesday, February 22, 2012 11:35 AM
>>>>>> To: 'users@cxf.apache.org'
>>>>>> Subject: RE: REST performance discrepancies between CXF and Jersey
>>>>>>
>>>>>> Hi Sergey,
>>>>>>
>>>>>> I have just made a test run with the modified WebApplicationException
>>>>>> mapper that you have given a link to below, and it works fine: it
>>>>>> also
>>>>>> shows 10 times decrease of roundtrip duration as well. So, it seems
>>>>>> that you have really optimized it pretty good already.
>>>>>>
>>>>>> Sure, I don't have any particular objections or concerns as to how to
>>>>>> approach the change that we are talking about. I'd guess that as long
>>>>>> as the application has a way to configure its own exception mapper
>>>>>> (which is already the case) so that 404 overhead is gone for it,
>>>>>> things are fine. I wouldn't know for those implications and
>>>>>> complications that adding a property could lead to. Thanks for
>>>>>> sharing.
>>>>>>
>>>>>> We haven't identified particular 'slow' code fragments, other than
>>>>>> the
>>>>>> 404 processing. If we do, we will sure let you know. (or if you want,
>>>>>> you could share your suspicions and give us some hint of where to
>>>>>> take
>>>>>> a look at ?)
>>>>>>
>>>>>> Kindest Regards,
>>>>>> Krum.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>> Sent: Tuesday, February 21, 2012 8:34 PM
>>>>>> To: users@cxf.apache.org
>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>
>>>>>> Hi Krum,
>>>>>> On 20/02/12 16:47, Bakalsky, Krum wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Yes, thanks for the comments, they sound very logical to explaining
>>>>>>> the numbers from all the variants we tested.
>>>>>>> Didn't know that the default exception mapper was developed as a
>>>>>>> result of a user demand as well.
>>>>>>>
>>>>>>> For the record: how could I disable it ? I am registering the Simple
>>>>>>> mapper within the returned set of the Application#getClasses()
>>>>>>> method
>>>>>>> call - so maybe I should remove it there, i.e. from the
>>>>>>> super.getClasses() list, or it is in a different manner ?
>>>>>>>
>>>>>>> Yes, I think that a servlet context parameter would be the most
>>>>>>> flexible and elegant solution, for configuring the desired behavior
>>>>>>> for non-200/exceptional cases.
>>>>>>>
>>>>>> I've spent some today on optimizing a couple of things,
>>>>>> see the updated default WebApplicationException mapper:
>>>>>>
>>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>>
>>>>>>
>>>>>>
>>>>>> I think it's more effective now, yes still it is configurable and
>>>>>> able
>>>>>> to interact with CXF FaultLoggers and optionally fine-log the
>>>>>> exception
>>>>>> message.
>>>>>>
>>>>>> I was thinking yesterday of adding a property to get this default
>>>>>> mapper
>>>>>> ignored. I'm not sure now it's worth going this route.
>>>>>>
>>>>>> One way to 'disable' it is to offer a custom WebApplicationException
>>>>>> mapper, something which we've explored. Another one - is to add this
>>>>>> would be property but I've realized that it's not that good for CXF
>>>>>> after all: this would require the CXF servlet to process unmapped
>>>>>> WebApplicationExceptions to be compliant and this would
>>>>>> effectively mean
>>>>>> repeating at the servlet level what is done at the CXF chain level
>>>>>> but
>>>>>> with limitations, ex, the Response produced by the mapper would
>>>>>> still be
>>>>>> routed through the out filters and indeed the optional CXF out faulty
>>>>>> interceptors which can be handy. Say we can have an out filter which
>>>>>> time-stamps all the responses - this would be ignored at the servlet
>>>>>> level and would effectively have to be duplicated if needed...
>>>>>>
>>>>>> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
>>>>>> will not be necessarily installed in some OSGI containers by default.
>>>>>>
>>>>>> So it appears to be the best way is to optimize the default mapper
>>>>>> which
>>>>>> I hope I did, and it would be always possible to offer custom mappers
>>>>>> instead on per-Application basis.
>>>>>>
>>>>>> Let me know please if you have any concerns with the above,
>>>>>>
>>>>>> In meantime I also updated some code on the in chain to optimize
>>>>>> the way
>>>>>> the HTTP path to match is calculated - it may well save us few
>>>>>> milliseconds :-)
>>>>>>
>>>>>> If you can share some more results from your tests which point to few
>>>>>> unoptimized code spots then it would be good; we are going to
>>>>>> review few
>>>>>> code paths anyway,
>>>>>>
>>>>>> Thanks, Sergey
>>>>>>
>>>>>>>
>>>>>>> Kindest Regards and many thanks again,
>>>>>>> Krum.
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>>> Sent: Monday, February 20, 2012 5:49 PM
>>>>>>> To: users@cxf.apache.org
>>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>>
>>>>>>> Hi,
>>>>>>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>>>>>>> Hi Krum
>>>>>>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>>>>>>> Hi Sergey,
>>>>>>>>>
>>>>>>>>> Wow! I tested your idea and it produced dramatically better
>>>>>>>>> results!
>>>>>>>>> The roundtrip time for 404 requests decreased by a factor of 10
>>>>>>>>> (from
>>>>>>>>> 0.500 seconds down to 0.030 seconds). That is a huge success
>>>>>>>>> for our
>>>>>>>>> investigation, and I believe that our observation is really
>>>>>>>>> true that
>>>>>>>>> the overhead is spent in the error handling code. Just out of
>>>>>>>>> curiosity: with the same SimpleApplicationMapper:
>>>>>>>>>
>>>>>>>> Thanks for the confirmation, so we've identified the spot where the
>>>>>>>> problem is :-)
>>>>>>>>
>>>>>>>>> public class SimpleApplicationExceptionMapper implements
>>>>>>>>> ExceptionMapper<WebApplicationException> {
>>>>>>>>>
>>>>>>>>> @Override
>>>>>>>>> public Response toResponse(WebApplicationException exception) {
>>>>>>>>> return Response.status(404).build();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> However, without this class, Jersey performed even faster -
>>>>>>>>> most of
>>>>>>>>> the times were 10 times faster, i.e. around 0.003 seconds,
>>>>>>>>> while in
>>>>>>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>>>>>>
>>>>>>>
>>>>>>> Forgot to comment to this one... I think the fact that both
>>>>>>> frameworks
>>>>>>> show approximately the same time with
>>>>>>> SimpleApplicationExceptionMapper
>>>>>>> in place and that Jersey scales really well without it is that
>>>>>>> Jersey
>>>>>>> basically either writes to HTTP out stream or propagates the
>>>>>>> exception
>>>>>>> up to the ServletException filter immediately if no custom mapper is
>>>>>>> available.
>>>>>>> In case of CXF the immediate propagation will also happen if no
>>>>>>> custom
>>>>>>> mapper is available. The reason the default WebApplicationException
>>>>>>> Mapper was installed was to do with the fact that few users were not
>>>>>>> quite happy with having to write their custom mappers in order to
>>>>>>> report
>>>>>>> differently to the exceptions generated at the runtime level or
>>>>>>> some of
>>>>>>> the providers' level or indeed having to write a filter catching the
>>>>>>> escaped exceptions there...
>>>>>>>
>>>>>>> What I might need to do additionally is to introduce a contextual
>>>>>>> property that will tell the runtime to ignore even the default
>>>>>>> WebApplicationException mapper, for example, when the user wishes to
>>>>>>> get all the exceptions propagated to the filter level, to
>>>>>>> simplify the
>>>>>>> performance data comparison, etc
>>>>>>>
>>>>>>> Cheers, Sergey
>>>>>>>
>>>>>>>>>
>>>>>>>>> Thank a lot for your idea and suggestion! It definitely shed
>>>>>>>>> light on
>>>>>>>>> where the discrepancies come from.
>>>>>>>>>
>>>>>>>> I've opened
>>>>>>>> https://issues.apache.org/jira/browse/CXF-4121
>>>>>>>>
>>>>>>>> I suspect now that it is the specific lines which get the basic
>>>>>>>> warning
>>>>>>>> message from the resource bundle are main 'culprits', I guess I
>>>>>>>> will
>>>>>>>> just have an internal final static String - will need to
>>>>>>>> experiment a
>>>>>>>> bit...It is good even the default providers can be
>>>>>>>> customized/replaced,
>>>>>>>> but I'll look into trying to make the default exception mapper
>>>>>>>> performing a bit better :-)
>>>>>>>>
>>>>>>>> Thanks, Sergey
>>>>>>>>>
>>>>>>>>> Kindest Regards,
>>>>>>>>> Krum.
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>>>>>>> To: users@cxf.apache.org
>>>>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>>>>
>>>>>>>>> Hi Krum,
>>>>>>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>>>>>>> Hi to all Apache CXF users!,
>>>>>>>>>>
>>>>>>>>>> I would kindly like to bring to your attention a particular
>>>>>>>>>> problem
>>>>>>>>>> that we are currently being stuck into. We are experimenting
>>>>>>>>>> with a
>>>>>>>>>> simple web application, which exposes its functionality via
>>>>>>>>>> REST. We
>>>>>>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our
>>>>>>>>>> application
>>>>>>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>>>>>>> REST frameworks.
>>>>>>>>>>
>>>>>>>>>> For calculating our performance measurements, we simply
>>>>>>>>>> execute REST
>>>>>>>>>> requests, and in the client we measure the time that the
>>>>>>>>>> roundtrip
>>>>>>>>>> takes. Apart from that, on the server side, we measure as well
>>>>>>>>>> the
>>>>>>>>>> time that our REST resource consumes, i.e. the method call
>>>>>>>>>> time. In
>>>>>>>>>> summary, we have some huge differences between CXF and Jersey
>>>>>>>>>> - in
>>>>>>>>>> favor of Jersey being much faster - in the cases where we test
>>>>>>>>>> against non-existing resources. In all other cases, the
>>>>>>>>>> numbers we
>>>>>>>>>> got were really very close for the two REST frameworks.
>>>>>>>>>>
>>>>>>>>>> But when testing against non-existing REST paths, the
>>>>>>>>>> roundtrip time
>>>>>>>>>> in CXF case is approximately 0.5 seconds, while in the case of
>>>>>>>>>> Jersey
>>>>>>>>>> it is 0.01 seconds. We think that this is quite strange, and
>>>>>>>>>> probably
>>>>>>>>>> we are not using CXF in its proper configuration. Could you
>>>>>>>>>> please
>>>>>>>>>> give us some hints ? Maybe we have to tune and tweak it a
>>>>>>>>>> little bit,
>>>>>>>>>> so that we get the best out of it.
>>>>>>>>>>
>>>>>>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>>>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>>>>>>> application logic, in the case when the path is not correct,
>>>>>>>>>> we throw
>>>>>>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw
>>>>>>>>>> that
>>>>>>>>>> this exception is getting caught in CXF, and heavily processed
>>>>>>>>>> further down the stack.
>>>>>>>>>>
>>>>>>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>>>>>>> situation in a very heavyweight manner ? Maybe it does some
>>>>>>>>>> thorough
>>>>>>>>>> processing to try to map the URI to some existing REST
>>>>>>>>>> resource, I
>>>>>>>>>> don't know ... maybe performance here could depend on the
>>>>>>>>>> particular
>>>>>>>>>> JSON framework that is being used. ...
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I suspect it could be the default WebApplicationExceptionMapper
>>>>>>>>> which is
>>>>>>>>> to blame.
>>>>>>>>> In CXF, a given Exception, once wrapped in a Response, is
>>>>>>>>> treated like
>>>>>>>>> any other regular response, but I'm not sure it would explain the
>>>>>>>>> difference.
>>>>>>>>>
>>>>>>>>> Can you give me a favor please and register a custom
>>>>>>>>> WebApplicationException mapper which would only return:
>>>>>>>>>
>>>>>>>>> Response.status(exception.getStatus()).build() ?
>>>>>>>>>
>>>>>>>>> Have a look please at the default mapper's code:
>>>>>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> As you can see, we are trying to go to some length there to
>>>>>>>>> figure out
>>>>>>>>> what and how to report a given exception and I reckon it all adds
>>>>>>>>> up to
>>>>>>>>> the response time.
>>>>>>>>>
>>>>>>>>> Please try eliminating the default mapper and let us know the
>>>>>>>>> results
>>>>>>>>>
>>>>>>>>> Thanks, Sergey
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kindest Regards,
>>>>>>>>>> Krum.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi,
On 23/02/12 14:50, Bakalsky, Krum wrote:
> Hi Sergey,
>
> Thanks for the experimenting you have performed.
>
> [Yes, I checked the WAR archive, and it doesn't contain any Jersey stuff inside.]
>
The reason I asked is that I recall Jersey Servlet scanning custom wars 
for Applications on Glassfish, but indeed, should not be the case in the 
Tomcat installation.

> Which client/testing framework do you use in the pasted code snippet ?

I'm using CXF client API:
http://cxf.apache.org/docs/jax-rs-client-api.html.

it uses HttpUrlConnection... The client code runs in a diff process...

> I am asking that, because strangely enough we got much better performance when switching the client framework from Meterware's HttpUnit to Apache HttpClient. Could that really bring the significant difference ?

I'm not sure, given it is the basic processing on the client side,

>
> Maybe the responses that CXF returns are not exactly the same as those that Jersey returns, and are somehow processed much slower by the HttpUnit framework (maybe we have some issue of lazy loading the response body, headers ... or just the HTTP parser is more mature in Apache HttpClient...).
>

FYI, here is tcp trace fragment of what CXF returns in case of negative 404:

HTTP/1.1 404 Not Found
Date: Thu, 23 Feb 2012 15:47:20 GMT
Content-Length: 0
Server: Jetty(6.1.15)

similarly for Tomcat

HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Date: Thu, 23 Feb 2012 15:49:07 GMT
Content-Length: 0

> Thanks a lot once again for the serious attention that you are paying on that topic!
>
not at all, I'm motivated to get CXF JAX-RS producing good perf results 
:-),

thanks for the feedback,

Sergey


> Kindest Regards,
> Krum.
>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Thursday, February 23, 2012 2:16 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi Krum
> On 22/02/12 16:09, Bakalsky, Krum wrote:
>> Just a short update: the only way I could successfully enter my custom exception mapper with the debugger, was to annotate it with the @Provider annotation: I found that after debugging this<http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cxf/cxf-rt-frontend-jaxrs/2.5.1/org/apache/cxf/jaxrs/utils/ResourceUtils.java>   code  (if (c.getAnnotation<http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Class.java>(Provider<http://grepcode.com/file/repo1.maven.org/maven2/javax.ws.rs/jsr311-api/1.1.1/javax/ws/rs/ext/Provider.java>.class) != null) )
>>
>> Still 404 is slow ... So seems to be not an exception mapper issue...
>
>
> My tests show 0.006 - 0.007 for the negative case.
> I've deployed the war to the jetty server, run the 100 iterations first
> to warm things up - that gave me 0.009
>
> from then on I get 0.006 - 0.007 all the time which I guess is still not
> perfect but definitely not 0.5 :-)
>
> In the server I just do
> throw new WebApplicationException(404) from within the resource method,
>
> on the client:
>
> long total = 0;
>           for (int i = 0; i<  100; i++) {
>                  long l1 = System.currentTimeMillis();
>                  try {
>                          client.useSearchService();
>                  } catch (Exception ex) {
>                  }
>                  long l2 = System.currentTimeMillis();
>                  total += l2-l1;
>           }
>           System.out.println("Totall in milliseconds: " + total / 100);
>
> where
> useSearchService() looks like this:
>
> WebClient wc = WebClient.create("http://localhost:" + port +
> "/services/personservice/search");
>           wc.accept(MediaType.APPLICATION_XML);
>           wc.query("name", "Fred", "Lorraine");
>           wc.get(Response.class);
>
> so nothing unusual there...
>
> What I'm still curious about is why you got the unexpected improvements
> after erroneously registering the custom mapper - there must be some
> explanation for that :-). Can you also please check Jersey is not loaded
> in the process which runs the CXF endpoint ?
>
> btw, run the war in Tomcat too - 0.007 all the time
>
> Thanks, Sergey
>
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Wednesday, February 22, 2012 5:17 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> On 22/02/12 15:11, Bakalsky, Krum wrote:
>>> Yes, that's right :)
>>>
>>> So, maybe the recipe for being faster here is: just throw some exception within getClasses()  ;-) - just kidding...
>>
>> LOL :-)
>>
>>>
>>> Maybe I should try to see with some breakpoints where exactly the work flow goes through ...
>>
>> I'm really intrigued now in seeing where the cost savings are coming
>> from in this case :-), please try to check if you can get a chance...
>>>
>>> Anyway, do you know how to register custom exception mappers, in the non-Spring case ? Is the getClasses() result the only way possible for that ?
>>
>> as far as providers are concerned, they can be returned in either
>> getClasses() - the runtime will have to load them or in getSingletons(),
>> alongside the root resource classes if any...
>>
>> I'm finalizing my current main task soon, so will be looking into it all
>> seriously soon
>>
>> Cheers, Sergey
>>
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Wednesday, February 22, 2012 4:30 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi, By the way,
>>>
>>> On 22/02/12 14:19, Sergey Beryozkin wrote:
>>>> Hi Krum,
>>>>
>>>> Before commenting further, let me ask you how do you take the
>>>> measurements ? Do you check on the client side ? On the server side ?
>>>>
>>>> Thanks, Sergey
>>>>
>>>>
>>>> On 22/02/12 13:49, Bakalsky, Krum wrote:
>>>>> Hi Sergey, again,
>>>>>
>>>>> I am afraid that we are in the middle of some big misunderstanding, me
>>>>> being the one who has started it. Unfortunately, it turned out that I
>>>>> didn't quite realize, that javax.ws.rs.core.Application#getClasses()
>>>>> returns a non-modifiable collection (Collections.emptySet()), and in
>>>>> my exception mapper I used:
>>>>>
>>>>> public Set<Class<?>>    getClasses() {
>>>>> // TODO Auto-generated method stub
>>>>> Set<Class<?>>    classes = super.getClasses();
>>>>> classes.add(SimpleApplicationExceptionMapper.class);
>>>>> return classes;
>>>>> }
>>>>>
>>>>> which probably leads to an java.lang.UnsupportedOperationException at
>>>>> runtime, and thus to some work flow/behavior that I am not aware of.
>>>>> This made mu numbers faster, but obviously the scenario is broken, so
>>>>> the results are irrelevant.
>>>
>>> This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored
>>> getClasses() due to this exception, then it means that the default CXF
>>> mapper should've been used, otherwise you'd not see 404 on the client
>>> side, you'd just get 500 due to your custom exception simply escaping.
>>> Yet apparently the results have become better :-) - thins is something
>>> I'd like to understand...
>>>
>>>
>>>>>
>>>>> Next, I took a look at some examples, like
>>>>> this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
>>>>> one, and modified my code accordingly. Unfortunately, I got the same
>>>>> results: 0.500 seconds for the 404 cases. I am not even sure that my
>>>>> exception mapper was really used, since when I inserted some dumps,
>>>>> and even when I threw some exceptions in my toResponse() method, I
>>>>> didn't see them nowhere.
>>>>>
>>>>> Next, I made a step further, and patched the
>>>>> WebApplicationExceptionMapper that comes with CXF to look like this:
>>>>>
>>>>> public class WebApplicationExceptionMapper implements
>>>>> ExceptionMapper<WebApplicationException>    {
>>>>>
>>>>> public Response toResponse(WebApplicationException ex) {
>>>>> return ex.getResponse();
>>>>> }
>>>>> }
>>>>>
>>>>> I made sure that it was really used. Unfortunately, I got the same
>>>>> results: roundtrip time was again ~0.500 seconds.
>>>>>
>>>>> I am really not sure whether the CXF-4121 bug makes sense any more.
>>>>> Problem is still here ;(
>>>>>
>>>
>>> Sure, we'll get to the bottom of it, however, lets try to see where the
>>> unexpected improvement after registering a custom mapper came from...
>>>
>>> Actually, I think I may know why your custom mapper may be ignored. It
>>> should have a @Provider annotation, for it to be recognized as a
>>> provider in the list returned from getClasses()
>>>
>>> Cheers, Sergey
>>>
>>>>>
>>>>> Kindest Regards,
>>>>> Krum.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Bakalsky, Krum
>>>>> Sent: Wednesday, February 22, 2012 11:35 AM
>>>>> To: 'users@cxf.apache.org'
>>>>> Subject: RE: REST performance discrepancies between CXF and Jersey
>>>>>
>>>>> Hi Sergey,
>>>>>
>>>>> I have just made a test run with the modified WebApplicationException
>>>>> mapper that you have given a link to below, and it works fine: it also
>>>>> shows 10 times decrease of roundtrip duration as well. So, it seems
>>>>> that you have really optimized it pretty good already.
>>>>>
>>>>> Sure, I don't have any particular objections or concerns as to how to
>>>>> approach the change that we are talking about. I'd guess that as long
>>>>> as the application has a way to configure its own exception mapper
>>>>> (which is already the case) so that 404 overhead is gone for it,
>>>>> things are fine. I wouldn't know for those implications and
>>>>> complications that adding a property could lead to. Thanks for sharing.
>>>>>
>>>>> We haven't identified particular 'slow' code fragments, other than the
>>>>> 404 processing. If we do, we will sure let you know. (or if you want,
>>>>> you could share your suspicions and give us some hint of where to take
>>>>> a look at ?)
>>>>>
>>>>> Kindest Regards,
>>>>> Krum.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>> Sent: Tuesday, February 21, 2012 8:34 PM
>>>>> To: users@cxf.apache.org
>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>
>>>>> Hi Krum,
>>>>> On 20/02/12 16:47, Bakalsky, Krum wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Yes, thanks for the comments, they sound very logical to explaining
>>>>>> the numbers from all the variants we tested.
>>>>>> Didn't know that the default exception mapper was developed as a
>>>>>> result of a user demand as well.
>>>>>>
>>>>>> For the record: how could I disable it ? I am registering the Simple
>>>>>> mapper within the returned set of the Application#getClasses() method
>>>>>> call - so maybe I should remove it there, i.e. from the
>>>>>> super.getClasses() list, or it is in a different manner ?
>>>>>>
>>>>>> Yes, I think that a servlet context parameter would be the most
>>>>>> flexible and elegant solution, for configuring the desired behavior
>>>>>> for non-200/exceptional cases.
>>>>>>
>>>>> I've spent some today on optimizing a couple of things,
>>>>> see the updated default WebApplicationException mapper:
>>>>>
>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>
>>>>>
>>>>> I think it's more effective now, yes still it is configurable and able
>>>>> to interact with CXF FaultLoggers and optionally fine-log the exception
>>>>> message.
>>>>>
>>>>> I was thinking yesterday of adding a property to get this default mapper
>>>>> ignored. I'm not sure now it's worth going this route.
>>>>>
>>>>> One way to 'disable' it is to offer a custom WebApplicationException
>>>>> mapper, something which we've explored. Another one - is to add this
>>>>> would be property but I've realized that it's not that good for CXF
>>>>> after all: this would require the CXF servlet to process unmapped
>>>>> WebApplicationExceptions to be compliant and this would effectively mean
>>>>> repeating at the servlet level what is done at the CXF chain level but
>>>>> with limitations, ex, the Response produced by the mapper would still be
>>>>> routed through the out filters and indeed the optional CXF out faulty
>>>>> interceptors which can be handy. Say we can have an out filter which
>>>>> time-stamps all the responses - this would be ignored at the servlet
>>>>> level and would effectively have to be duplicated if needed...
>>>>>
>>>>> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
>>>>> will not be necessarily installed in some OSGI containers by default.
>>>>>
>>>>> So it appears to be the best way is to optimize the default mapper which
>>>>> I hope I did, and it would be always possible to offer custom mappers
>>>>> instead on per-Application basis.
>>>>>
>>>>> Let me know please if you have any concerns with the above,
>>>>>
>>>>> In meantime I also updated some code on the in chain to optimize the way
>>>>> the HTTP path to match is calculated - it may well save us few
>>>>> milliseconds :-)
>>>>>
>>>>> If you can share some more results from your tests which point to few
>>>>> unoptimized code spots then it would be good; we are going to review few
>>>>> code paths anyway,
>>>>>
>>>>> Thanks, Sergey
>>>>>
>>>>>>
>>>>>> Kindest Regards and many thanks again,
>>>>>> Krum.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>> Sent: Monday, February 20, 2012 5:49 PM
>>>>>> To: users@cxf.apache.org
>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>
>>>>>> Hi,
>>>>>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>>>>>> Hi Krum
>>>>>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>>>>>> Hi Sergey,
>>>>>>>>
>>>>>>>> Wow! I tested your idea and it produced dramatically better results!
>>>>>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>>>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>>>>>> investigation, and I believe that our observation is really true that
>>>>>>>> the overhead is spent in the error handling code. Just out of
>>>>>>>> curiosity: with the same SimpleApplicationMapper:
>>>>>>>>
>>>>>>> Thanks for the confirmation, so we've identified the spot where the
>>>>>>> problem is :-)
>>>>>>>
>>>>>>>> public class SimpleApplicationExceptionMapper implements
>>>>>>>> ExceptionMapper<WebApplicationException>    {
>>>>>>>>
>>>>>>>> @Override
>>>>>>>> public Response toResponse(WebApplicationException exception) {
>>>>>>>> return Response.status(404).build();
>>>>>>>> }
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>>>>>
>>>>>>>>
>>>>>>>> However, without this class, Jersey performed even faster - most of
>>>>>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>>>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>>>>>
>>>>>>
>>>>>> Forgot to comment to this one... I think the fact that both frameworks
>>>>>> show approximately the same time with SimpleApplicationExceptionMapper
>>>>>> in place and that Jersey scales really well without it is that Jersey
>>>>>> basically either writes to HTTP out stream or propagates the exception
>>>>>> up to the ServletException filter immediately if no custom mapper is
>>>>>> available.
>>>>>> In case of CXF the immediate propagation will also happen if no custom
>>>>>> mapper is available. The reason the default WebApplicationException
>>>>>> Mapper was installed was to do with the fact that few users were not
>>>>>> quite happy with having to write their custom mappers in order to report
>>>>>> differently to the exceptions generated at the runtime level or some of
>>>>>> the providers' level or indeed having to write a filter catching the
>>>>>> escaped exceptions there...
>>>>>>
>>>>>> What I might need to do additionally is to introduce a contextual
>>>>>> property that will tell the runtime to ignore even the default
>>>>>> WebApplicationException mapper, for example, when the user wishes to
>>>>>> get all the exceptions propagated to the filter level, to simplify the
>>>>>> performance data comparison, etc
>>>>>>
>>>>>> Cheers, Sergey
>>>>>>
>>>>>>>>
>>>>>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>>>>>> where the discrepancies come from.
>>>>>>>>
>>>>>>> I've opened
>>>>>>> https://issues.apache.org/jira/browse/CXF-4121
>>>>>>>
>>>>>>> I suspect now that it is the specific lines which get the basic warning
>>>>>>> message from the resource bundle are main 'culprits', I guess I will
>>>>>>> just have an internal final static String - will need to experiment a
>>>>>>> bit...It is good even the default providers can be customized/replaced,
>>>>>>> but I'll look into trying to make the default exception mapper
>>>>>>> performing a bit better :-)
>>>>>>>
>>>>>>> Thanks, Sergey
>>>>>>>>
>>>>>>>> Kindest Regards,
>>>>>>>> Krum.
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>>>>>> To: users@cxf.apache.org
>>>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>>>
>>>>>>>> Hi Krum,
>>>>>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>>>>>> Hi to all Apache CXF users!,
>>>>>>>>>
>>>>>>>>> I would kindly like to bring to your attention a particular problem
>>>>>>>>> that we are currently being stuck into. We are experimenting with a
>>>>>>>>> simple web application, which exposes its functionality via REST. We
>>>>>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>>>>>> REST frameworks.
>>>>>>>>>
>>>>>>>>> For calculating our performance measurements, we simply execute REST
>>>>>>>>> requests, and in the client we measure the time that the roundtrip
>>>>>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>>>>>> favor of Jersey being much faster - in the cases where we test
>>>>>>>>> against non-existing resources. In all other cases, the numbers we
>>>>>>>>> got were really very close for the two REST frameworks.
>>>>>>>>>
>>>>>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>>>>>> we are not using CXF in its proper configuration. Could you please
>>>>>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>>>>>> so that we get the best out of it.
>>>>>>>>>
>>>>>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>>>>>> application logic, in the case when the path is not correct, we throw
>>>>>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>>>>>> this exception is getting caught in CXF, and heavily processed
>>>>>>>>> further down the stack.
>>>>>>>>>
>>>>>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>>>>>> processing to try to map the URI to some existing REST resource, I
>>>>>>>>> don't know ... maybe performance here could depend on the particular
>>>>>>>>> JSON framework that is being used. ...
>>>>>>>>>
>>>>>>>>
>>>>>>>> I suspect it could be the default WebApplicationExceptionMapper
>>>>>>>> which is
>>>>>>>> to blame.
>>>>>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>>>>>> any other regular response, but I'm not sure it would explain the
>>>>>>>> difference.
>>>>>>>>
>>>>>>>> Can you give me a favor please and register a custom
>>>>>>>> WebApplicationException mapper which would only return:
>>>>>>>>
>>>>>>>> Response.status(exception.getStatus()).build() ?
>>>>>>>>
>>>>>>>> Have a look please at the default mapper's code:
>>>>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> As you can see, we are trying to go to some length there to figure out
>>>>>>>> what and how to report a given exception and I reckon it all adds
>>>>>>>> up to
>>>>>>>> the response time.
>>>>>>>>
>>>>>>>> Please try eliminating the default mapper and let us know the results
>>>>>>>>
>>>>>>>> Thanks, Sergey
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Kindest Regards,
>>>>>>>>> Krum.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --

RE: REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Hi Sergey,

Thanks for the experimenting you have performed.

[Yes, I checked the WAR archive, and it doesn't contain any Jersey stuff inside.]

Which client/testing framework do you use in the pasted code snippet ? I am asking that, because strangely enough we got much better performance when switching the client framework from Meterware's HttpUnit to Apache HttpClient. Could that really bring the significant difference ?

Maybe the responses that CXF returns are not exactly the same as those that Jersey returns, and are somehow processed much slower by the HttpUnit framework (maybe we have some issue of lazy loading the response body, headers ... or just the HTTP parser is more mature in Apache HttpClient...).

Thanks a lot once again for the serious attention that you are paying on that topic!

Kindest Regards,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
Sent: Thursday, February 23, 2012 2:16 PM
To: users@cxf.apache.org
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi Krum
On 22/02/12 16:09, Bakalsky, Krum wrote:
> Just a short update: the only way I could successfully enter my custom exception mapper with the debugger, was to annotate it with the @Provider annotation: I found that after debugging this<http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cxf/cxf-rt-frontend-jaxrs/2.5.1/org/apache/cxf/jaxrs/utils/ResourceUtils.java>  code  (if (c.getAnnotation<http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Class.java>(Provider<http://grepcode.com/file/repo1.maven.org/maven2/javax.ws.rs/jsr311-api/1.1.1/javax/ws/rs/ext/Provider.java>.class) != null) )
>
> Still 404 is slow ... So seems to be not an exception mapper issue...


My tests show 0.006 - 0.007 for the negative case.
I've deployed the war to the jetty server, run the 100 iterations first
to warm things up - that gave me 0.009

from then on I get 0.006 - 0.007 all the time which I guess is still not
perfect but definitely not 0.5 :-)

In the server I just do
throw new WebApplicationException(404) from within the resource method,

on the client:

long total = 0;
         for (int i = 0; i < 100; i++) {
                long l1 = System.currentTimeMillis();
                try {
                        client.useSearchService();
                } catch (Exception ex) {
                }
                long l2 = System.currentTimeMillis();
                total += l2-l1;
         }
         System.out.println("Totall in milliseconds: " + total / 100);

where
useSearchService() looks like this:

WebClient wc = WebClient.create("http://localhost:" + port +
"/services/personservice/search");
         wc.accept(MediaType.APPLICATION_XML);
         wc.query("name", "Fred", "Lorraine");
         wc.get(Response.class);

so nothing unusual there...

What I'm still curious about is why you got the unexpected improvements
after erroneously registering the custom mapper - there must be some
explanation for that :-). Can you also please check Jersey is not loaded
in the process which runs the CXF endpoint ?

btw, run the war in Tomcat too - 0.007 all the time

Thanks, Sergey

>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Wednesday, February 22, 2012 5:17 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> On 22/02/12 15:11, Bakalsky, Krum wrote:
>> Yes, that's right :)
>>
>> So, maybe the recipe for being faster here is: just throw some exception within getClasses()  ;-) - just kidding...
>
> LOL :-)
>
>>
>> Maybe I should try to see with some breakpoints where exactly the work flow goes through ...
>
> I'm really intrigued now in seeing where the cost savings are coming
> from in this case :-), please try to check if you can get a chance...
>>
>> Anyway, do you know how to register custom exception mappers, in the non-Spring case ? Is the getClasses() result the only way possible for that ?
>
> as far as providers are concerned, they can be returned in either
> getClasses() - the runtime will have to load them or in getSingletons(),
> alongside the root resource classes if any...
>
> I'm finalizing my current main task soon, so will be looking into it all
> seriously soon
>
> Cheers, Sergey
>
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Wednesday, February 22, 2012 4:30 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi, By the way,
>>
>> On 22/02/12 14:19, Sergey Beryozkin wrote:
>>> Hi Krum,
>>>
>>> Before commenting further, let me ask you how do you take the
>>> measurements ? Do you check on the client side ? On the server side ?
>>>
>>> Thanks, Sergey
>>>
>>>
>>> On 22/02/12 13:49, Bakalsky, Krum wrote:
>>>> Hi Sergey, again,
>>>>
>>>> I am afraid that we are in the middle of some big misunderstanding, me
>>>> being the one who has started it. Unfortunately, it turned out that I
>>>> didn't quite realize, that javax.ws.rs.core.Application#getClasses()
>>>> returns a non-modifiable collection (Collections.emptySet()), and in
>>>> my exception mapper I used:
>>>>
>>>> public Set<Class<?>>   getClasses() {
>>>> // TODO Auto-generated method stub
>>>> Set<Class<?>>   classes = super.getClasses();
>>>> classes.add(SimpleApplicationExceptionMapper.class);
>>>> return classes;
>>>> }
>>>>
>>>> which probably leads to an java.lang.UnsupportedOperationException at
>>>> runtime, and thus to some work flow/behavior that I am not aware of.
>>>> This made mu numbers faster, but obviously the scenario is broken, so
>>>> the results are irrelevant.
>>
>> This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored
>> getClasses() due to this exception, then it means that the default CXF
>> mapper should've been used, otherwise you'd not see 404 on the client
>> side, you'd just get 500 due to your custom exception simply escaping.
>> Yet apparently the results have become better :-) - thins is something
>> I'd like to understand...
>>
>>
>>>>
>>>> Next, I took a look at some examples, like
>>>> this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
>>>> one, and modified my code accordingly. Unfortunately, I got the same
>>>> results: 0.500 seconds for the 404 cases. I am not even sure that my
>>>> exception mapper was really used, since when I inserted some dumps,
>>>> and even when I threw some exceptions in my toResponse() method, I
>>>> didn't see them nowhere.
>>>>
>>>> Next, I made a step further, and patched the
>>>> WebApplicationExceptionMapper that comes with CXF to look like this:
>>>>
>>>> public class WebApplicationExceptionMapper implements
>>>> ExceptionMapper<WebApplicationException>   {
>>>>
>>>> public Response toResponse(WebApplicationException ex) {
>>>> return ex.getResponse();
>>>> }
>>>> }
>>>>
>>>> I made sure that it was really used. Unfortunately, I got the same
>>>> results: roundtrip time was again ~0.500 seconds.
>>>>
>>>> I am really not sure whether the CXF-4121 bug makes sense any more.
>>>> Problem is still here ;(
>>>>
>>
>> Sure, we'll get to the bottom of it, however, lets try to see where the
>> unexpected improvement after registering a custom mapper came from...
>>
>> Actually, I think I may know why your custom mapper may be ignored. It
>> should have a @Provider annotation, for it to be recognized as a
>> provider in the list returned from getClasses()
>>
>> Cheers, Sergey
>>
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Bakalsky, Krum
>>>> Sent: Wednesday, February 22, 2012 11:35 AM
>>>> To: 'users@cxf.apache.org'
>>>> Subject: RE: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi Sergey,
>>>>
>>>> I have just made a test run with the modified WebApplicationException
>>>> mapper that you have given a link to below, and it works fine: it also
>>>> shows 10 times decrease of roundtrip duration as well. So, it seems
>>>> that you have really optimized it pretty good already.
>>>>
>>>> Sure, I don't have any particular objections or concerns as to how to
>>>> approach the change that we are talking about. I'd guess that as long
>>>> as the application has a way to configure its own exception mapper
>>>> (which is already the case) so that 404 overhead is gone for it,
>>>> things are fine. I wouldn't know for those implications and
>>>> complications that adding a property could lead to. Thanks for sharing.
>>>>
>>>> We haven't identified particular 'slow' code fragments, other than the
>>>> 404 processing. If we do, we will sure let you know. (or if you want,
>>>> you could share your suspicions and give us some hint of where to take
>>>> a look at ?)
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>> Sent: Tuesday, February 21, 2012 8:34 PM
>>>> To: users@cxf.apache.org
>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi Krum,
>>>> On 20/02/12 16:47, Bakalsky, Krum wrote:
>>>>> Hi,
>>>>>
>>>>> Yes, thanks for the comments, they sound very logical to explaining
>>>>> the numbers from all the variants we tested.
>>>>> Didn't know that the default exception mapper was developed as a
>>>>> result of a user demand as well.
>>>>>
>>>>> For the record: how could I disable it ? I am registering the Simple
>>>>> mapper within the returned set of the Application#getClasses() method
>>>>> call - so maybe I should remove it there, i.e. from the
>>>>> super.getClasses() list, or it is in a different manner ?
>>>>>
>>>>> Yes, I think that a servlet context parameter would be the most
>>>>> flexible and elegant solution, for configuring the desired behavior
>>>>> for non-200/exceptional cases.
>>>>>
>>>> I've spent some today on optimizing a couple of things,
>>>> see the updated default WebApplicationException mapper:
>>>>
>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>
>>>>
>>>> I think it's more effective now, yes still it is configurable and able
>>>> to interact with CXF FaultLoggers and optionally fine-log the exception
>>>> message.
>>>>
>>>> I was thinking yesterday of adding a property to get this default mapper
>>>> ignored. I'm not sure now it's worth going this route.
>>>>
>>>> One way to 'disable' it is to offer a custom WebApplicationException
>>>> mapper, something which we've explored. Another one - is to add this
>>>> would be property but I've realized that it's not that good for CXF
>>>> after all: this would require the CXF servlet to process unmapped
>>>> WebApplicationExceptions to be compliant and this would effectively mean
>>>> repeating at the servlet level what is done at the CXF chain level but
>>>> with limitations, ex, the Response produced by the mapper would still be
>>>> routed through the out filters and indeed the optional CXF out faulty
>>>> interceptors which can be handy. Say we can have an out filter which
>>>> time-stamps all the responses - this would be ignored at the servlet
>>>> level and would effectively have to be duplicated if needed...
>>>>
>>>> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
>>>> will not be necessarily installed in some OSGI containers by default.
>>>>
>>>> So it appears to be the best way is to optimize the default mapper which
>>>> I hope I did, and it would be always possible to offer custom mappers
>>>> instead on per-Application basis.
>>>>
>>>> Let me know please if you have any concerns with the above,
>>>>
>>>> In meantime I also updated some code on the in chain to optimize the way
>>>> the HTTP path to match is calculated - it may well save us few
>>>> milliseconds :-)
>>>>
>>>> If you can share some more results from your tests which point to few
>>>> unoptimized code spots then it would be good; we are going to review few
>>>> code paths anyway,
>>>>
>>>> Thanks, Sergey
>>>>
>>>>>
>>>>> Kindest Regards and many thanks again,
>>>>> Krum.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>> Sent: Monday, February 20, 2012 5:49 PM
>>>>> To: users@cxf.apache.org
>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>
>>>>> Hi,
>>>>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>>>>> Hi Krum
>>>>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>>>>> Hi Sergey,
>>>>>>>
>>>>>>> Wow! I tested your idea and it produced dramatically better results!
>>>>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>>>>> investigation, and I believe that our observation is really true that
>>>>>>> the overhead is spent in the error handling code. Just out of
>>>>>>> curiosity: with the same SimpleApplicationMapper:
>>>>>>>
>>>>>> Thanks for the confirmation, so we've identified the spot where the
>>>>>> problem is :-)
>>>>>>
>>>>>>> public class SimpleApplicationExceptionMapper implements
>>>>>>> ExceptionMapper<WebApplicationException>   {
>>>>>>>
>>>>>>> @Override
>>>>>>> public Response toResponse(WebApplicationException exception) {
>>>>>>> return Response.status(404).build();
>>>>>>> }
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>>>>
>>>>>>>
>>>>>>> However, without this class, Jersey performed even faster - most of
>>>>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>>>>
>>>>>
>>>>> Forgot to comment to this one... I think the fact that both frameworks
>>>>> show approximately the same time with SimpleApplicationExceptionMapper
>>>>> in place and that Jersey scales really well without it is that Jersey
>>>>> basically either writes to HTTP out stream or propagates the exception
>>>>> up to the ServletException filter immediately if no custom mapper is
>>>>> available.
>>>>> In case of CXF the immediate propagation will also happen if no custom
>>>>> mapper is available. The reason the default WebApplicationException
>>>>> Mapper was installed was to do with the fact that few users were not
>>>>> quite happy with having to write their custom mappers in order to report
>>>>> differently to the exceptions generated at the runtime level or some of
>>>>> the providers' level or indeed having to write a filter catching the
>>>>> escaped exceptions there...
>>>>>
>>>>> What I might need to do additionally is to introduce a contextual
>>>>> property that will tell the runtime to ignore even the default
>>>>> WebApplicationException mapper, for example, when the user wishes to
>>>>> get all the exceptions propagated to the filter level, to simplify the
>>>>> performance data comparison, etc
>>>>>
>>>>> Cheers, Sergey
>>>>>
>>>>>>>
>>>>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>>>>> where the discrepancies come from.
>>>>>>>
>>>>>> I've opened
>>>>>> https://issues.apache.org/jira/browse/CXF-4121
>>>>>>
>>>>>> I suspect now that it is the specific lines which get the basic warning
>>>>>> message from the resource bundle are main 'culprits', I guess I will
>>>>>> just have an internal final static String - will need to experiment a
>>>>>> bit...It is good even the default providers can be customized/replaced,
>>>>>> but I'll look into trying to make the default exception mapper
>>>>>> performing a bit better :-)
>>>>>>
>>>>>> Thanks, Sergey
>>>>>>>
>>>>>>> Kindest Regards,
>>>>>>> Krum.
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>>>>> To: users@cxf.apache.org
>>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>>
>>>>>>> Hi Krum,
>>>>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>>>>> Hi to all Apache CXF users!,
>>>>>>>>
>>>>>>>> I would kindly like to bring to your attention a particular problem
>>>>>>>> that we are currently being stuck into. We are experimenting with a
>>>>>>>> simple web application, which exposes its functionality via REST. We
>>>>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>>>>> REST frameworks.
>>>>>>>>
>>>>>>>> For calculating our performance measurements, we simply execute REST
>>>>>>>> requests, and in the client we measure the time that the roundtrip
>>>>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>>>>> favor of Jersey being much faster - in the cases where we test
>>>>>>>> against non-existing resources. In all other cases, the numbers we
>>>>>>>> got were really very close for the two REST frameworks.
>>>>>>>>
>>>>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>>>>> we are not using CXF in its proper configuration. Could you please
>>>>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>>>>> so that we get the best out of it.
>>>>>>>>
>>>>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>>>>> application logic, in the case when the path is not correct, we throw
>>>>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>>>>> this exception is getting caught in CXF, and heavily processed
>>>>>>>> further down the stack.
>>>>>>>>
>>>>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>>>>> processing to try to map the URI to some existing REST resource, I
>>>>>>>> don't know ... maybe performance here could depend on the particular
>>>>>>>> JSON framework that is being used. ...
>>>>>>>>
>>>>>>>
>>>>>>> I suspect it could be the default WebApplicationExceptionMapper
>>>>>>> which is
>>>>>>> to blame.
>>>>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>>>>> any other regular response, but I'm not sure it would explain the
>>>>>>> difference.
>>>>>>>
>>>>>>> Can you give me a favor please and register a custom
>>>>>>> WebApplicationException mapper which would only return:
>>>>>>>
>>>>>>> Response.status(exception.getStatus()).build() ?
>>>>>>>
>>>>>>> Have a look please at the default mapper's code:
>>>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> As you can see, we are trying to go to some length there to figure out
>>>>>>> what and how to report a given exception and I reckon it all adds
>>>>>>> up to
>>>>>>> the response time.
>>>>>>>
>>>>>>> Please try eliminating the default mapper and let us know the results
>>>>>>>
>>>>>>> Thanks, Sergey
>>>>>>>
>>>>>>>>
>>>>>>>> Kindest Regards,
>>>>>>>> Krum.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>> --
>> Sergey Beryozkin
>>
>> Talend Community Coders
>> http://coders.talend.com/
>>
>> Blog: http://sberyozkin.blogspot.com
>
>


--
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi Krum
On 22/02/12 16:09, Bakalsky, Krum wrote:
> Just a short update: the only way I could successfully enter my custom exception mapper with the debugger, was to annotate it with the @Provider annotation: I found that after debugging this<http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cxf/cxf-rt-frontend-jaxrs/2.5.1/org/apache/cxf/jaxrs/utils/ResourceUtils.java>  code  (if (c.getAnnotation<http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Class.java>(Provider<http://grepcode.com/file/repo1.maven.org/maven2/javax.ws.rs/jsr311-api/1.1.1/javax/ws/rs/ext/Provider.java>.class) != null) )
>
> Still 404 is slow ... So seems to be not an exception mapper issue...


My tests show 0.006 - 0.007 for the negative case.
I've deployed the war to the jetty server, run the 100 iterations first 
to warm things up - that gave me 0.009

from then on I get 0.006 - 0.007 all the time which I guess is still not 
perfect but definitely not 0.5 :-)

In the server I just do
throw new WebApplicationException(404) from within the resource method,

on the client:

long total = 0;
         for (int i = 0; i < 100; i++) {
         	long l1 = System.currentTimeMillis();
	        try {
	        	client.useSearchService();
	        } catch (Exception ex) {
	        }
	        long l2 = System.currentTimeMillis();
	        total += l2-l1;
         }
         System.out.println("Totall in milliseconds: " + total / 100);

where
useSearchService() looks like this:

WebClient wc = WebClient.create("http://localhost:" + port + 
"/services/personservice/search");
         wc.accept(MediaType.APPLICATION_XML);
         wc.query("name", "Fred", "Lorraine");
         wc.get(Response.class);

so nothing unusual there...

What I'm still curious about is why you got the unexpected improvements 
after erroneously registering the custom mapper - there must be some 
explanation for that :-). Can you also please check Jersey is not loaded 
in the process which runs the CXF endpoint ?

btw, run the war in Tomcat too - 0.007 all the time

Thanks, Sergey

>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Wednesday, February 22, 2012 5:17 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> On 22/02/12 15:11, Bakalsky, Krum wrote:
>> Yes, that's right :)
>>
>> So, maybe the recipe for being faster here is: just throw some exception within getClasses()  ;-) - just kidding...
>
> LOL :-)
>
>>
>> Maybe I should try to see with some breakpoints where exactly the work flow goes through ...
>
> I'm really intrigued now in seeing where the cost savings are coming
> from in this case :-), please try to check if you can get a chance...
>>
>> Anyway, do you know how to register custom exception mappers, in the non-Spring case ? Is the getClasses() result the only way possible for that ?
>
> as far as providers are concerned, they can be returned in either
> getClasses() - the runtime will have to load them or in getSingletons(),
> alongside the root resource classes if any...
>
> I'm finalizing my current main task soon, so will be looking into it all
> seriously soon
>
> Cheers, Sergey
>
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Wednesday, February 22, 2012 4:30 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi, By the way,
>>
>> On 22/02/12 14:19, Sergey Beryozkin wrote:
>>> Hi Krum,
>>>
>>> Before commenting further, let me ask you how do you take the
>>> measurements ? Do you check on the client side ? On the server side ?
>>>
>>> Thanks, Sergey
>>>
>>>
>>> On 22/02/12 13:49, Bakalsky, Krum wrote:
>>>> Hi Sergey, again,
>>>>
>>>> I am afraid that we are in the middle of some big misunderstanding, me
>>>> being the one who has started it. Unfortunately, it turned out that I
>>>> didn't quite realize, that javax.ws.rs.core.Application#getClasses()
>>>> returns a non-modifiable collection (Collections.emptySet()), and in
>>>> my exception mapper I used:
>>>>
>>>> public Set<Class<?>>   getClasses() {
>>>> // TODO Auto-generated method stub
>>>> Set<Class<?>>   classes = super.getClasses();
>>>> classes.add(SimpleApplicationExceptionMapper.class);
>>>> return classes;
>>>> }
>>>>
>>>> which probably leads to an java.lang.UnsupportedOperationException at
>>>> runtime, and thus to some work flow/behavior that I am not aware of.
>>>> This made mu numbers faster, but obviously the scenario is broken, so
>>>> the results are irrelevant.
>>
>> This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored
>> getClasses() due to this exception, then it means that the default CXF
>> mapper should've been used, otherwise you'd not see 404 on the client
>> side, you'd just get 500 due to your custom exception simply escaping.
>> Yet apparently the results have become better :-) - thins is something
>> I'd like to understand...
>>
>>
>>>>
>>>> Next, I took a look at some examples, like
>>>> this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
>>>> one, and modified my code accordingly. Unfortunately, I got the same
>>>> results: 0.500 seconds for the 404 cases. I am not even sure that my
>>>> exception mapper was really used, since when I inserted some dumps,
>>>> and even when I threw some exceptions in my toResponse() method, I
>>>> didn't see them nowhere.
>>>>
>>>> Next, I made a step further, and patched the
>>>> WebApplicationExceptionMapper that comes with CXF to look like this:
>>>>
>>>> public class WebApplicationExceptionMapper implements
>>>> ExceptionMapper<WebApplicationException>   {
>>>>
>>>> public Response toResponse(WebApplicationException ex) {
>>>> return ex.getResponse();
>>>> }
>>>> }
>>>>
>>>> I made sure that it was really used. Unfortunately, I got the same
>>>> results: roundtrip time was again ~0.500 seconds.
>>>>
>>>> I am really not sure whether the CXF-4121 bug makes sense any more.
>>>> Problem is still here ;(
>>>>
>>
>> Sure, we'll get to the bottom of it, however, lets try to see where the
>> unexpected improvement after registering a custom mapper came from...
>>
>> Actually, I think I may know why your custom mapper may be ignored. It
>> should have a @Provider annotation, for it to be recognized as a
>> provider in the list returned from getClasses()
>>
>> Cheers, Sergey
>>
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Bakalsky, Krum
>>>> Sent: Wednesday, February 22, 2012 11:35 AM
>>>> To: 'users@cxf.apache.org'
>>>> Subject: RE: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi Sergey,
>>>>
>>>> I have just made a test run with the modified WebApplicationException
>>>> mapper that you have given a link to below, and it works fine: it also
>>>> shows 10 times decrease of roundtrip duration as well. So, it seems
>>>> that you have really optimized it pretty good already.
>>>>
>>>> Sure, I don't have any particular objections or concerns as to how to
>>>> approach the change that we are talking about. I'd guess that as long
>>>> as the application has a way to configure its own exception mapper
>>>> (which is already the case) so that 404 overhead is gone for it,
>>>> things are fine. I wouldn't know for those implications and
>>>> complications that adding a property could lead to. Thanks for sharing.
>>>>
>>>> We haven't identified particular 'slow' code fragments, other than the
>>>> 404 processing. If we do, we will sure let you know. (or if you want,
>>>> you could share your suspicions and give us some hint of where to take
>>>> a look at ?)
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>> Sent: Tuesday, February 21, 2012 8:34 PM
>>>> To: users@cxf.apache.org
>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi Krum,
>>>> On 20/02/12 16:47, Bakalsky, Krum wrote:
>>>>> Hi,
>>>>>
>>>>> Yes, thanks for the comments, they sound very logical to explaining
>>>>> the numbers from all the variants we tested.
>>>>> Didn't know that the default exception mapper was developed as a
>>>>> result of a user demand as well.
>>>>>
>>>>> For the record: how could I disable it ? I am registering the Simple
>>>>> mapper within the returned set of the Application#getClasses() method
>>>>> call - so maybe I should remove it there, i.e. from the
>>>>> super.getClasses() list, or it is in a different manner ?
>>>>>
>>>>> Yes, I think that a servlet context parameter would be the most
>>>>> flexible and elegant solution, for configuring the desired behavior
>>>>> for non-200/exceptional cases.
>>>>>
>>>> I've spent some today on optimizing a couple of things,
>>>> see the updated default WebApplicationException mapper:
>>>>
>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>
>>>>
>>>> I think it's more effective now, yes still it is configurable and able
>>>> to interact with CXF FaultLoggers and optionally fine-log the exception
>>>> message.
>>>>
>>>> I was thinking yesterday of adding a property to get this default mapper
>>>> ignored. I'm not sure now it's worth going this route.
>>>>
>>>> One way to 'disable' it is to offer a custom WebApplicationException
>>>> mapper, something which we've explored. Another one - is to add this
>>>> would be property but I've realized that it's not that good for CXF
>>>> after all: this would require the CXF servlet to process unmapped
>>>> WebApplicationExceptions to be compliant and this would effectively mean
>>>> repeating at the servlet level what is done at the CXF chain level but
>>>> with limitations, ex, the Response produced by the mapper would still be
>>>> routed through the out filters and indeed the optional CXF out faulty
>>>> interceptors which can be handy. Say we can have an out filter which
>>>> time-stamps all the responses - this would be ignored at the servlet
>>>> level and would effectively have to be duplicated if needed...
>>>>
>>>> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
>>>> will not be necessarily installed in some OSGI containers by default.
>>>>
>>>> So it appears to be the best way is to optimize the default mapper which
>>>> I hope I did, and it would be always possible to offer custom mappers
>>>> instead on per-Application basis.
>>>>
>>>> Let me know please if you have any concerns with the above,
>>>>
>>>> In meantime I also updated some code on the in chain to optimize the way
>>>> the HTTP path to match is calculated - it may well save us few
>>>> milliseconds :-)
>>>>
>>>> If you can share some more results from your tests which point to few
>>>> unoptimized code spots then it would be good; we are going to review few
>>>> code paths anyway,
>>>>
>>>> Thanks, Sergey
>>>>
>>>>>
>>>>> Kindest Regards and many thanks again,
>>>>> Krum.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>> Sent: Monday, February 20, 2012 5:49 PM
>>>>> To: users@cxf.apache.org
>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>
>>>>> Hi,
>>>>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>>>>> Hi Krum
>>>>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>>>>> Hi Sergey,
>>>>>>>
>>>>>>> Wow! I tested your idea and it produced dramatically better results!
>>>>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>>>>> investigation, and I believe that our observation is really true that
>>>>>>> the overhead is spent in the error handling code. Just out of
>>>>>>> curiosity: with the same SimpleApplicationMapper:
>>>>>>>
>>>>>> Thanks for the confirmation, so we've identified the spot where the
>>>>>> problem is :-)
>>>>>>
>>>>>>> public class SimpleApplicationExceptionMapper implements
>>>>>>> ExceptionMapper<WebApplicationException>   {
>>>>>>>
>>>>>>> @Override
>>>>>>> public Response toResponse(WebApplicationException exception) {
>>>>>>> return Response.status(404).build();
>>>>>>> }
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>>>>
>>>>>>>
>>>>>>> However, without this class, Jersey performed even faster - most of
>>>>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>>>>
>>>>>
>>>>> Forgot to comment to this one... I think the fact that both frameworks
>>>>> show approximately the same time with SimpleApplicationExceptionMapper
>>>>> in place and that Jersey scales really well without it is that Jersey
>>>>> basically either writes to HTTP out stream or propagates the exception
>>>>> up to the ServletException filter immediately if no custom mapper is
>>>>> available.
>>>>> In case of CXF the immediate propagation will also happen if no custom
>>>>> mapper is available. The reason the default WebApplicationException
>>>>> Mapper was installed was to do with the fact that few users were not
>>>>> quite happy with having to write their custom mappers in order to report
>>>>> differently to the exceptions generated at the runtime level or some of
>>>>> the providers' level or indeed having to write a filter catching the
>>>>> escaped exceptions there...
>>>>>
>>>>> What I might need to do additionally is to introduce a contextual
>>>>> property that will tell the runtime to ignore even the default
>>>>> WebApplicationException mapper, for example, when the user wishes to
>>>>> get all the exceptions propagated to the filter level, to simplify the
>>>>> performance data comparison, etc
>>>>>
>>>>> Cheers, Sergey
>>>>>
>>>>>>>
>>>>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>>>>> where the discrepancies come from.
>>>>>>>
>>>>>> I've opened
>>>>>> https://issues.apache.org/jira/browse/CXF-4121
>>>>>>
>>>>>> I suspect now that it is the specific lines which get the basic warning
>>>>>> message from the resource bundle are main 'culprits', I guess I will
>>>>>> just have an internal final static String - will need to experiment a
>>>>>> bit...It is good even the default providers can be customized/replaced,
>>>>>> but I'll look into trying to make the default exception mapper
>>>>>> performing a bit better :-)
>>>>>>
>>>>>> Thanks, Sergey
>>>>>>>
>>>>>>> Kindest Regards,
>>>>>>> Krum.
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>>>>> To: users@cxf.apache.org
>>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>>
>>>>>>> Hi Krum,
>>>>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>>>>> Hi to all Apache CXF users!,
>>>>>>>>
>>>>>>>> I would kindly like to bring to your attention a particular problem
>>>>>>>> that we are currently being stuck into. We are experimenting with a
>>>>>>>> simple web application, which exposes its functionality via REST. We
>>>>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>>>>> REST frameworks.
>>>>>>>>
>>>>>>>> For calculating our performance measurements, we simply execute REST
>>>>>>>> requests, and in the client we measure the time that the roundtrip
>>>>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>>>>> favor of Jersey being much faster - in the cases where we test
>>>>>>>> against non-existing resources. In all other cases, the numbers we
>>>>>>>> got were really very close for the two REST frameworks.
>>>>>>>>
>>>>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>>>>> we are not using CXF in its proper configuration. Could you please
>>>>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>>>>> so that we get the best out of it.
>>>>>>>>
>>>>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>>>>> application logic, in the case when the path is not correct, we throw
>>>>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>>>>> this exception is getting caught in CXF, and heavily processed
>>>>>>>> further down the stack.
>>>>>>>>
>>>>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>>>>> processing to try to map the URI to some existing REST resource, I
>>>>>>>> don't know ... maybe performance here could depend on the particular
>>>>>>>> JSON framework that is being used. ...
>>>>>>>>
>>>>>>>
>>>>>>> I suspect it could be the default WebApplicationExceptionMapper
>>>>>>> which is
>>>>>>> to blame.
>>>>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>>>>> any other regular response, but I'm not sure it would explain the
>>>>>>> difference.
>>>>>>>
>>>>>>> Can you give me a favor please and register a custom
>>>>>>> WebApplicationException mapper which would only return:
>>>>>>>
>>>>>>> Response.status(exception.getStatus()).build() ?
>>>>>>>
>>>>>>> Have a look please at the default mapper's code:
>>>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> As you can see, we are trying to go to some length there to figure out
>>>>>>> what and how to report a given exception and I reckon it all adds
>>>>>>> up to
>>>>>>> the response time.
>>>>>>>
>>>>>>> Please try eliminating the default mapper and let us know the results
>>>>>>>
>>>>>>> Thanks, Sergey
>>>>>>>
>>>>>>>>
>>>>>>>> Kindest Regards,
>>>>>>>> Krum.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>> --
>> Sergey Beryozkin
>>
>> Talend Community Coders
>> http://coders.talend.com/
>>
>> Blog: http://sberyozkin.blogspot.com
>
>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

RE: REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Just a short update: the only way I could successfully enter my custom exception mapper with the debugger, was to annotate it with the @Provider annotation: I found that after debugging this<http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cxf/cxf-rt-frontend-jaxrs/2.5.1/org/apache/cxf/jaxrs/utils/ResourceUtils.java> code  (if (c.getAnnotation<http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Class.java>(Provider<http://grepcode.com/file/repo1.maven.org/maven2/javax.ws.rs/jsr311-api/1.1.1/javax/ws/rs/ext/Provider.java>.class) != null) )

Still 404 is slow ... So seems to be not an exception mapper issue...

-----Original Message-----
From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
Sent: Wednesday, February 22, 2012 5:17 PM
To: users@cxf.apache.org
Subject: Re: REST performance discrepancies between CXF and Jersey

On 22/02/12 15:11, Bakalsky, Krum wrote:
> Yes, that's right :)
>
> So, maybe the recipe for being faster here is: just throw some exception within getClasses()  ;-) - just kidding...

LOL :-)

>
> Maybe I should try to see with some breakpoints where exactly the work flow goes through ...

I'm really intrigued now in seeing where the cost savings are coming
from in this case :-), please try to check if you can get a chance...
>
> Anyway, do you know how to register custom exception mappers, in the non-Spring case ? Is the getClasses() result the only way possible for that ?

as far as providers are concerned, they can be returned in either
getClasses() - the runtime will have to load them or in getSingletons(),
alongside the root resource classes if any...

I'm finalizing my current main task soon, so will be looking into it all
seriously soon

Cheers, Sergey

>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Wednesday, February 22, 2012 4:30 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi, By the way,
>
> On 22/02/12 14:19, Sergey Beryozkin wrote:
>> Hi Krum,
>>
>> Before commenting further, let me ask you how do you take the
>> measurements ? Do you check on the client side ? On the server side ?
>>
>> Thanks, Sergey
>>
>>
>> On 22/02/12 13:49, Bakalsky, Krum wrote:
>>> Hi Sergey, again,
>>>
>>> I am afraid that we are in the middle of some big misunderstanding, me
>>> being the one who has started it. Unfortunately, it turned out that I
>>> didn't quite realize, that javax.ws.rs.core.Application#getClasses()
>>> returns a non-modifiable collection (Collections.emptySet()), and in
>>> my exception mapper I used:
>>>
>>> public Set<Class<?>>  getClasses() {
>>> // TODO Auto-generated method stub
>>> Set<Class<?>>  classes = super.getClasses();
>>> classes.add(SimpleApplicationExceptionMapper.class);
>>> return classes;
>>> }
>>>
>>> which probably leads to an java.lang.UnsupportedOperationException at
>>> runtime, and thus to some work flow/behavior that I am not aware of.
>>> This made mu numbers faster, but obviously the scenario is broken, so
>>> the results are irrelevant.
>
> This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored
> getClasses() due to this exception, then it means that the default CXF
> mapper should've been used, otherwise you'd not see 404 on the client
> side, you'd just get 500 due to your custom exception simply escaping.
> Yet apparently the results have become better :-) - thins is something
> I'd like to understand...
>
>
>>>
>>> Next, I took a look at some examples, like
>>> this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
>>> one, and modified my code accordingly. Unfortunately, I got the same
>>> results: 0.500 seconds for the 404 cases. I am not even sure that my
>>> exception mapper was really used, since when I inserted some dumps,
>>> and even when I threw some exceptions in my toResponse() method, I
>>> didn't see them nowhere.
>>>
>>> Next, I made a step further, and patched the
>>> WebApplicationExceptionMapper that comes with CXF to look like this:
>>>
>>> public class WebApplicationExceptionMapper implements
>>> ExceptionMapper<WebApplicationException>  {
>>>
>>> public Response toResponse(WebApplicationException ex) {
>>> return ex.getResponse();
>>> }
>>> }
>>>
>>> I made sure that it was really used. Unfortunately, I got the same
>>> results: roundtrip time was again ~0.500 seconds.
>>>
>>> I am really not sure whether the CXF-4121 bug makes sense any more.
>>> Problem is still here ;(
>>>
>
> Sure, we'll get to the bottom of it, however, lets try to see where the
> unexpected improvement after registering a custom mapper came from...
>
> Actually, I think I may know why your custom mapper may be ignored. It
> should have a @Provider annotation, for it to be recognized as a
> provider in the list returned from getClasses()
>
> Cheers, Sergey
>
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Bakalsky, Krum
>>> Sent: Wednesday, February 22, 2012 11:35 AM
>>> To: 'users@cxf.apache.org'
>>> Subject: RE: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi Sergey,
>>>
>>> I have just made a test run with the modified WebApplicationException
>>> mapper that you have given a link to below, and it works fine: it also
>>> shows 10 times decrease of roundtrip duration as well. So, it seems
>>> that you have really optimized it pretty good already.
>>>
>>> Sure, I don't have any particular objections or concerns as to how to
>>> approach the change that we are talking about. I'd guess that as long
>>> as the application has a way to configure its own exception mapper
>>> (which is already the case) so that 404 overhead is gone for it,
>>> things are fine. I wouldn't know for those implications and
>>> complications that adding a property could lead to. Thanks for sharing.
>>>
>>> We haven't identified particular 'slow' code fragments, other than the
>>> 404 processing. If we do, we will sure let you know. (or if you want,
>>> you could share your suspicions and give us some hint of where to take
>>> a look at ?)
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Tuesday, February 21, 2012 8:34 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi Krum,
>>> On 20/02/12 16:47, Bakalsky, Krum wrote:
>>>> Hi,
>>>>
>>>> Yes, thanks for the comments, they sound very logical to explaining
>>>> the numbers from all the variants we tested.
>>>> Didn't know that the default exception mapper was developed as a
>>>> result of a user demand as well.
>>>>
>>>> For the record: how could I disable it ? I am registering the Simple
>>>> mapper within the returned set of the Application#getClasses() method
>>>> call - so maybe I should remove it there, i.e. from the
>>>> super.getClasses() list, or it is in a different manner ?
>>>>
>>>> Yes, I think that a servlet context parameter would be the most
>>>> flexible and elegant solution, for configuring the desired behavior
>>>> for non-200/exceptional cases.
>>>>
>>> I've spent some today on optimizing a couple of things,
>>> see the updated default WebApplicationException mapper:
>>>
>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>
>>>
>>> I think it's more effective now, yes still it is configurable and able
>>> to interact with CXF FaultLoggers and optionally fine-log the exception
>>> message.
>>>
>>> I was thinking yesterday of adding a property to get this default mapper
>>> ignored. I'm not sure now it's worth going this route.
>>>
>>> One way to 'disable' it is to offer a custom WebApplicationException
>>> mapper, something which we've explored. Another one - is to add this
>>> would be property but I've realized that it's not that good for CXF
>>> after all: this would require the CXF servlet to process unmapped
>>> WebApplicationExceptions to be compliant and this would effectively mean
>>> repeating at the servlet level what is done at the CXF chain level but
>>> with limitations, ex, the Response produced by the mapper would still be
>>> routed through the out filters and indeed the optional CXF out faulty
>>> interceptors which can be handy. Say we can have an out filter which
>>> time-stamps all the responses - this would be ignored at the servlet
>>> level and would effectively have to be duplicated if needed...
>>>
>>> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
>>> will not be necessarily installed in some OSGI containers by default.
>>>
>>> So it appears to be the best way is to optimize the default mapper which
>>> I hope I did, and it would be always possible to offer custom mappers
>>> instead on per-Application basis.
>>>
>>> Let me know please if you have any concerns with the above,
>>>
>>> In meantime I also updated some code on the in chain to optimize the way
>>> the HTTP path to match is calculated - it may well save us few
>>> milliseconds :-)
>>>
>>> If you can share some more results from your tests which point to few
>>> unoptimized code spots then it would be good; we are going to review few
>>> code paths anyway,
>>>
>>> Thanks, Sergey
>>>
>>>>
>>>> Kindest Regards and many thanks again,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>> Sent: Monday, February 20, 2012 5:49 PM
>>>> To: users@cxf.apache.org
>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi,
>>>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>>>> Hi Krum
>>>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>>>> Hi Sergey,
>>>>>>
>>>>>> Wow! I tested your idea and it produced dramatically better results!
>>>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>>>> investigation, and I believe that our observation is really true that
>>>>>> the overhead is spent in the error handling code. Just out of
>>>>>> curiosity: with the same SimpleApplicationMapper:
>>>>>>
>>>>> Thanks for the confirmation, so we've identified the spot where the
>>>>> problem is :-)
>>>>>
>>>>>> public class SimpleApplicationExceptionMapper implements
>>>>>> ExceptionMapper<WebApplicationException>  {
>>>>>>
>>>>>> @Override
>>>>>> public Response toResponse(WebApplicationException exception) {
>>>>>> return Response.status(404).build();
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>>>
>>>>>>
>>>>>> However, without this class, Jersey performed even faster - most of
>>>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>>>
>>>>
>>>> Forgot to comment to this one... I think the fact that both frameworks
>>>> show approximately the same time with SimpleApplicationExceptionMapper
>>>> in place and that Jersey scales really well without it is that Jersey
>>>> basically either writes to HTTP out stream or propagates the exception
>>>> up to the ServletException filter immediately if no custom mapper is
>>>> available.
>>>> In case of CXF the immediate propagation will also happen if no custom
>>>> mapper is available. The reason the default WebApplicationException
>>>> Mapper was installed was to do with the fact that few users were not
>>>> quite happy with having to write their custom mappers in order to report
>>>> differently to the exceptions generated at the runtime level or some of
>>>> the providers' level or indeed having to write a filter catching the
>>>> escaped exceptions there...
>>>>
>>>> What I might need to do additionally is to introduce a contextual
>>>> property that will tell the runtime to ignore even the default
>>>> WebApplicationException mapper, for example, when the user wishes to
>>>> get all the exceptions propagated to the filter level, to simplify the
>>>> performance data comparison, etc
>>>>
>>>> Cheers, Sergey
>>>>
>>>>>>
>>>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>>>> where the discrepancies come from.
>>>>>>
>>>>> I've opened
>>>>> https://issues.apache.org/jira/browse/CXF-4121
>>>>>
>>>>> I suspect now that it is the specific lines which get the basic warning
>>>>> message from the resource bundle are main 'culprits', I guess I will
>>>>> just have an internal final static String - will need to experiment a
>>>>> bit...It is good even the default providers can be customized/replaced,
>>>>> but I'll look into trying to make the default exception mapper
>>>>> performing a bit better :-)
>>>>>
>>>>> Thanks, Sergey
>>>>>>
>>>>>> Kindest Regards,
>>>>>> Krum.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>>>> To: users@cxf.apache.org
>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>
>>>>>> Hi Krum,
>>>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>>>> Hi to all Apache CXF users!,
>>>>>>>
>>>>>>> I would kindly like to bring to your attention a particular problem
>>>>>>> that we are currently being stuck into. We are experimenting with a
>>>>>>> simple web application, which exposes its functionality via REST. We
>>>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>>>> REST frameworks.
>>>>>>>
>>>>>>> For calculating our performance measurements, we simply execute REST
>>>>>>> requests, and in the client we measure the time that the roundtrip
>>>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>>>> favor of Jersey being much faster - in the cases where we test
>>>>>>> against non-existing resources. In all other cases, the numbers we
>>>>>>> got were really very close for the two REST frameworks.
>>>>>>>
>>>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>>>> we are not using CXF in its proper configuration. Could you please
>>>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>>>> so that we get the best out of it.
>>>>>>>
>>>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>>>> application logic, in the case when the path is not correct, we throw
>>>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>>>> this exception is getting caught in CXF, and heavily processed
>>>>>>> further down the stack.
>>>>>>>
>>>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>>>> processing to try to map the URI to some existing REST resource, I
>>>>>>> don't know ... maybe performance here could depend on the particular
>>>>>>> JSON framework that is being used. ...
>>>>>>>
>>>>>>
>>>>>> I suspect it could be the default WebApplicationExceptionMapper
>>>>>> which is
>>>>>> to blame.
>>>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>>>> any other regular response, but I'm not sure it would explain the
>>>>>> difference.
>>>>>>
>>>>>> Can you give me a favor please and register a custom
>>>>>> WebApplicationException mapper which would only return:
>>>>>>
>>>>>> Response.status(exception.getStatus()).build() ?
>>>>>>
>>>>>> Have a look please at the default mapper's code:
>>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>>
>>>>>>
>>>>>>
>>>>>> As you can see, we are trying to go to some length there to figure out
>>>>>> what and how to report a given exception and I reckon it all adds
>>>>>> up to
>>>>>> the response time.
>>>>>>
>>>>>> Please try eliminating the default mapper and let us know the results
>>>>>>
>>>>>> Thanks, Sergey
>>>>>>
>>>>>>>
>>>>>>> Kindest Regards,
>>>>>>> Krum.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
> --
> Sergey Beryozkin
>
> Talend Community Coders
> http://coders.talend.com/
>
> Blog: http://sberyozkin.blogspot.com

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

On 22/02/12 15:11, Bakalsky, Krum wrote:
> Yes, that's right :)
>
> So, maybe the recipe for being faster here is: just throw some exception within getClasses()  ;-) - just kidding...

LOL :-)

>
> Maybe I should try to see with some breakpoints where exactly the work flow goes through ...

I'm really intrigued now in seeing where the cost savings are coming 
from in this case :-), please try to check if you can get a chance...
>
> Anyway, do you know how to register custom exception mappers, in the non-Spring case ? Is the getClasses() result the only way possible for that ?

as far as providers are concerned, they can be returned in either 
getClasses() - the runtime will have to load them or in getSingletons(), 
alongside the root resource classes if any...

I'm finalizing my current main task soon, so will be looking into it all 
seriously soon

Cheers, Sergey

>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Wednesday, February 22, 2012 4:30 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi, By the way,
>
> On 22/02/12 14:19, Sergey Beryozkin wrote:
>> Hi Krum,
>>
>> Before commenting further, let me ask you how do you take the
>> measurements ? Do you check on the client side ? On the server side ?
>>
>> Thanks, Sergey
>>
>>
>> On 22/02/12 13:49, Bakalsky, Krum wrote:
>>> Hi Sergey, again,
>>>
>>> I am afraid that we are in the middle of some big misunderstanding, me
>>> being the one who has started it. Unfortunately, it turned out that I
>>> didn't quite realize, that javax.ws.rs.core.Application#getClasses()
>>> returns a non-modifiable collection (Collections.emptySet()), and in
>>> my exception mapper I used:
>>>
>>> public Set<Class<?>>  getClasses() {
>>> // TODO Auto-generated method stub
>>> Set<Class<?>>  classes = super.getClasses();
>>> classes.add(SimpleApplicationExceptionMapper.class);
>>> return classes;
>>> }
>>>
>>> which probably leads to an java.lang.UnsupportedOperationException at
>>> runtime, and thus to some work flow/behavior that I am not aware of.
>>> This made mu numbers faster, but obviously the scenario is broken, so
>>> the results are irrelevant.
>
> This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored
> getClasses() due to this exception, then it means that the default CXF
> mapper should've been used, otherwise you'd not see 404 on the client
> side, you'd just get 500 due to your custom exception simply escaping.
> Yet apparently the results have become better :-) - thins is something
> I'd like to understand...
>
>
>>>
>>> Next, I took a look at some examples, like
>>> this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
>>> one, and modified my code accordingly. Unfortunately, I got the same
>>> results: 0.500 seconds for the 404 cases. I am not even sure that my
>>> exception mapper was really used, since when I inserted some dumps,
>>> and even when I threw some exceptions in my toResponse() method, I
>>> didn't see them nowhere.
>>>
>>> Next, I made a step further, and patched the
>>> WebApplicationExceptionMapper that comes with CXF to look like this:
>>>
>>> public class WebApplicationExceptionMapper implements
>>> ExceptionMapper<WebApplicationException>  {
>>>
>>> public Response toResponse(WebApplicationException ex) {
>>> return ex.getResponse();
>>> }
>>> }
>>>
>>> I made sure that it was really used. Unfortunately, I got the same
>>> results: roundtrip time was again ~0.500 seconds.
>>>
>>> I am really not sure whether the CXF-4121 bug makes sense any more.
>>> Problem is still here ;(
>>>
>
> Sure, we'll get to the bottom of it, however, lets try to see where the
> unexpected improvement after registering a custom mapper came from...
>
> Actually, I think I may know why your custom mapper may be ignored. It
> should have a @Provider annotation, for it to be recognized as a
> provider in the list returned from getClasses()
>
> Cheers, Sergey
>
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Bakalsky, Krum
>>> Sent: Wednesday, February 22, 2012 11:35 AM
>>> To: 'users@cxf.apache.org'
>>> Subject: RE: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi Sergey,
>>>
>>> I have just made a test run with the modified WebApplicationException
>>> mapper that you have given a link to below, and it works fine: it also
>>> shows 10 times decrease of roundtrip duration as well. So, it seems
>>> that you have really optimized it pretty good already.
>>>
>>> Sure, I don't have any particular objections or concerns as to how to
>>> approach the change that we are talking about. I'd guess that as long
>>> as the application has a way to configure its own exception mapper
>>> (which is already the case) so that 404 overhead is gone for it,
>>> things are fine. I wouldn't know for those implications and
>>> complications that adding a property could lead to. Thanks for sharing.
>>>
>>> We haven't identified particular 'slow' code fragments, other than the
>>> 404 processing. If we do, we will sure let you know. (or if you want,
>>> you could share your suspicions and give us some hint of where to take
>>> a look at ?)
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Tuesday, February 21, 2012 8:34 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi Krum,
>>> On 20/02/12 16:47, Bakalsky, Krum wrote:
>>>> Hi,
>>>>
>>>> Yes, thanks for the comments, they sound very logical to explaining
>>>> the numbers from all the variants we tested.
>>>> Didn't know that the default exception mapper was developed as a
>>>> result of a user demand as well.
>>>>
>>>> For the record: how could I disable it ? I am registering the Simple
>>>> mapper within the returned set of the Application#getClasses() method
>>>> call - so maybe I should remove it there, i.e. from the
>>>> super.getClasses() list, or it is in a different manner ?
>>>>
>>>> Yes, I think that a servlet context parameter would be the most
>>>> flexible and elegant solution, for configuring the desired behavior
>>>> for non-200/exceptional cases.
>>>>
>>> I've spent some today on optimizing a couple of things,
>>> see the updated default WebApplicationException mapper:
>>>
>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>
>>>
>>> I think it's more effective now, yes still it is configurable and able
>>> to interact with CXF FaultLoggers and optionally fine-log the exception
>>> message.
>>>
>>> I was thinking yesterday of adding a property to get this default mapper
>>> ignored. I'm not sure now it's worth going this route.
>>>
>>> One way to 'disable' it is to offer a custom WebApplicationException
>>> mapper, something which we've explored. Another one - is to add this
>>> would be property but I've realized that it's not that good for CXF
>>> after all: this would require the CXF servlet to process unmapped
>>> WebApplicationExceptions to be compliant and this would effectively mean
>>> repeating at the servlet level what is done at the CXF chain level but
>>> with limitations, ex, the Response produced by the mapper would still be
>>> routed through the out filters and indeed the optional CXF out faulty
>>> interceptors which can be handy. Say we can have an out filter which
>>> time-stamps all the responses - this would be ignored at the servlet
>>> level and would effectively have to be duplicated if needed...
>>>
>>> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
>>> will not be necessarily installed in some OSGI containers by default.
>>>
>>> So it appears to be the best way is to optimize the default mapper which
>>> I hope I did, and it would be always possible to offer custom mappers
>>> instead on per-Application basis.
>>>
>>> Let me know please if you have any concerns with the above,
>>>
>>> In meantime I also updated some code on the in chain to optimize the way
>>> the HTTP path to match is calculated - it may well save us few
>>> milliseconds :-)
>>>
>>> If you can share some more results from your tests which point to few
>>> unoptimized code spots then it would be good; we are going to review few
>>> code paths anyway,
>>>
>>> Thanks, Sergey
>>>
>>>>
>>>> Kindest Regards and many thanks again,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>> Sent: Monday, February 20, 2012 5:49 PM
>>>> To: users@cxf.apache.org
>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi,
>>>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>>>> Hi Krum
>>>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>>>> Hi Sergey,
>>>>>>
>>>>>> Wow! I tested your idea and it produced dramatically better results!
>>>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>>>> investigation, and I believe that our observation is really true that
>>>>>> the overhead is spent in the error handling code. Just out of
>>>>>> curiosity: with the same SimpleApplicationMapper:
>>>>>>
>>>>> Thanks for the confirmation, so we've identified the spot where the
>>>>> problem is :-)
>>>>>
>>>>>> public class SimpleApplicationExceptionMapper implements
>>>>>> ExceptionMapper<WebApplicationException>  {
>>>>>>
>>>>>> @Override
>>>>>> public Response toResponse(WebApplicationException exception) {
>>>>>> return Response.status(404).build();
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>>>
>>>>>>
>>>>>> However, without this class, Jersey performed even faster - most of
>>>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>>>
>>>>
>>>> Forgot to comment to this one... I think the fact that both frameworks
>>>> show approximately the same time with SimpleApplicationExceptionMapper
>>>> in place and that Jersey scales really well without it is that Jersey
>>>> basically either writes to HTTP out stream or propagates the exception
>>>> up to the ServletException filter immediately if no custom mapper is
>>>> available.
>>>> In case of CXF the immediate propagation will also happen if no custom
>>>> mapper is available. The reason the default WebApplicationException
>>>> Mapper was installed was to do with the fact that few users were not
>>>> quite happy with having to write their custom mappers in order to report
>>>> differently to the exceptions generated at the runtime level or some of
>>>> the providers' level or indeed having to write a filter catching the
>>>> escaped exceptions there...
>>>>
>>>> What I might need to do additionally is to introduce a contextual
>>>> property that will tell the runtime to ignore even the default
>>>> WebApplicationException mapper, for example, when the user wishes to
>>>> get all the exceptions propagated to the filter level, to simplify the
>>>> performance data comparison, etc
>>>>
>>>> Cheers, Sergey
>>>>
>>>>>>
>>>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>>>> where the discrepancies come from.
>>>>>>
>>>>> I've opened
>>>>> https://issues.apache.org/jira/browse/CXF-4121
>>>>>
>>>>> I suspect now that it is the specific lines which get the basic warning
>>>>> message from the resource bundle are main 'culprits', I guess I will
>>>>> just have an internal final static String - will need to experiment a
>>>>> bit...It is good even the default providers can be customized/replaced,
>>>>> but I'll look into trying to make the default exception mapper
>>>>> performing a bit better :-)
>>>>>
>>>>> Thanks, Sergey
>>>>>>
>>>>>> Kindest Regards,
>>>>>> Krum.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>>>> To: users@cxf.apache.org
>>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>>
>>>>>> Hi Krum,
>>>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>>>> Hi to all Apache CXF users!,
>>>>>>>
>>>>>>> I would kindly like to bring to your attention a particular problem
>>>>>>> that we are currently being stuck into. We are experimenting with a
>>>>>>> simple web application, which exposes its functionality via REST. We
>>>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>>>> REST frameworks.
>>>>>>>
>>>>>>> For calculating our performance measurements, we simply execute REST
>>>>>>> requests, and in the client we measure the time that the roundtrip
>>>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>>>> favor of Jersey being much faster - in the cases where we test
>>>>>>> against non-existing resources. In all other cases, the numbers we
>>>>>>> got were really very close for the two REST frameworks.
>>>>>>>
>>>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>>>> we are not using CXF in its proper configuration. Could you please
>>>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>>>> so that we get the best out of it.
>>>>>>>
>>>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>>>> application logic, in the case when the path is not correct, we throw
>>>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>>>> this exception is getting caught in CXF, and heavily processed
>>>>>>> further down the stack.
>>>>>>>
>>>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>>>> processing to try to map the URI to some existing REST resource, I
>>>>>>> don't know ... maybe performance here could depend on the particular
>>>>>>> JSON framework that is being used. ...
>>>>>>>
>>>>>>
>>>>>> I suspect it could be the default WebApplicationExceptionMapper
>>>>>> which is
>>>>>> to blame.
>>>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>>>> any other regular response, but I'm not sure it would explain the
>>>>>> difference.
>>>>>>
>>>>>> Can you give me a favor please and register a custom
>>>>>> WebApplicationException mapper which would only return:
>>>>>>
>>>>>> Response.status(exception.getStatus()).build() ?
>>>>>>
>>>>>> Have a look please at the default mapper's code:
>>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>>
>>>>>>
>>>>>>
>>>>>> As you can see, we are trying to go to some length there to figure out
>>>>>> what and how to report a given exception and I reckon it all adds
>>>>>> up to
>>>>>> the response time.
>>>>>>
>>>>>> Please try eliminating the default mapper and let us know the results
>>>>>>
>>>>>> Thanks, Sergey
>>>>>>
>>>>>>>
>>>>>>> Kindest Regards,
>>>>>>> Krum.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
> --
> Sergey Beryozkin
>
> Talend Community Coders
> http://coders.talend.com/
>
> Blog: http://sberyozkin.blogspot.com

RE: REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Yes, that's right :)

So, maybe the recipe for being faster here is: just throw some exception within getClasses()  ;-) - just kidding...

Maybe I should try to see with some breakpoints where exactly the work flow goes through ...

Anyway, do you know how to register custom exception mappers, in the non-Spring case ? Is the getClasses() result the only way possible for that ?

-----Original Message-----
From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
Sent: Wednesday, February 22, 2012 4:30 PM
To: users@cxf.apache.org
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi, By the way,

On 22/02/12 14:19, Sergey Beryozkin wrote:
> Hi Krum,
>
> Before commenting further, let me ask you how do you take the
> measurements ? Do you check on the client side ? On the server side ?
>
> Thanks, Sergey
>
>
> On 22/02/12 13:49, Bakalsky, Krum wrote:
>> Hi Sergey, again,
>>
>> I am afraid that we are in the middle of some big misunderstanding, me
>> being the one who has started it. Unfortunately, it turned out that I
>> didn't quite realize, that javax.ws.rs.core.Application#getClasses()
>> returns a non-modifiable collection (Collections.emptySet()), and in
>> my exception mapper I used:
>>
>> public Set<Class<?>> getClasses() {
>> // TODO Auto-generated method stub
>> Set<Class<?>> classes = super.getClasses();
>> classes.add(SimpleApplicationExceptionMapper.class);
>> return classes;
>> }
>>
>> which probably leads to an java.lang.UnsupportedOperationException at
>> runtime, and thus to some work flow/behavior that I am not aware of.
>> This made mu numbers faster, but obviously the scenario is broken, so
>> the results are irrelevant.

This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored
getClasses() due to this exception, then it means that the default CXF
mapper should've been used, otherwise you'd not see 404 on the client
side, you'd just get 500 due to your custom exception simply escaping.
Yet apparently the results have become better :-) - thins is something
I'd like to understand...


>>
>> Next, I took a look at some examples, like
>> this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
>> one, and modified my code accordingly. Unfortunately, I got the same
>> results: 0.500 seconds for the 404 cases. I am not even sure that my
>> exception mapper was really used, since when I inserted some dumps,
>> and even when I threw some exceptions in my toResponse() method, I
>> didn't see them nowhere.
>>
>> Next, I made a step further, and patched the
>> WebApplicationExceptionMapper that comes with CXF to look like this:
>>
>> public class WebApplicationExceptionMapper implements
>> ExceptionMapper<WebApplicationException> {
>>
>> public Response toResponse(WebApplicationException ex) {
>> return ex.getResponse();
>> }
>> }
>>
>> I made sure that it was really used. Unfortunately, I got the same
>> results: roundtrip time was again ~0.500 seconds.
>>
>> I am really not sure whether the CXF-4121 bug makes sense any more.
>> Problem is still here ;(
>>

Sure, we'll get to the bottom of it, however, lets try to see where the
unexpected improvement after registering a custom mapper came from...

Actually, I think I may know why your custom mapper may be ignored. It
should have a @Provider annotation, for it to be recognized as a
provider in the list returned from getClasses()

Cheers, Sergey

>>
>> Kindest Regards,
>> Krum.
>>
>> -----Original Message-----
>> From: Bakalsky, Krum
>> Sent: Wednesday, February 22, 2012 11:35 AM
>> To: 'users@cxf.apache.org'
>> Subject: RE: REST performance discrepancies between CXF and Jersey
>>
>> Hi Sergey,
>>
>> I have just made a test run with the modified WebApplicationException
>> mapper that you have given a link to below, and it works fine: it also
>> shows 10 times decrease of roundtrip duration as well. So, it seems
>> that you have really optimized it pretty good already.
>>
>> Sure, I don't have any particular objections or concerns as to how to
>> approach the change that we are talking about. I'd guess that as long
>> as the application has a way to configure its own exception mapper
>> (which is already the case) so that 404 overhead is gone for it,
>> things are fine. I wouldn't know for those implications and
>> complications that adding a property could lead to. Thanks for sharing.
>>
>> We haven't identified particular 'slow' code fragments, other than the
>> 404 processing. If we do, we will sure let you know. (or if you want,
>> you could share your suspicions and give us some hint of where to take
>> a look at ?)
>>
>> Kindest Regards,
>> Krum.
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Tuesday, February 21, 2012 8:34 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi Krum,
>> On 20/02/12 16:47, Bakalsky, Krum wrote:
>>> Hi,
>>>
>>> Yes, thanks for the comments, they sound very logical to explaining
>>> the numbers from all the variants we tested.
>>> Didn't know that the default exception mapper was developed as a
>>> result of a user demand as well.
>>>
>>> For the record: how could I disable it ? I am registering the Simple
>>> mapper within the returned set of the Application#getClasses() method
>>> call - so maybe I should remove it there, i.e. from the
>>> super.getClasses() list, or it is in a different manner ?
>>>
>>> Yes, I think that a servlet context parameter would be the most
>>> flexible and elegant solution, for configuring the desired behavior
>>> for non-200/exceptional cases.
>>>
>> I've spent some today on optimizing a couple of things,
>> see the updated default WebApplicationException mapper:
>>
>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>
>>
>> I think it's more effective now, yes still it is configurable and able
>> to interact with CXF FaultLoggers and optionally fine-log the exception
>> message.
>>
>> I was thinking yesterday of adding a property to get this default mapper
>> ignored. I'm not sure now it's worth going this route.
>>
>> One way to 'disable' it is to offer a custom WebApplicationException
>> mapper, something which we've explored. Another one - is to add this
>> would be property but I've realized that it's not that good for CXF
>> after all: this would require the CXF servlet to process unmapped
>> WebApplicationExceptions to be compliant and this would effectively mean
>> repeating at the servlet level what is done at the CXF chain level but
>> with limitations, ex, the Response produced by the mapper would still be
>> routed through the out filters and indeed the optional CXF out faulty
>> interceptors which can be handy. Say we can have an out filter which
>> time-stamps all the responses - this would be ignored at the servlet
>> level and would effectively have to be duplicated if needed...
>>
>> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
>> will not be necessarily installed in some OSGI containers by default.
>>
>> So it appears to be the best way is to optimize the default mapper which
>> I hope I did, and it would be always possible to offer custom mappers
>> instead on per-Application basis.
>>
>> Let me know please if you have any concerns with the above,
>>
>> In meantime I also updated some code on the in chain to optimize the way
>> the HTTP path to match is calculated - it may well save us few
>> milliseconds :-)
>>
>> If you can share some more results from your tests which point to few
>> unoptimized code spots then it would be good; we are going to review few
>> code paths anyway,
>>
>> Thanks, Sergey
>>
>>>
>>> Kindest Regards and many thanks again,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Monday, February 20, 2012 5:49 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi,
>>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>>> Hi Krum
>>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>>> Hi Sergey,
>>>>>
>>>>> Wow! I tested your idea and it produced dramatically better results!
>>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>>> investigation, and I believe that our observation is really true that
>>>>> the overhead is spent in the error handling code. Just out of
>>>>> curiosity: with the same SimpleApplicationMapper:
>>>>>
>>>> Thanks for the confirmation, so we've identified the spot where the
>>>> problem is :-)
>>>>
>>>>> public class SimpleApplicationExceptionMapper implements
>>>>> ExceptionMapper<WebApplicationException> {
>>>>>
>>>>> @Override
>>>>> public Response toResponse(WebApplicationException exception) {
>>>>> return Response.status(404).build();
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>>
>>>>>
>>>>> However, without this class, Jersey performed even faster - most of
>>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>>
>>>
>>> Forgot to comment to this one... I think the fact that both frameworks
>>> show approximately the same time with SimpleApplicationExceptionMapper
>>> in place and that Jersey scales really well without it is that Jersey
>>> basically either writes to HTTP out stream or propagates the exception
>>> up to the ServletException filter immediately if no custom mapper is
>>> available.
>>> In case of CXF the immediate propagation will also happen if no custom
>>> mapper is available. The reason the default WebApplicationException
>>> Mapper was installed was to do with the fact that few users were not
>>> quite happy with having to write their custom mappers in order to report
>>> differently to the exceptions generated at the runtime level or some of
>>> the providers' level or indeed having to write a filter catching the
>>> escaped exceptions there...
>>>
>>> What I might need to do additionally is to introduce a contextual
>>> property that will tell the runtime to ignore even the default
>>> WebApplicationException mapper, for example, when the user wishes to
>>> get all the exceptions propagated to the filter level, to simplify the
>>> performance data comparison, etc
>>>
>>> Cheers, Sergey
>>>
>>>>>
>>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>>> where the discrepancies come from.
>>>>>
>>>> I've opened
>>>> https://issues.apache.org/jira/browse/CXF-4121
>>>>
>>>> I suspect now that it is the specific lines which get the basic warning
>>>> message from the resource bundle are main 'culprits', I guess I will
>>>> just have an internal final static String - will need to experiment a
>>>> bit...It is good even the default providers can be customized/replaced,
>>>> but I'll look into trying to make the default exception mapper
>>>> performing a bit better :-)
>>>>
>>>> Thanks, Sergey
>>>>>
>>>>> Kindest Regards,
>>>>> Krum.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>>> To: users@cxf.apache.org
>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>
>>>>> Hi Krum,
>>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>>> Hi to all Apache CXF users!,
>>>>>>
>>>>>> I would kindly like to bring to your attention a particular problem
>>>>>> that we are currently being stuck into. We are experimenting with a
>>>>>> simple web application, which exposes its functionality via REST. We
>>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>>> REST frameworks.
>>>>>>
>>>>>> For calculating our performance measurements, we simply execute REST
>>>>>> requests, and in the client we measure the time that the roundtrip
>>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>>> favor of Jersey being much faster - in the cases where we test
>>>>>> against non-existing resources. In all other cases, the numbers we
>>>>>> got were really very close for the two REST frameworks.
>>>>>>
>>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>>> we are not using CXF in its proper configuration. Could you please
>>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>>> so that we get the best out of it.
>>>>>>
>>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>>> application logic, in the case when the path is not correct, we throw
>>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>>> this exception is getting caught in CXF, and heavily processed
>>>>>> further down the stack.
>>>>>>
>>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>>> processing to try to map the URI to some existing REST resource, I
>>>>>> don't know ... maybe performance here could depend on the particular
>>>>>> JSON framework that is being used. ...
>>>>>>
>>>>>
>>>>> I suspect it could be the default WebApplicationExceptionMapper
>>>>> which is
>>>>> to blame.
>>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>>> any other regular response, but I'm not sure it would explain the
>>>>> difference.
>>>>>
>>>>> Can you give me a favor please and register a custom
>>>>> WebApplicationException mapper which would only return:
>>>>>
>>>>> Response.status(exception.getStatus()).build() ?
>>>>>
>>>>> Have a look please at the default mapper's code:
>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>
>>>>>
>>>>>
>>>>> As you can see, we are trying to go to some length there to figure out
>>>>> what and how to report a given exception and I reckon it all adds
>>>>> up to
>>>>> the response time.
>>>>>
>>>>> Please try eliminating the default mapper and let us know the results
>>>>>
>>>>> Thanks, Sergey
>>>>>
>>>>>>
>>>>>> Kindest Regards,
>>>>>> Krum.
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>


--
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi, By the way,

On 22/02/12 14:19, Sergey Beryozkin wrote:
> Hi Krum,
>
> Before commenting further, let me ask you how do you take the
> measurements ? Do you check on the client side ? On the server side ?
>
> Thanks, Sergey
>
>
> On 22/02/12 13:49, Bakalsky, Krum wrote:
>> Hi Sergey, again,
>>
>> I am afraid that we are in the middle of some big misunderstanding, me
>> being the one who has started it. Unfortunately, it turned out that I
>> didn't quite realize, that javax.ws.rs.core.Application#getClasses()
>> returns a non-modifiable collection (Collections.emptySet()), and in
>> my exception mapper I used:
>>
>> public Set<Class<?>> getClasses() {
>> // TODO Auto-generated method stub
>> Set<Class<?>> classes = super.getClasses();
>> classes.add(SimpleApplicationExceptionMapper.class);
>> return classes;
>> }
>>
>> which probably leads to an java.lang.UnsupportedOperationException at
>> runtime, and thus to some work flow/behavior that I am not aware of.
>> This made mu numbers faster, but obviously the scenario is broken, so
>> the results are irrelevant.

This actually surprises me. Assuming CXFNonSpringJaxrsServlet ignored 
getClasses() due to this exception, then it means that the default CXF 
mapper should've been used, otherwise you'd not see 404 on the client 
side, you'd just get 500 due to your custom exception simply escaping.
Yet apparently the results have become better :-) - thins is something 
I'd like to understand...


>>
>> Next, I took a look at some examples, like
>> this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>
>> one, and modified my code accordingly. Unfortunately, I got the same
>> results: 0.500 seconds for the 404 cases. I am not even sure that my
>> exception mapper was really used, since when I inserted some dumps,
>> and even when I threw some exceptions in my toResponse() method, I
>> didn't see them nowhere.
>>
>> Next, I made a step further, and patched the
>> WebApplicationExceptionMapper that comes with CXF to look like this:
>>
>> public class WebApplicationExceptionMapper implements
>> ExceptionMapper<WebApplicationException> {
>>
>> public Response toResponse(WebApplicationException ex) {
>> return ex.getResponse();
>> }
>> }
>>
>> I made sure that it was really used. Unfortunately, I got the same
>> results: roundtrip time was again ~0.500 seconds.
>>
>> I am really not sure whether the CXF-4121 bug makes sense any more.
>> Problem is still here ;(
>>

Sure, we'll get to the bottom of it, however, lets try to see where the 
unexpected improvement after registering a custom mapper came from...

Actually, I think I may know why your custom mapper may be ignored. It 
should have a @Provider annotation, for it to be recognized as a 
provider in the list returned from getClasses()

Cheers, Sergey

>>
>> Kindest Regards,
>> Krum.
>>
>> -----Original Message-----
>> From: Bakalsky, Krum
>> Sent: Wednesday, February 22, 2012 11:35 AM
>> To: 'users@cxf.apache.org'
>> Subject: RE: REST performance discrepancies between CXF and Jersey
>>
>> Hi Sergey,
>>
>> I have just made a test run with the modified WebApplicationException
>> mapper that you have given a link to below, and it works fine: it also
>> shows 10 times decrease of roundtrip duration as well. So, it seems
>> that you have really optimized it pretty good already.
>>
>> Sure, I don't have any particular objections or concerns as to how to
>> approach the change that we are talking about. I'd guess that as long
>> as the application has a way to configure its own exception mapper
>> (which is already the case) so that 404 overhead is gone for it,
>> things are fine. I wouldn't know for those implications and
>> complications that adding a property could lead to. Thanks for sharing.
>>
>> We haven't identified particular 'slow' code fragments, other than the
>> 404 processing. If we do, we will sure let you know. (or if you want,
>> you could share your suspicions and give us some hint of where to take
>> a look at ?)
>>
>> Kindest Regards,
>> Krum.
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Tuesday, February 21, 2012 8:34 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi Krum,
>> On 20/02/12 16:47, Bakalsky, Krum wrote:
>>> Hi,
>>>
>>> Yes, thanks for the comments, they sound very logical to explaining
>>> the numbers from all the variants we tested.
>>> Didn't know that the default exception mapper was developed as a
>>> result of a user demand as well.
>>>
>>> For the record: how could I disable it ? I am registering the Simple
>>> mapper within the returned set of the Application#getClasses() method
>>> call - so maybe I should remove it there, i.e. from the
>>> super.getClasses() list, or it is in a different manner ?
>>>
>>> Yes, I think that a servlet context parameter would be the most
>>> flexible and elegant solution, for configuring the desired behavior
>>> for non-200/exceptional cases.
>>>
>> I've spent some today on optimizing a couple of things,
>> see the updated default WebApplicationException mapper:
>>
>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>
>>
>> I think it's more effective now, yes still it is configurable and able
>> to interact with CXF FaultLoggers and optionally fine-log the exception
>> message.
>>
>> I was thinking yesterday of adding a property to get this default mapper
>> ignored. I'm not sure now it's worth going this route.
>>
>> One way to 'disable' it is to offer a custom WebApplicationException
>> mapper, something which we've explored. Another one - is to add this
>> would be property but I've realized that it's not that good for CXF
>> after all: this would require the CXF servlet to process unmapped
>> WebApplicationExceptions to be compliant and this would effectively mean
>> repeating at the servlet level what is done at the CXF chain level but
>> with limitations, ex, the Response produced by the mapper would still be
>> routed through the out filters and indeed the optional CXF out faulty
>> interceptors which can be handy. Say we can have an out filter which
>> time-stamps all the responses - this would be ignored at the servlet
>> level and would effectively have to be duplicated if needed...
>>
>> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
>> will not be necessarily installed in some OSGI containers by default.
>>
>> So it appears to be the best way is to optimize the default mapper which
>> I hope I did, and it would be always possible to offer custom mappers
>> instead on per-Application basis.
>>
>> Let me know please if you have any concerns with the above,
>>
>> In meantime I also updated some code on the in chain to optimize the way
>> the HTTP path to match is calculated - it may well save us few
>> milliseconds :-)
>>
>> If you can share some more results from your tests which point to few
>> unoptimized code spots then it would be good; we are going to review few
>> code paths anyway,
>>
>> Thanks, Sergey
>>
>>>
>>> Kindest Regards and many thanks again,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Monday, February 20, 2012 5:49 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi,
>>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>>> Hi Krum
>>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>>> Hi Sergey,
>>>>>
>>>>> Wow! I tested your idea and it produced dramatically better results!
>>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>>> investigation, and I believe that our observation is really true that
>>>>> the overhead is spent in the error handling code. Just out of
>>>>> curiosity: with the same SimpleApplicationMapper:
>>>>>
>>>> Thanks for the confirmation, so we've identified the spot where the
>>>> problem is :-)
>>>>
>>>>> public class SimpleApplicationExceptionMapper implements
>>>>> ExceptionMapper<WebApplicationException> {
>>>>>
>>>>> @Override
>>>>> public Response toResponse(WebApplicationException exception) {
>>>>> return Response.status(404).build();
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>>
>>>>>
>>>>> However, without this class, Jersey performed even faster - most of
>>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>>
>>>
>>> Forgot to comment to this one... I think the fact that both frameworks
>>> show approximately the same time with SimpleApplicationExceptionMapper
>>> in place and that Jersey scales really well without it is that Jersey
>>> basically either writes to HTTP out stream or propagates the exception
>>> up to the ServletException filter immediately if no custom mapper is
>>> available.
>>> In case of CXF the immediate propagation will also happen if no custom
>>> mapper is available. The reason the default WebApplicationException
>>> Mapper was installed was to do with the fact that few users were not
>>> quite happy with having to write their custom mappers in order to report
>>> differently to the exceptions generated at the runtime level or some of
>>> the providers' level or indeed having to write a filter catching the
>>> escaped exceptions there...
>>>
>>> What I might need to do additionally is to introduce a contextual
>>> property that will tell the runtime to ignore even the default
>>> WebApplicationException mapper, for example, when the user wishes to
>>> get all the exceptions propagated to the filter level, to simplify the
>>> performance data comparison, etc
>>>
>>> Cheers, Sergey
>>>
>>>>>
>>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>>> where the discrepancies come from.
>>>>>
>>>> I've opened
>>>> https://issues.apache.org/jira/browse/CXF-4121
>>>>
>>>> I suspect now that it is the specific lines which get the basic warning
>>>> message from the resource bundle are main 'culprits', I guess I will
>>>> just have an internal final static String - will need to experiment a
>>>> bit...It is good even the default providers can be customized/replaced,
>>>> but I'll look into trying to make the default exception mapper
>>>> performing a bit better :-)
>>>>
>>>> Thanks, Sergey
>>>>>
>>>>> Kindest Regards,
>>>>> Krum.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>>> To: users@cxf.apache.org
>>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>>
>>>>> Hi Krum,
>>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>>> Hi to all Apache CXF users!,
>>>>>>
>>>>>> I would kindly like to bring to your attention a particular problem
>>>>>> that we are currently being stuck into. We are experimenting with a
>>>>>> simple web application, which exposes its functionality via REST. We
>>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>>> REST frameworks.
>>>>>>
>>>>>> For calculating our performance measurements, we simply execute REST
>>>>>> requests, and in the client we measure the time that the roundtrip
>>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>>> favor of Jersey being much faster - in the cases where we test
>>>>>> against non-existing resources. In all other cases, the numbers we
>>>>>> got were really very close for the two REST frameworks.
>>>>>>
>>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>>> we are not using CXF in its proper configuration. Could you please
>>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>>> so that we get the best out of it.
>>>>>>
>>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>>> application logic, in the case when the path is not correct, we throw
>>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>>> this exception is getting caught in CXF, and heavily processed
>>>>>> further down the stack.
>>>>>>
>>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>>> processing to try to map the URI to some existing REST resource, I
>>>>>> don't know ... maybe performance here could depend on the particular
>>>>>> JSON framework that is being used. ...
>>>>>>
>>>>>
>>>>> I suspect it could be the default WebApplicationExceptionMapper
>>>>> which is
>>>>> to blame.
>>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>>> any other regular response, but I'm not sure it would explain the
>>>>> difference.
>>>>>
>>>>> Can you give me a favor please and register a custom
>>>>> WebApplicationException mapper which would only return:
>>>>>
>>>>> Response.status(exception.getStatus()).build() ?
>>>>>
>>>>> Have a look please at the default mapper's code:
>>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>>
>>>>>
>>>>>
>>>>> As you can see, we are trying to go to some length there to figure out
>>>>> what and how to report a given exception and I reckon it all adds
>>>>> up to
>>>>> the response time.
>>>>>
>>>>> Please try eliminating the default mapper and let us know the results
>>>>>
>>>>> Thanks, Sergey
>>>>>
>>>>>>
>>>>>> Kindest Regards,
>>>>>> Krum.
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi Krum,

Before commenting further, let me ask you how do you take the 
measurements ? Do you check on the client side ? On the server side ?

Thanks, Sergey


On 22/02/12 13:49, Bakalsky, Krum wrote:
> Hi Sergey, again,
>
> I am afraid that we are in the middle of some big misunderstanding, me being the one who has started it. Unfortunately, it turned out that I didn't quite realize, that javax.ws.rs.core.Application#getClasses() returns a non-modifiable collection (Collections.emptySet()), and in my exception mapper I used:
>
>          public Set<Class<?>>  getClasses() {
>                  // TODO Auto-generated method stub
>                  Set<Class<?>>  classes = super.getClasses();
>                  classes.add(SimpleApplicationExceptionMapper.class);
>                  return classes;
>          }
>
> which probably leads to an java.lang.UnsupportedOperationException at runtime, and thus to some work flow/behavior that I am not aware of. This made mu numbers faster, but obviously the scenario is broken, so the results are irrelevant.
>
> Next, I took a look at some examples, like this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f>  one, and modified my code accordingly. Unfortunately, I got the same results: 0.500 seconds for the 404 cases. I am not even sure that my exception mapper was really used, since when I inserted some dumps, and even when I threw some exceptions in my toResponse() method, I didn't see them nowhere.
>
> Next, I made a step further, and patched the WebApplicationExceptionMapper that comes with CXF to look like this:
>
> public class WebApplicationExceptionMapper implements ExceptionMapper<WebApplicationException>  {
>
>          public Response toResponse(WebApplicationException ex) {
>                  return ex.getResponse();
>          }
> }
>
> I made sure that it was really used. Unfortunately, I got the same results: roundtrip time was again ~0.500 seconds.
>
> I am really not sure whether the CXF-4121 bug makes sense any more. Problem is still here ;(
>
>
> Kindest Regards,
> Krum.
>
> -----Original Message-----
> From: Bakalsky, Krum
> Sent: Wednesday, February 22, 2012 11:35 AM
> To: 'users@cxf.apache.org'
> Subject: RE: REST performance discrepancies between CXF and Jersey
>
> Hi Sergey,
>
> I have just made a test run with the modified WebApplicationException mapper that you have given a link to below, and it works fine: it also shows 10 times decrease of roundtrip duration as well. So, it seems that you have really optimized it pretty good already.
>
> Sure, I don't have any particular objections or concerns as to how to approach the change that we are talking about. I'd guess that as long as the application has a way to configure its own exception mapper (which is already the case) so that 404 overhead is gone for it, things are fine. I wouldn't know for those implications and complications that adding a property could lead to. Thanks for sharing.
>
> We haven't identified particular 'slow' code fragments, other than the 404 processing. If we do, we will sure let you know. (or if you want, you could share your suspicions and give us some hint of where to take a look at ?)
>
> Kindest Regards,
> Krum.
>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Tuesday, February 21, 2012 8:34 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi Krum,
> On 20/02/12 16:47, Bakalsky, Krum wrote:
>> Hi,
>>
>> Yes, thanks for the comments, they sound very logical to explaining the numbers from all the variants we tested.
>> Didn't know that the default exception mapper was developed as a result of a user demand as well.
>>
>> For the record: how could I disable it ? I am registering the Simple mapper within the returned set of the Application#getClasses() method call - so maybe I should remove it there, i.e. from the super.getClasses() list, or it is in a different manner ?
>>
>> Yes, I think that a servlet context parameter would be the most flexible and elegant solution, for configuring the desired behavior for non-200/exceptional cases.
>>
> I've spent some today on optimizing a couple of things,
> see the updated default WebApplicationException mapper:
>
> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>
> I think it's more effective now, yes still it is configurable and able
> to interact with CXF FaultLoggers and optionally fine-log the exception
> message.
>
> I was thinking yesterday of adding a property to get this default mapper
> ignored. I'm not sure now it's worth going this route.
>
> One way to 'disable' it is to offer a custom WebApplicationException
> mapper, something which we've explored. Another one - is to add this
> would be property but I've realized that it's not that good for CXF
> after all:  this would require the CXF servlet to process unmapped
> WebApplicationExceptions to be compliant and this would effectively mean
> repeating at the servlet level what is done at the CXF chain level but
> with limitations, ex, the Response produced by the mapper would still be
> routed through the out filters and indeed the optional CXF out faulty
> interceptors which can be handy. Say we can have an out filter which
> time-stamps all the responses - this would be ignored at the servlet
> level and would effectively have to be duplicated if needed...
>
> The other thing is that CXFNonSpringJaxrsServlet and servlet filters
> will not be necessarily installed in some OSGI containers by default.
>
> So it appears to be the best way is to optimize the default mapper which
> I hope I did, and it would be always possible to offer custom mappers
> instead on per-Application basis.
>
> Let me know please if you have any concerns with the above,
>
> In meantime I also updated some code on the in chain to optimize the way
> the HTTP path to match is calculated - it may well save us few
> milliseconds :-)
>
> If you can share some more results from your tests which point to few
> unoptimized code spots then it would be good; we are going to review few
> code paths anyway,
>
> Thanks, Sergey
>
>>
>> Kindest Regards and many thanks again,
>> Krum.
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Monday, February 20, 2012 5:49 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi,
>> On 20/02/12 15:28, Sergey Beryozkin wrote:
>>> Hi Krum
>>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>>> Hi Sergey,
>>>>
>>>> Wow! I tested your idea and it produced dramatically better results!
>>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>>> investigation, and I believe that our observation is really true that
>>>> the overhead is spent in the error handling code. Just out of
>>>> curiosity: with the same SimpleApplicationMapper:
>>>>
>>> Thanks for the confirmation, so we've identified the spot where the
>>> problem is :-)
>>>
>>>> public class SimpleApplicationExceptionMapper implements
>>>> ExceptionMapper<WebApplicationException>   {
>>>>
>>>> @Override
>>>> public Response toResponse(WebApplicationException exception) {
>>>> return Response.status(404).build();
>>>> }
>>>>
>>>> }
>>>>
>>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>>
>>>>
>>>> However, without this class, Jersey performed even faster - most of
>>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>>> the CXF case ... you already know it :) - 0.500 seconds
>>>>
>>
>> Forgot to comment to this one... I think the fact that both frameworks
>> show approximately the same time with SimpleApplicationExceptionMapper
>> in place and that Jersey scales really well without it is that Jersey
>> basically either writes  to HTTP out stream or propagates the exception
>> up to the ServletException filter immediately if no custom mapper is
>> available.
>> In case of CXF the immediate propagation will also happen if no custom
>> mapper is available. The reason the default WebApplicationException
>> Mapper was installed was to do with the fact that few users were not
>> quite happy with having to write their custom mappers in order to report
>> differently to the exceptions generated at the runtime level or some of
>> the providers' level or indeed having to write a filter catching the
>> escaped exceptions there...
>>
>> What I might need to do additionally is to introduce a contextual
>> property that will tell the runtime to ignore even the default
>> WebApplicationException mapper, for example, when the user wishes to
>> get all the exceptions propagated to the filter level, to simplify the
>> performance data comparison, etc
>>
>> Cheers, Sergey
>>
>>>>
>>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>>> where the discrepancies come from.
>>>>
>>> I've opened
>>> https://issues.apache.org/jira/browse/CXF-4121
>>>
>>> I suspect now that it is the specific lines which get the basic warning
>>> message from the resource bundle are main 'culprits', I guess I will
>>> just have an internal final static String - will need to experiment a
>>> bit...It is good even the default providers can be customized/replaced,
>>> but I'll look into trying to make the default exception mapper
>>> performing a bit better :-)
>>>
>>> Thanks, Sergey
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>> -----Original Message-----
>>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>>> Sent: Friday, February 17, 2012 6:14 PM
>>>> To: users@cxf.apache.org
>>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>>
>>>> Hi Krum,
>>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>>> Hi to all Apache CXF users!,
>>>>>
>>>>> I would kindly like to bring to your attention a particular problem
>>>>> that we are currently being stuck into. We are experimenting with a
>>>>> simple web application, which exposes its functionality via REST. We
>>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>>> REST frameworks.
>>>>>
>>>>> For calculating our performance measurements, we simply execute REST
>>>>> requests, and in the client we measure the time that the roundtrip
>>>>> takes. Apart from that, on the server side, we measure as well the
>>>>> time that our REST resource consumes, i.e. the method call time. In
>>>>> summary, we have some huge differences between CXF and Jersey - in
>>>>> favor of Jersey being much faster - in the cases where we test
>>>>> against non-existing resources. In all other cases, the numbers we
>>>>> got were really very close for the two REST frameworks.
>>>>>
>>>>> But when testing against non-existing REST paths, the roundtrip time
>>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>>> we are not using CXF in its proper configuration. Could you please
>>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>>> so that we get the best out of it.
>>>>>
>>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>>> seconds, so we wonder where is that half second spent in ? In our
>>>>> application logic, in the case when the path is not correct, we throw
>>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>>> this exception is getting caught in CXF, and heavily processed
>>>>> further down the stack.
>>>>>
>>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>>> processing to try to map the URI to some existing REST resource, I
>>>>> don't know ... maybe performance here could depend on the particular
>>>>> JSON framework that is being used. ...
>>>>>
>>>>
>>>> I suspect it could be the default WebApplicationExceptionMapper which is
>>>> to blame.
>>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>>> any other regular response, but I'm not sure it would explain the
>>>> difference.
>>>>
>>>> Can you give me a favor please and register a custom
>>>> WebApplicationException mapper which would only return:
>>>>
>>>> Response.status(exception.getStatus()).build() ?
>>>>
>>>> Have a look please at the default mapper's code:
>>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>>
>>>>
>>>> As you can see, we are trying to go to some length there to figure out
>>>> what and how to report a given exception and I reckon it all adds up to
>>>> the response time.
>>>>
>>>> Please try eliminating the default mapper and let us know the results
>>>>
>>>> Thanks, Sergey
>>>>
>>>>>
>>>>> Kindest Regards,
>>>>> Krum.
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>
>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

RE: REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Hi Sergey, again,

I am afraid that we are in the middle of some big misunderstanding, me being the one who has started it. Unfortunately, it turned out that I didn't quite realize, that javax.ws.rs.core.Application#getClasses() returns a non-modifiable collection (Collections.emptySet()), and in my exception mapper I used:

        public Set<Class<?>> getClasses() {
                // TODO Auto-generated method stub
                Set<Class<?>> classes = super.getClasses();
                classes.add(SimpleApplicationExceptionMapper.class);
                return classes;
        }

which probably leads to an java.lang.UnsupportedOperationException at runtime, and thus to some work flow/behavior that I am not aware of. This made mu numbers faster, but obviously the scenario is broken, so the results are irrelevant.

Next, I took a look at some examples, like this<http://grepcode.com/file/repo1.maven.org/maven2/com.sun.jersey.samples/jersey-ejb/1.11/com/sun/jersey/samples/jersey_ejb/resources/MyApplication.java?av=f> one, and modified my code accordingly. Unfortunately, I got the same results: 0.500 seconds for the 404 cases. I am not even sure that my exception mapper was really used, since when I inserted some dumps, and even when I threw some exceptions in my toResponse() method, I didn't see them nowhere.

Next, I made a step further, and patched the WebApplicationExceptionMapper that comes with CXF to look like this:

public class WebApplicationExceptionMapper implements ExceptionMapper<WebApplicationException> {

        public Response toResponse(WebApplicationException ex) {
                return ex.getResponse();
        }
}

I made sure that it was really used. Unfortunately, I got the same results: roundtrip time was again ~0.500 seconds.

I am really not sure whether the CXF-4121 bug makes sense any more. Problem is still here ;(


Kindest Regards,
Krum.

-----Original Message-----
From: Bakalsky, Krum
Sent: Wednesday, February 22, 2012 11:35 AM
To: 'users@cxf.apache.org'
Subject: RE: REST performance discrepancies between CXF and Jersey

Hi Sergey,

I have just made a test run with the modified WebApplicationException mapper that you have given a link to below, and it works fine: it also shows 10 times decrease of roundtrip duration as well. So, it seems that you have really optimized it pretty good already.

Sure, I don't have any particular objections or concerns as to how to approach the change that we are talking about. I'd guess that as long as the application has a way to configure its own exception mapper (which is already the case) so that 404 overhead is gone for it, things are fine. I wouldn't know for those implications and complications that adding a property could lead to. Thanks for sharing.

We haven't identified particular 'slow' code fragments, other than the 404 processing. If we do, we will sure let you know. (or if you want, you could share your suspicions and give us some hint of where to take a look at ?)

Kindest Regards,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
Sent: Tuesday, February 21, 2012 8:34 PM
To: users@cxf.apache.org
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi Krum,
On 20/02/12 16:47, Bakalsky, Krum wrote:
> Hi,
>
> Yes, thanks for the comments, they sound very logical to explaining the numbers from all the variants we tested.
> Didn't know that the default exception mapper was developed as a result of a user demand as well.
>
> For the record: how could I disable it ? I am registering the Simple mapper within the returned set of the Application#getClasses() method call - so maybe I should remove it there, i.e. from the super.getClasses() list, or it is in a different manner ?
>
> Yes, I think that a servlet context parameter would be the most flexible and elegant solution, for configuring the desired behavior for non-200/exceptional cases.
>
I've spent some today on optimizing a couple of things,
see the updated default WebApplicationException mapper:

http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java

I think it's more effective now, yes still it is configurable and able
to interact with CXF FaultLoggers and optionally fine-log the exception
message.

I was thinking yesterday of adding a property to get this default mapper
ignored. I'm not sure now it's worth going this route.

One way to 'disable' it is to offer a custom WebApplicationException
mapper, something which we've explored. Another one - is to add this
would be property but I've realized that it's not that good for CXF
after all:  this would require the CXF servlet to process unmapped
WebApplicationExceptions to be compliant and this would effectively mean
repeating at the servlet level what is done at the CXF chain level but
with limitations, ex, the Response produced by the mapper would still be
routed through the out filters and indeed the optional CXF out faulty
interceptors which can be handy. Say we can have an out filter which
time-stamps all the responses - this would be ignored at the servlet
level and would effectively have to be duplicated if needed...

The other thing is that CXFNonSpringJaxrsServlet and servlet filters
will not be necessarily installed in some OSGI containers by default.

So it appears to be the best way is to optimize the default mapper which
I hope I did, and it would be always possible to offer custom mappers
instead on per-Application basis.

Let me know please if you have any concerns with the above,

In meantime I also updated some code on the in chain to optimize the way
the HTTP path to match is calculated - it may well save us few
milliseconds :-)

If you can share some more results from your tests which point to few
unoptimized code spots then it would be good; we are going to review few
code paths anyway,

Thanks, Sergey

>
> Kindest Regards and many thanks again,
> Krum.
>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Monday, February 20, 2012 5:49 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi,
> On 20/02/12 15:28, Sergey Beryozkin wrote:
>> Hi Krum
>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>> Hi Sergey,
>>>
>>> Wow! I tested your idea and it produced dramatically better results!
>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>> investigation, and I believe that our observation is really true that
>>> the overhead is spent in the error handling code. Just out of
>>> curiosity: with the same SimpleApplicationMapper:
>>>
>> Thanks for the confirmation, so we've identified the spot where the
>> problem is :-)
>>
>>> public class SimpleApplicationExceptionMapper implements
>>> ExceptionMapper<WebApplicationException>  {
>>>
>>> @Override
>>> public Response toResponse(WebApplicationException exception) {
>>> return Response.status(404).build();
>>> }
>>>
>>> }
>>>
>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>
>>>
>>> However, without this class, Jersey performed even faster - most of
>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>> the CXF case ... you already know it :) - 0.500 seconds
>>>
>
> Forgot to comment to this one... I think the fact that both frameworks
> show approximately the same time with SimpleApplicationExceptionMapper
> in place and that Jersey scales really well without it is that Jersey
> basically either writes  to HTTP out stream or propagates the exception
> up to the ServletException filter immediately if no custom mapper is
> available.
> In case of CXF the immediate propagation will also happen if no custom
> mapper is available. The reason the default WebApplicationException
> Mapper was installed was to do with the fact that few users were not
> quite happy with having to write their custom mappers in order to report
> differently to the exceptions generated at the runtime level or some of
> the providers' level or indeed having to write a filter catching the
> escaped exceptions there...
>
> What I might need to do additionally is to introduce a contextual
> property that will tell the runtime to ignore even the default
> WebApplicationException mapper, for example, when the user wishes to
> get all the exceptions propagated to the filter level, to simplify the
> performance data comparison, etc
>
> Cheers, Sergey
>
>>>
>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>> where the discrepancies come from.
>>>
>> I've opened
>> https://issues.apache.org/jira/browse/CXF-4121
>>
>> I suspect now that it is the specific lines which get the basic warning
>> message from the resource bundle are main 'culprits', I guess I will
>> just have an internal final static String - will need to experiment a
>> bit...It is good even the default providers can be customized/replaced,
>> but I'll look into trying to make the default exception mapper
>> performing a bit better :-)
>>
>> Thanks, Sergey
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Friday, February 17, 2012 6:14 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi Krum,
>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>> Hi to all Apache CXF users!,
>>>>
>>>> I would kindly like to bring to your attention a particular problem
>>>> that we are currently being stuck into. We are experimenting with a
>>>> simple web application, which exposes its functionality via REST. We
>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>> REST frameworks.
>>>>
>>>> For calculating our performance measurements, we simply execute REST
>>>> requests, and in the client we measure the time that the roundtrip
>>>> takes. Apart from that, on the server side, we measure as well the
>>>> time that our REST resource consumes, i.e. the method call time. In
>>>> summary, we have some huge differences between CXF and Jersey - in
>>>> favor of Jersey being much faster - in the cases where we test
>>>> against non-existing resources. In all other cases, the numbers we
>>>> got were really very close for the two REST frameworks.
>>>>
>>>> But when testing against non-existing REST paths, the roundtrip time
>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>> we are not using CXF in its proper configuration. Could you please
>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>> so that we get the best out of it.
>>>>
>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>> seconds, so we wonder where is that half second spent in ? In our
>>>> application logic, in the case when the path is not correct, we throw
>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>> this exception is getting caught in CXF, and heavily processed
>>>> further down the stack.
>>>>
>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>> processing to try to map the URI to some existing REST resource, I
>>>> don't know ... maybe performance here could depend on the particular
>>>> JSON framework that is being used. ...
>>>>
>>>
>>> I suspect it could be the default WebApplicationExceptionMapper which is
>>> to blame.
>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>> any other regular response, but I'm not sure it would explain the
>>> difference.
>>>
>>> Can you give me a favor please and register a custom
>>> WebApplicationException mapper which would only return:
>>>
>>> Response.status(exception.getStatus()).build() ?
>>>
>>> Have a look please at the default mapper's code:
>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>
>>>
>>> As you can see, we are trying to go to some length there to figure out
>>> what and how to report a given exception and I reckon it all adds up to
>>> the response time.
>>>
>>> Please try eliminating the default mapper and let us know the results
>>>
>>> Thanks, Sergey
>>>
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>>
>>>
>>
>>
>
>

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi Krum,
On 20/02/12 16:47, Bakalsky, Krum wrote:
> Hi,
>
> Yes, thanks for the comments, they sound very logical to explaining the numbers from all the variants we tested.
> Didn't know that the default exception mapper was developed as a result of a user demand as well.
>
> For the record: how could I disable it ? I am registering the Simple mapper within the returned set of the Application#getClasses() method call - so maybe I should remove it there, i.e. from the super.getClasses() list, or it is in a different manner ?
>
> Yes, I think that a servlet context parameter would be the most flexible and elegant solution, for configuring the desired behavior for non-200/exceptional cases.
>
I've spent some today on optimizing a couple of things,
see the updated default WebApplicationException mapper:

http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java

I think it's more effective now, yes still it is configurable and able 
to interact with CXF FaultLoggers and optionally fine-log the exception 
message.

I was thinking yesterday of adding a property to get this default mapper 
ignored. I'm not sure now it's worth going this route.

One way to 'disable' it is to offer a custom WebApplicationException 
mapper, something which we've explored. Another one - is to add this 
would be property but I've realized that it's not that good for CXF 
after all:  this would require the CXF servlet to process unmapped 
WebApplicationExceptions to be compliant and this would effectively mean 
repeating at the servlet level what is done at the CXF chain level but 
with limitations, ex, the Response produced by the mapper would still be 
routed through the out filters and indeed the optional CXF out faulty 
interceptors which can be handy. Say we can have an out filter which 
time-stamps all the responses - this would be ignored at the servlet 
level and would effectively have to be duplicated if needed...

The other thing is that CXFNonSpringJaxrsServlet and servlet filters 
will not be necessarily installed in some OSGI containers by default.

So it appears to be the best way is to optimize the default mapper which 
I hope I did, and it would be always possible to offer custom mappers 
instead on per-Application basis.

Let me know please if you have any concerns with the above,

In meantime I also updated some code on the in chain to optimize the way 
the HTTP path to match is calculated - it may well save us few 
milliseconds :-)

If you can share some more results from your tests which point to few 
unoptimized code spots then it would be good; we are going to review few 
code paths anyway,

Thanks, Sergey

>
> Kindest Regards and many thanks again,
> Krum.
>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Monday, February 20, 2012 5:49 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi,
> On 20/02/12 15:28, Sergey Beryozkin wrote:
>> Hi Krum
>> On 20/02/12 15:05, Bakalsky, Krum wrote:
>>> Hi Sergey,
>>>
>>> Wow! I tested your idea and it produced dramatically better results!
>>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>>> investigation, and I believe that our observation is really true that
>>> the overhead is spent in the error handling code. Just out of
>>> curiosity: with the same SimpleApplicationMapper:
>>>
>> Thanks for the confirmation, so we've identified the spot where the
>> problem is :-)
>>
>>> public class SimpleApplicationExceptionMapper implements
>>> ExceptionMapper<WebApplicationException>  {
>>>
>>> @Override
>>> public Response toResponse(WebApplicationException exception) {
>>> return Response.status(404).build();
>>> }
>>>
>>> }
>>>
>>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>>
>>>
>>> However, without this class, Jersey performed even faster - most of
>>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>>> the CXF case ... you already know it :) - 0.500 seconds
>>>
>
> Forgot to comment to this one... I think the fact that both frameworks
> show approximately the same time with SimpleApplicationExceptionMapper
> in place and that Jersey scales really well without it is that Jersey
> basically either writes  to HTTP out stream or propagates the exception
> up to the ServletException filter immediately if no custom mapper is
> available.
> In case of CXF the immediate propagation will also happen if no custom
> mapper is available. The reason the default WebApplicationException
> Mapper was installed was to do with the fact that few users were not
> quite happy with having to write their custom mappers in order to report
> differently to the exceptions generated at the runtime level or some of
> the providers' level or indeed having to write a filter catching the
> escaped exceptions there...
>
> What I might need to do additionally is to introduce a contextual
> property that will tell the runtime to ignore even the default
> WebApplicationException mapper, for example, when the user wishes to
> get all the exceptions propagated to the filter level, to simplify the
> performance data comparison, etc
>
> Cheers, Sergey
>
>>>
>>> Thank a lot for your idea and suggestion! It definitely shed light on
>>> where the discrepancies come from.
>>>
>> I've opened
>> https://issues.apache.org/jira/browse/CXF-4121
>>
>> I suspect now that it is the specific lines which get the basic warning
>> message from the resource bundle are main 'culprits', I guess I will
>> just have an internal final static String - will need to experiment a
>> bit...It is good even the default providers can be customized/replaced,
>> but I'll look into trying to make the default exception mapper
>> performing a bit better :-)
>>
>> Thanks, Sergey
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>> -----Original Message-----
>>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>>> Sent: Friday, February 17, 2012 6:14 PM
>>> To: users@cxf.apache.org
>>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>>
>>> Hi Krum,
>>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>>> Hi to all Apache CXF users!,
>>>>
>>>> I would kindly like to bring to your attention a particular problem
>>>> that we are currently being stuck into. We are experimenting with a
>>>> simple web application, which exposes its functionality via REST. We
>>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>>> REST frameworks.
>>>>
>>>> For calculating our performance measurements, we simply execute REST
>>>> requests, and in the client we measure the time that the roundtrip
>>>> takes. Apart from that, on the server side, we measure as well the
>>>> time that our REST resource consumes, i.e. the method call time. In
>>>> summary, we have some huge differences between CXF and Jersey - in
>>>> favor of Jersey being much faster - in the cases where we test
>>>> against non-existing resources. In all other cases, the numbers we
>>>> got were really very close for the two REST frameworks.
>>>>
>>>> But when testing against non-existing REST paths, the roundtrip time
>>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>>> we are not using CXF in its proper configuration. Could you please
>>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>>> so that we get the best out of it.
>>>>
>>>> The server side time, in the case of CXF, is rather small - 0.001
>>>> seconds, so we wonder where is that half second spent in ? In our
>>>> application logic, in the case when the path is not correct, we throw
>>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>>> this exception is getting caught in CXF, and heavily processed
>>>> further down the stack.
>>>>
>>>> Are you aware of such kind of issues, with CXF handling the 404
>>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>>> processing to try to map the URI to some existing REST resource, I
>>>> don't know ... maybe performance here could depend on the particular
>>>> JSON framework that is being used. ...
>>>>
>>>
>>> I suspect it could be the default WebApplicationExceptionMapper which is
>>> to blame.
>>> In CXF, a given Exception, once wrapped in a Response, is treated like
>>> any other regular response, but I'm not sure it would explain the
>>> difference.
>>>
>>> Can you give me a favor please and register a custom
>>> WebApplicationException mapper which would only return:
>>>
>>> Response.status(exception.getStatus()).build() ?
>>>
>>> Have a look please at the default mapper's code:
>>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>>
>>>
>>> As you can see, we are trying to go to some length there to figure out
>>> what and how to report a given exception and I reckon it all adds up to
>>> the response time.
>>>
>>> Please try eliminating the default mapper and let us know the results
>>>
>>> Thanks, Sergey
>>>
>>>>
>>>> Kindest Regards,
>>>> Krum.
>>>>
>>>>
>>>
>>
>>
>
>

RE: REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Hi,

Yes, thanks for the comments, they sound very logical to explaining the numbers from all the variants we tested.
Didn't know that the default exception mapper was developed as a result of a user demand as well.

For the record: how could I disable it ? I am registering the Simple mapper within the returned set of the Application#getClasses() method call - so maybe I should remove it there, i.e. from the super.getClasses() list, or it is in a different manner ?

Yes, I think that a servlet context parameter would be the most flexible and elegant solution, for configuring the desired behavior for non-200/exceptional cases.


Kindest Regards and many thanks again,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:sberyozkin@gmail.com] 
Sent: Monday, February 20, 2012 5:49 PM
To: users@cxf.apache.org
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi,
On 20/02/12 15:28, Sergey Beryozkin wrote:
> Hi Krum
> On 20/02/12 15:05, Bakalsky, Krum wrote:
>> Hi Sergey,
>>
>> Wow! I tested your idea and it produced dramatically better results!
>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>> investigation, and I believe that our observation is really true that
>> the overhead is spent in the error handling code. Just out of
>> curiosity: with the same SimpleApplicationMapper:
>>
> Thanks for the confirmation, so we've identified the spot where the
> problem is :-)
>
>> public class SimpleApplicationExceptionMapper implements
>> ExceptionMapper<WebApplicationException> {
>>
>> @Override
>> public Response toResponse(WebApplicationException exception) {
>> return Response.status(404).build();
>> }
>>
>> }
>>
>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>
>>
>> However, without this class, Jersey performed even faster - most of
>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>> the CXF case ... you already know it :) - 0.500 seconds
>>

Forgot to comment to this one... I think the fact that both frameworks 
show approximately the same time with SimpleApplicationExceptionMapper 
in place and that Jersey scales really well without it is that Jersey 
basically either writes  to HTTP out stream or propagates the exception 
up to the ServletException filter immediately if no custom mapper is 
available.
In case of CXF the immediate propagation will also happen if no custom 
mapper is available. The reason the default WebApplicationException 
Mapper was installed was to do with the fact that few users were not 
quite happy with having to write their custom mappers in order to report 
differently to the exceptions generated at the runtime level or some of 
the providers' level or indeed having to write a filter catching the 
escaped exceptions there...

What I might need to do additionally is to introduce a contextual 
property that will tell the runtime to ignore even the default 
WebApplicationException mapper, for example, when the user wishes to 
get all the exceptions propagated to the filter level, to simplify the 
performance data comparison, etc

Cheers, Sergey

>>
>> Thank a lot for your idea and suggestion! It definitely shed light on
>> where the discrepancies come from.
>>
> I've opened
> https://issues.apache.org/jira/browse/CXF-4121
>
> I suspect now that it is the specific lines which get the basic warning
> message from the resource bundle are main 'culprits', I guess I will
> just have an internal final static String - will need to experiment a
> bit...It is good even the default providers can be customized/replaced,
> but I'll look into trying to make the default exception mapper
> performing a bit better :-)
>
> Thanks, Sergey
>>
>> Kindest Regards,
>> Krum.
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Friday, February 17, 2012 6:14 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi Krum,
>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>> Hi to all Apache CXF users!,
>>>
>>> I would kindly like to bring to your attention a particular problem
>>> that we are currently being stuck into. We are experimenting with a
>>> simple web application, which exposes its functionality via REST. We
>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>> REST frameworks.
>>>
>>> For calculating our performance measurements, we simply execute REST
>>> requests, and in the client we measure the time that the roundtrip
>>> takes. Apart from that, on the server side, we measure as well the
>>> time that our REST resource consumes, i.e. the method call time. In
>>> summary, we have some huge differences between CXF and Jersey - in
>>> favor of Jersey being much faster - in the cases where we test
>>> against non-existing resources. In all other cases, the numbers we
>>> got were really very close for the two REST frameworks.
>>>
>>> But when testing against non-existing REST paths, the roundtrip time
>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>> we are not using CXF in its proper configuration. Could you please
>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>> so that we get the best out of it.
>>>
>>> The server side time, in the case of CXF, is rather small - 0.001
>>> seconds, so we wonder where is that half second spent in ? In our
>>> application logic, in the case when the path is not correct, we throw
>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>> this exception is getting caught in CXF, and heavily processed
>>> further down the stack.
>>>
>>> Are you aware of such kind of issues, with CXF handling the 404
>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>> processing to try to map the URI to some existing REST resource, I
>>> don't know ... maybe performance here could depend on the particular
>>> JSON framework that is being used. ...
>>>
>>
>> I suspect it could be the default WebApplicationExceptionMapper which is
>> to blame.
>> In CXF, a given Exception, once wrapped in a Response, is treated like
>> any other regular response, but I'm not sure it would explain the
>> difference.
>>
>> Can you give me a favor please and register a custom
>> WebApplicationException mapper which would only return:
>>
>> Response.status(exception.getStatus()).build() ?
>>
>> Have a look please at the default mapper's code:
>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>
>>
>> As you can see, we are trying to go to some length there to figure out
>> what and how to report a given exception and I reckon it all adds up to
>> the response time.
>>
>> Please try eliminating the default mapper and let us know the results
>>
>> Thanks, Sergey
>>
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>>
>>
>
>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi,
On 20/02/12 15:28, Sergey Beryozkin wrote:
> Hi Krum
> On 20/02/12 15:05, Bakalsky, Krum wrote:
>> Hi Sergey,
>>
>> Wow! I tested your idea and it produced dramatically better results!
>> The roundtrip time for 404 requests decreased by a factor of 10 (from
>> 0.500 seconds down to 0.030 seconds). That is a huge success for our
>> investigation, and I believe that our observation is really true that
>> the overhead is spent in the error handling code. Just out of
>> curiosity: with the same SimpleApplicationMapper:
>>
> Thanks for the confirmation, so we've identified the spot where the
> problem is :-)
>
>> public class SimpleApplicationExceptionMapper implements
>> ExceptionMapper<WebApplicationException> {
>>
>> @Override
>> public Response toResponse(WebApplicationException exception) {
>> return Response.status(404).build();
>> }
>>
>> }
>>
>> Jersey behaved very similarly to CXF in terms of roundtrip time.
>>
>>
>> However, without this class, Jersey performed even faster - most of
>> the times were 10 times faster, i.e. around 0.003 seconds, while in
>> the CXF case ... you already know it :) - 0.500 seconds
>>

Forgot to comment to this one... I think the fact that both frameworks 
show approximately the same time with SimpleApplicationExceptionMapper 
in place and that Jersey scales really well without it is that Jersey 
basically either writes  to HTTP out stream or propagates the exception 
up to the ServletException filter immediately if no custom mapper is 
available.
In case of CXF the immediate propagation will also happen if no custom 
mapper is available. The reason the default WebApplicationException 
Mapper was installed was to do with the fact that few users were not 
quite happy with having to write their custom mappers in order to report 
differently to the exceptions generated at the runtime level or some of 
the providers' level or indeed having to write a filter catching the 
escaped exceptions there...

What I might need to do additionally is to introduce a contextual 
property that will tell the runtime to ignore even the default 
WebApplicationException mapper, for example, when the user wishes to 
get all the exceptions propagated to the filter level, to simplify the 
performance data comparison, etc

Cheers, Sergey

>>
>> Thank a lot for your idea and suggestion! It definitely shed light on
>> where the discrepancies come from.
>>
> I've opened
> https://issues.apache.org/jira/browse/CXF-4121
>
> I suspect now that it is the specific lines which get the basic warning
> message from the resource bundle are main 'culprits', I guess I will
> just have an internal final static String - will need to experiment a
> bit...It is good even the default providers can be customized/replaced,
> but I'll look into trying to make the default exception mapper
> performing a bit better :-)
>
> Thanks, Sergey
>>
>> Kindest Regards,
>> Krum.
>>
>> -----Original Message-----
>> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
>> Sent: Friday, February 17, 2012 6:14 PM
>> To: users@cxf.apache.org
>> Subject: Re: REST performance discrepancies between CXF and Jersey
>>
>> Hi Krum,
>> On 17/02/12 15:26, Bakalsky, Krum wrote:
>>> Hi to all Apache CXF users!,
>>>
>>> I would kindly like to bring to your attention a particular problem
>>> that we are currently being stuck into. We are experimenting with a
>>> simple web application, which exposes its functionality via REST. We
>>> are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application
>>> on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as
>>> REST frameworks.
>>>
>>> For calculating our performance measurements, we simply execute REST
>>> requests, and in the client we measure the time that the roundtrip
>>> takes. Apart from that, on the server side, we measure as well the
>>> time that our REST resource consumes, i.e. the method call time. In
>>> summary, we have some huge differences between CXF and Jersey - in
>>> favor of Jersey being much faster - in the cases where we test
>>> against non-existing resources. In all other cases, the numbers we
>>> got were really very close for the two REST frameworks.
>>>
>>> But when testing against non-existing REST paths, the roundtrip time
>>> in CXF case is approximately 0.5 seconds, while in the case of Jersey
>>> it is 0.01 seconds. We think that this is quite strange, and probably
>>> we are not using CXF in its proper configuration. Could you please
>>> give us some hints ? Maybe we have to tune and tweak it a little bit,
>>> so that we get the best out of it.
>>>
>>> The server side time, in the case of CXF, is rather small - 0.001
>>> seconds, so we wonder where is that half second spent in ? In our
>>> application logic, in the case when the path is not correct, we throw
>>> javax.ws.rs.WebApplicationException and with a debugger we saw that
>>> this exception is getting caught in CXF, and heavily processed
>>> further down the stack.
>>>
>>> Are you aware of such kind of issues, with CXF handling the 404
>>> situation in a very heavyweight manner ? Maybe it does some thorough
>>> processing to try to map the URI to some existing REST resource, I
>>> don't know ... maybe performance here could depend on the particular
>>> JSON framework that is being used. ...
>>>
>>
>> I suspect it could be the default WebApplicationExceptionMapper which is
>> to blame.
>> In CXF, a given Exception, once wrapped in a Response, is treated like
>> any other regular response, but I'm not sure it would explain the
>> difference.
>>
>> Can you give me a favor please and register a custom
>> WebApplicationException mapper which would only return:
>>
>> Response.status(exception.getStatus()).build() ?
>>
>> Have a look please at the default mapper's code:
>> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>>
>>
>> As you can see, we are trying to go to some length there to figure out
>> what and how to report a given exception and I reckon it all adds up to
>> the response time.
>>
>> Please try eliminating the default mapper and let us know the results
>>
>> Thanks, Sergey
>>
>>>
>>> Kindest Regards,
>>> Krum.
>>>
>>>
>>
>
>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi Krum
On 20/02/12 15:05, Bakalsky, Krum wrote:
> Hi Sergey,
>
> Wow! I tested your idea and it produced dramatically better results! The roundtrip time for 404 requests decreased by a factor of 10 (from 0.500 seconds down to 0.030 seconds). That is a huge success for our investigation, and I believe that our observation is really true that the overhead is spent in the error handling code. Just out of curiosity: with the same SimpleApplicationMapper:
>
Thanks for the confirmation, so we've identified the spot where the 
problem is :-)

> public class SimpleApplicationExceptionMapper implements ExceptionMapper<WebApplicationException>  {
>
> 	@Override
> 	public Response toResponse(WebApplicationException exception) {
> 		return Response.status(404).build();
> 	}
>
> }
>
> Jersey behaved very similarly to CXF in terms of roundtrip time.
>
>
> However, without this class, Jersey performed even faster - most of the times were 10 times faster, i.e. around 0.003 seconds, while in the CXF case ... you already know it :) - 0.500 seconds
>
>
> Thank a lot for your idea and suggestion! It definitely shed light on where the discrepancies come from.
>
I've opened
https://issues.apache.org/jira/browse/CXF-4121

I suspect now that it is the specific lines which get the basic warning 
message from the resource bundle are main 'culprits', I guess I will 
just have an internal final static String - will need to experiment a 
bit...It is good even the default providers can be customized/replaced, 
but I'll look into trying to make the default exception mapper 
performing a bit better :-)

Thanks, Sergey
>
> Kindest Regards,
> Krum.
>
> -----Original Message-----
> From: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
> Sent: Friday, February 17, 2012 6:14 PM
> To: users@cxf.apache.org
> Subject: Re: REST performance discrepancies between CXF and Jersey
>
> Hi Krum,
> On 17/02/12 15:26, Bakalsky, Krum wrote:
>> Hi to all Apache CXF users!,
>>
>> I would kindly like to bring to your attention a particular problem that we are currently being stuck into. We are experimenting with a simple web application, which exposes its functionality via REST. We are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as REST frameworks.
>>
>> For calculating our performance measurements, we simply execute REST requests, and in the client we measure the time that the roundtrip takes. Apart from that, on the server side, we measure as well the time that our REST resource consumes, i.e. the method call time. In summary, we have some huge differences between CXF and Jersey - in favor of Jersey being much faster - in the cases where we test against non-existing resources. In all other cases, the numbers we got were really very close for the two REST frameworks.
>>
>> But when testing against non-existing REST paths, the roundtrip time in CXF case is approximately 0.5 seconds, while in the case of Jersey it is 0.01 seconds. We think that this is quite strange, and probably we are not using CXF in its proper configuration. Could you please give us some hints ? Maybe we have to tune and tweak it a little bit, so that we get the best out of it.
>>
>> The server side time, in the case of CXF, is rather small - 0.001 seconds, so we wonder where is that half second spent in ? In our application logic, in the case when the path is not correct, we throw javax.ws.rs.WebApplicationException and with a debugger we saw that this exception is getting caught in CXF, and heavily processed further down the stack.
>>
>> Are you aware of such kind of issues, with CXF handling the 404 situation in a very heavyweight manner ? Maybe it does some thorough processing to try to map the URI to some existing REST resource, I don't know ... maybe performance here could depend on the particular JSON framework that is being used. ...
>>
>
> I suspect it could be the default WebApplicationExceptionMapper which is
> to blame.
> In CXF, a given Exception, once wrapped in a Response, is treated like
> any other regular response, but I'm not sure it would explain the
> difference.
>
> Can you give me a favor please and register a custom
> WebApplicationException mapper which would only return:
>
> Response.status(exception.getStatus()).build() ?
>
> Have a look please at the default mapper's code:
> http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java
>
> As you can see, we are trying to go to some length there to figure out
> what and how to report a given exception and I reckon it all adds up to
> the response time.
>
> Please try eliminating the default mapper and let us know the results
>
> Thanks, Sergey
>
>>
>> Kindest Regards,
>> Krum.
>>
>>
>


-- 
Sergey Beryozkin

Talend Community Coders
http://coders.talend.com/

Blog: http://sberyozkin.blogspot.com

RE: REST performance discrepancies between CXF and Jersey

["Bakalsky, Krum" <kr...@sap.com>]

Hi Sergey,

Wow! I tested your idea and it produced dramatically better results! The roundtrip time for 404 requests decreased by a factor of 10 (from 0.500 seconds down to 0.030 seconds). That is a huge success for our investigation, and I believe that our observation is really true that the overhead is spent in the error handling code. Just out of curiosity: with the same SimpleApplicationMapper:

public class SimpleApplicationExceptionMapper implements ExceptionMapper<WebApplicationException> {

	@Override
	public Response toResponse(WebApplicationException exception) {
		return Response.status(404).build();
	}

}

Jersey behaved very similarly to CXF in terms of roundtrip time.


However, without this class, Jersey performed even faster - most of the times were 10 times faster, i.e. around 0.003 seconds, while in the CXF case ... you already know it :) - 0.500 seconds


Thank a lot for your idea and suggestion! It definitely shed light on where the discrepancies come from.


Kindest Regards,
Krum.

-----Original Message-----
From: Sergey Beryozkin [mailto:sberyozkin@gmail.com] 
Sent: Friday, February 17, 2012 6:14 PM
To: users@cxf.apache.org
Subject: Re: REST performance discrepancies between CXF and Jersey

Hi Krum,
On 17/02/12 15:26, Bakalsky, Krum wrote:
> Hi to all Apache CXF users!,
>
> I would kindly like to bring to your attention a particular problem that we are currently being stuck into. We are experimenting with a simple web application, which exposes its functionality via REST. We are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as REST frameworks.
>
> For calculating our performance measurements, we simply execute REST requests, and in the client we measure the time that the roundtrip takes. Apart from that, on the server side, we measure as well the time that our REST resource consumes, i.e. the method call time. In summary, we have some huge differences between CXF and Jersey - in favor of Jersey being much faster - in the cases where we test against non-existing resources. In all other cases, the numbers we got were really very close for the two REST frameworks.
>
> But when testing against non-existing REST paths, the roundtrip time in CXF case is approximately 0.5 seconds, while in the case of Jersey it is 0.01 seconds. We think that this is quite strange, and probably we are not using CXF in its proper configuration. Could you please give us some hints ? Maybe we have to tune and tweak it a little bit, so that we get the best out of it.
>
> The server side time, in the case of CXF, is rather small - 0.001 seconds, so we wonder where is that half second spent in ? In our application logic, in the case when the path is not correct, we throw javax.ws.rs.WebApplicationException and with a debugger we saw that this exception is getting caught in CXF, and heavily processed further down the stack.
>
> Are you aware of such kind of issues, with CXF handling the 404 situation in a very heavyweight manner ? Maybe it does some thorough processing to try to map the URI to some existing REST resource, I don't know ... maybe performance here could depend on the particular JSON framework that is being used. ...
>

I suspect it could be the default WebApplicationExceptionMapper which is 
to blame.
In CXF, a given Exception, once wrapped in a Response, is treated like 
any other regular response, but I'm not sure it would explain the 
difference.

Can you give me a favor please and register a custom 
WebApplicationException mapper which would only return:

Response.status(exception.getStatus()).build() ?

Have a look please at the default mapper's code:
http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java

As you can see, we are trying to go to some length there to figure out 
what and how to report a given exception and I reckon it all adds up to 
the response time.

Please try eliminating the default mapper and let us know the results

Thanks, Sergey

>
> Kindest Regards,
> Krum.
>
>

Re: REST performance discrepancies between CXF and Jersey

[Sergey Beryozkin <sb...@gmail.com>]

Hi Krum,
On 17/02/12 15:26, Bakalsky, Krum wrote:
> Hi to all Apache CXF users!,
>
> I would kindly like to bring to your attention a particular problem that we are currently being stuck into. We are experimenting with a simple web application, which exposes its functionality via REST. We are using JAX-RS/JSON, no SOAP, no JAX-WS. We deploy our application on top of Tomcat 7, and test both against CXF 2.5.2 and Jersey as REST frameworks.
>
> For calculating our performance measurements, we simply execute REST requests, and in the client we measure the time that the roundtrip takes. Apart from that, on the server side, we measure as well the time that our REST resource consumes, i.e. the method call time. In summary, we have some huge differences between CXF and Jersey - in favor of Jersey being much faster - in the cases where we test against non-existing resources. In all other cases, the numbers we got were really very close for the two REST frameworks.
>
> But when testing against non-existing REST paths, the roundtrip time in CXF case is approximately 0.5 seconds, while in the case of Jersey it is 0.01 seconds. We think that this is quite strange, and probably we are not using CXF in its proper configuration. Could you please give us some hints ? Maybe we have to tune and tweak it a little bit, so that we get the best out of it.
>
> The server side time, in the case of CXF, is rather small - 0.001 seconds, so we wonder where is that half second spent in ? In our application logic, in the case when the path is not correct, we throw javax.ws.rs.WebApplicationException and with a debugger we saw that this exception is getting caught in CXF, and heavily processed further down the stack.
>
> Are you aware of such kind of issues, with CXF handling the 404 situation in a very heavyweight manner ? Maybe it does some thorough processing to try to map the URI to some existing REST resource, I don't know ... maybe performance here could depend on the particular JSON framework that is being used. ...
>

I suspect it could be the default WebApplicationExceptionMapper which is 
to blame.
In CXF, a given Exception, once wrapped in a Response, is treated like 
any other regular response, but I'm not sure it would explain the 
difference.

Can you give me a favor please and register a custom 
WebApplicationException mapper which would only return:

Response.status(exception.getStatus()).build() ?

Have a look please at the default mapper's code:
http://svn.apache.org/repos/asf/cxf/trunk/rt/frontend/jaxrs/src/main/java/org/apache/cxf/jaxrs/impl/WebApplicationExceptionMapper.java

As you can see, we are trying to go to some length there to figure out 
what and how to report a given exception and I reckon it all adds up to 
the response time.

Please try eliminating the default mapper and let us know the results

Thanks, Sergey

>
> Kindest Regards,
> Krum.
>
>