You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by jaybytez <ja...@yahoo.com> on 2012/11/30 19:48:56 UTC
WebClient multithreading within a DefaultMessageListenerContainer
We have a service (DefaultMessageListenerContainer) that listens to messages
and when it processes one, we make an HTTP Post to a third party that
requires us to create a stateful session for our interaction. In the
DefaultMessageListenerContainer we have the min/max Consumers set to 5/10.
So when we send in about 20+ messages, we have what appears to be a
threading issues. We use the WebClient to:
1) Setup maintain session
2) Call login
3) Receive a cookie
4) Call method
5) Parse response
What winds up happening is that it appears like another thread comes in and
wiping out the login/Cookie of the previous thread before the previous
thread calls the method it is supposed to, so our 3rd party service assumes
we haven't logged in yet.
Here is our code:
public ResponseModel postSampleModel(RequestModel requestModel) {
WebClient webClient = null;
ResponseModel responseModel = null;
try {
//Create webClient in code with threadsafe set to true
webClient = WebClient.create(httpsUrl, jaxbProviders, true);
WebClient.getConfig(webClient).getRequestContext()
.put(org.apache.cxf.message.Message.MAINTAIN_SESSION,
Boolean.TRUE);
//create login xml and send request
RequestModel loginRequest = createLogin();
postRequest(webClient, loginRequest);
//send original after the login
Response response = postRequest(webClient, requestModel);
InputStream inputStream = (InputStream) response.getEntity();
//return the unmarshalled response
responseModel = ResponseModelUtil.unmarshal(ResponseModel.class,
inputStream);
} finally {
if (webClient != null) {
webClient.reset();
}
}
return responseModel;
}
private Response postRequest(WebClient webClient, RequestModel
requestModel) {
String xml = RequestModelUtil.marshal(RequestModel.class, requestModel);
Response response = webClient.post(xml);
if (response.getStatus() != 200) {
LOGGER.error("Request to fastrieve was unsuccessful");
}
return response;
}
So when something like this sits in a DefaultMessageListenerContainer, if a
few instances are running at a time, will the WebClient produce a unique
cookie for each request running in parallel? What other reason could be
causing this potential threading issue?
Thanks...jay
--
View this message in context: http://cxf.547215.n5.nabble.com/WebClient-multithreading-within-a-DefaultMessageListenerContainer-tp5719525.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: WebClient multithreading within a DefaultMessageListenerContainer
Posted by Sergey Beryozkin <sb...@gmail.com>.
On 05/12/12 17:03, jaybytez wrote:
> So to add a little detail to this, we have the following:
>
> One War
> 1) DefaultMessageListenerContainer -> SubscribeService -> External Service
> Wrapper -> External Service
>
> So our SubscribeService is the business logic of pulling messages off the
> queue. Our External Service Wrapper is a service that wraps our third party
> call because it is XML/HTTP, this is a local instance of a class. Within
> the local instance of the External Service Wrapper we make a WebClient call
> with information from External Service. We have problems with the
> Thread/Cookie Mgmt when done this way.
>
> Interesting thing is when we split the code into two wars and do the
> following:
>
> Two War
> 1) DefaultMessageListenerContainer -> SubscribeService (Web Service Call to
> other WAR) ->
> 2) External Service Wrapper -> External Service
>
> When the External Service Wrapper lives in it's own war, managing it's own
> threads, and the SubscribeService calls it remotely (as opposed to using it
> as a local instance), we have no Thread/Cookie Mgmt issues with WebClient.
interesting
>
> So maybe the question really is how the DefaultMessageListenerContainer
> min/max threads effects the threading of WebClient (stateful invocation).
>
I'd still say that given you create WebClient on the stack it does not
matter how it is being called. I'm assuming that HttpConduit is not
doing its own cookie management now, right ? (I'm not sure yet how it is
supposed to work in the multithreaded case - but lets review this
particular issue later)
Please check the logs - that can help shedding some light on the cause
of this issue
Cheers, Sergey
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/WebClient-multithreading-within-a-DefaultMessageListenerContainer-tp5719525p5719752.html
> Sent from the cxf-user mailing list archive at Nabble.com.
Re: WebClient multithreading within a
DefaultMessageListenerContainer
Posted by jaybytez <ja...@yahoo.com>.
So to add a little detail to this, we have the following:
One War
1) DefaultMessageListenerContainer -> SubscribeService -> External Service
Wrapper -> External Service
So our SubscribeService is the business logic of pulling messages off the
queue. Our External Service Wrapper is a service that wraps our third party
call because it is XML/HTTP, this is a local instance of a class. Within
the local instance of the External Service Wrapper we make a WebClient call
with information from External Service. We have problems with the
Thread/Cookie Mgmt when done this way.
Interesting thing is when we split the code into two wars and do the
following:
Two War
1) DefaultMessageListenerContainer -> SubscribeService (Web Service Call to
other WAR) ->
2) External Service Wrapper -> External Service
When the External Service Wrapper lives in it's own war, managing it's own
threads, and the SubscribeService calls it remotely (as opposed to using it
as a local instance), we have no Thread/Cookie Mgmt issues with WebClient.
So maybe the question really is how the DefaultMessageListenerContainer
min/max threads effects the threading of WebClient (stateful invocation).
--
View this message in context: http://cxf.547215.n5.nabble.com/WebClient-multithreading-within-a-DefaultMessageListenerContainer-tp5719525p5719752.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: WebClient multithreading within a DefaultMessageListenerContainer
Posted by Sergey Beryozkin <sb...@gmail.com>.
On 05/12/12 16:26, jaybytez wrote:
> So we modified our code and did the following logic:
>
> Object cookie = loginResponse.getMetadata().getFirst("Set-Cookie");
> webClient.header("Cookie", cookie);
>
> So now we get our response from our original login (3rd party invocation),
> we get the response and retrieve the first "Set-Cookie", which gives us a
> String containing our two cookies.
>
> We then use the header method to add the cookie back to the WebClient
> instance before we make our second invocation.
>
> This seems to work with a single request, but when we start to send 11
> requests in (as an example), we get our problem again where some requests
> succeed, and some requests seem to lose the cookie because our 3rd party
> service says we tried the second invocation service without logging in. So
> unless I set the cookie incorrectly, this didn't work for us yet.
>
I honestly do not think that the client created on the stack can have
any thread-related issues.
Can you please configure both loginResponse & webClient instances with
LoggingFeature ? In the older CXF it can be done by using
JAXRSClientfactoryBean and WebClient factory constructor in the latest
CXF, or add LoggingInInterceptor & LoggingOutInterceptor, and start
increasing the number of requests 1 by one till you see in the logs that
the in or out cookies being lost.
Let us know please what you find...
> The totally insane part of this is we are doing two parallel paths, one
> solution using WebClient and one with Apache's HttpClient. WebClient so far
> is tripping over itself in a multi-threaded fashion, but Apache's HttpClient
> is not (it works with the multiple requests). But HttpClient fails on the
> hostname verification (because WLS has a problem with wildcarded certs), yet
> the WebClient's hostname verification works within WLS. So we are trying to
> get both WebClient to work by working with the Cookies, and HttpClient to
> work by determining where WLS is invoking it's own classpath and ignoring
> Apache's AllowAllHostnameVerifier.
You can use Apache client now with the async support in CXF 2.7.0 -
Sergey
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/WebClient-multithreading-within-a-DefaultMessageListenerContainer-tp5719525p5719745.html
> Sent from the cxf-user mailing list archive at Nabble.com.
--
Sergey Beryozkin
Talend Community Coders
http://coders.talend.com/
Blog: http://sberyozkin.blogspot.com
Re: WebClient multithreading within a
DefaultMessageListenerContainer
Posted by jaybytez <ja...@yahoo.com>.
So we modified our code and did the following logic:
Object cookie = loginResponse.getMetadata().getFirst("Set-Cookie");
webClient.header("Cookie", cookie);
So now we get our response from our original login (3rd party invocation),
we get the response and retrieve the first "Set-Cookie", which gives us a
String containing our two cookies.
We then use the header method to add the cookie back to the WebClient
instance before we make our second invocation.
This seems to work with a single request, but when we start to send 11
requests in (as an example), we get our problem again where some requests
succeed, and some requests seem to lose the cookie because our 3rd party
service says we tried the second invocation service without logging in. So
unless I set the cookie incorrectly, this didn't work for us yet.
The totally insane part of this is we are doing two parallel paths, one
solution using WebClient and one with Apache's HttpClient. WebClient so far
is tripping over itself in a multi-threaded fashion, but Apache's HttpClient
is not (it works with the multiple requests). But HttpClient fails on the
hostname verification (because WLS has a problem with wildcarded certs), yet
the WebClient's hostname verification works within WLS. So we are trying to
get both WebClient to work by working with the Cookies, and HttpClient to
work by determining where WLS is invoking it's own classpath and ignoring
Apache's AllowAllHostnameVerifier.
--
View this message in context: http://cxf.547215.n5.nabble.com/WebClient-multithreading-within-a-DefaultMessageListenerContainer-tp5719525p5719745.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: WebClient multithreading within a DefaultMessageListenerContainer
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi
On 30/11/12 18:48, jaybytez wrote:
> We have a service (DefaultMessageListenerContainer) that listens to messages
> and when it processes one, we make an HTTP Post to a third party that
> requires us to create a stateful session for our interaction. In the
> DefaultMessageListenerContainer we have the min/max Consumers set to 5/10.
> So when we send in about 20+ messages, we have what appears to be a
> threading issues. We use the WebClient to:
>
> 1) Setup maintain session
> 2) Call login
> 3) Receive a cookie
> 4) Call method
> 5) Parse response
>
> What winds up happening is that it appears like another thread comes in and
> wiping out the login/Cookie of the previous thread before the previous
> thread calls the method it is supposed to, so our 3rd party service assumes
> we haven't logged in yet.
>
> Here is our code:
>
> public ResponseModel postSampleModel(RequestModel requestModel) {
> WebClient webClient = null;
>
> ResponseModel responseModel = null;
> try {
> //Create webClient in code with threadsafe set to true
> webClient = WebClient.create(httpsUrl, jaxbProviders, true);
> WebClient.getConfig(webClient).getRequestContext()
> .put(org.apache.cxf.message.Message.MAINTAIN_SESSION,
> Boolean.TRUE);
>
> //create login xml and send request
> RequestModel loginRequest = createLogin();
> postRequest(webClient, loginRequest);
>
> //send original after the login
> Response response = postRequest(webClient, requestModel);
> InputStream inputStream = (InputStream) response.getEntity();
>
> //return the unmarshalled response
> responseModel = ResponseModelUtil.unmarshal(ResponseModel.class,
> inputStream);
> } finally {
> if (webClient != null) {
> webClient.reset();
> }
> }
> return responseModel;
> }
>
> private Response postRequest(WebClient webClient, RequestModel
> requestModel) {
>
> String xml = RequestModelUtil.marshal(RequestModel.class, requestModel);
>
> Response response = webClient.post(xml);
>
> if (response.getStatus() != 200) {
> LOGGER.error("Request to fastrieve was unsuccessful");
> }
> return response;
>
> }
In the above case WebClient is completely thread-safe even without
setting a thread-safe flag because you create it on the current frame...
>
> So when something like this sits in a DefaultMessageListenerContainer, if a
> few instances are running at a time, will the WebClient produce a unique
> cookie for each request running in parallel? What other reason could be
> causing this potential threading issue?
I'm not sure right now how
org.apache.cxf.message.Message.MAINTAIN_SESSION is implemented, will
need to check the code...
Can you experiment with a different approach please, remove a thread
safe flag and work with cookies at the WebClient level, example, after a
call, do webClient.getResponse().getMetadata().getFirst("Set-Cookie"),
and then submit it back with "Cookie".
And if possible, create a simple test project which can help us
investigate what might be going wrong with the current approach
Let us how it goes please
Thanks, Sergey
>
> Thanks...jay
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/WebClient-multithreading-within-a-DefaultMessageListenerContainer-tp5719525.html
> Sent from the cxf-user mailing list archive at Nabble.com.
--
Sergey Beryozkin
Talend Community Coders
http://coders.talend.com/
Blog: http://sberyozkin.blogspot.com